The Power of Unpublished Research: You Be the Reviewer

We want our students to learn to read critically and to interrogate and evaluate what they read. Does the author have the right data? Do the conclusions actually follow from the data? Are other explanations missing from the argument? That’s what we want them to ask themselves. A quick look at students’ notes from reading – if they even took any – reveal a totally different set of information, usually focused on the literature review and sometimes the theory. After all, this is the main textually-based body of an empirical paper, so it’s easiest for them to read.

Beyond steps we can take to teach students to read articles effectively (see my previous post on R&U and the Article Sort activity), I like to engage my intermediate and upper-level courses in an activity we call “You Be the Reviewer.” Students in all of my classes have already done the R&U activity and read (briefly) about the process papers go through to get published. So at some point in the term, I assign an unpublished article manuscript – often from a colleague or a conference paper pulled from the conference archives with author permission – and ask students to write a journal-style review, including a decision of whether the item should be published.

As support for this assignment, I distribute a handout like the one available here. It suggests some questions for students to consider, reminds them to check R&U for more guidance, and gives them a framework for writing a review. Typically, they are asked to post their reviews to the course learning management site and to bring a hard copy to class for reference. The resulting conversations have been far more in-depth and wide-ranging than anything else I’ve tried. At the end of the discussion, we collectively decide on the disposition of the article. Several classes – including a freshman-level intro course – have voted to reject manuscripts, though, as in the real world, R&Rs are the most common response.

While this activity obviously works better with upper-division classes, even lower-level students have enjoyed it and given very piercing feedback. For lower-level classes, qualitative research or very simple quantitative analysis works best. I normally compile the students’ feedback (copying particularly relevant bits from the CMS and pasting into a document) and send it to the author as thanks for sharing the manuscript. In an undergraduate methods class, I once was able to have the author come and give a (previously prepared) conference style presentation to the class on the manuscript they had reviewed. The author also took questions, so that the class had a model presentation to use in preparing their own as well as a chance to ask the author about research design decisions and practice giving useful feedback on research-in-progress before their own peer review process.

I’ve found that using a manuscript – an honest to goodness pre-publication, looks-like-it-was-written-in-Word-then-PDF’d manuscript – gets a far better reaction than published research. Students are reluctant to question or challenge work by ‘experts’ that’s already been vetted and published, but papers are a different matter.

Have you used unpublished research (other than your own) with your students? What was their reaction?

Being a Research Consumer: The Article Sort

Becoming a competent consumer of political science scholarship is almost always an objective of my courses, especially general education courses intended to expose students to the social scientific way of thinking. To support this objective, a long ways back I wrote a document called “Reading and Understanding Political Science,” which is an undergraduate’s guide to types of scholarship in political science, the parts of an empirical article, and questions to ask oneself while reading quantitative, qualitative, and formal modeling publications. We typically read this for the second day of class, when most are still struggling to obtain textbooks in this new order-by-mail world. After a brief review of the typology and parts, we engage in The Great Article Sort.

To begin, we brainstorm a list of key words and other ways to tell what type of article an item is. Then I pair students off, have them introduce themselves, and distribute 2-3 articles from a pile that I’ve prepared to each pair. Their task is to classify as many articles as they can in 5-8 minutes; extras (and ones that pairs have finished) go in a stack up front for recirculation. The pair with the most correct classifications at the end gets 2 bonus points, so they make two copies of their findings – one to turn in at the conclusion of the sort period, and one to keep for discussion. At the conclusion of the work period, I collect a copy from each group and we review their responses as a class – both what they decided and how they knew. The whole activity, including debrief, takes about 20-25 minutes, depending on how many items they want to discuss.

Preparation for this activity took about 45 minutes and consisted mostly of using JSTOR and the internet to access publications where I knew I could find articles of various types (literature reviews, empirical, op-eds, modeling and other theoretical pieces, etc.) across the various subfields of political science. For longer items, usually I printed only the first 4 pages; printing two pages to a sheet and both sides of the paper meant that they’re still only one piece of paper in the stack. Sometimes I was able to reuse items I had in my personal collection that I no longer needed (e.g., spare copies from something distributed in a previous term). I had about 25 items labeled with letters, and usually two copies of each so that we had enough to go around. This wasn’t enough for a 35-person class. If I were prepping this activity again, I’d aim for 40 items and number them, and be very selective in the debrief discussion.

Thinking Comparatively

Last time, I posted about ways to get students involved in making hypotheses, forming expectations, and testing them with instructor-provided data. Today’s activity leverages technology in a different way, by allowing students to collaborate on data generation and encouraging comparative thinking as a means of drawing conclusions.

I begin by creating and sharing a Google spreadsheet (1) with several questions across the top of columns, and several cases listed one to a row. Then, pairs or small groups of students are assigned one or two cells of the resulting table. For example, my Intro to American Politics class did comparative civil rights struggles, comparing African Americans, women, Hispanic Americans, Asian Americans, and the LGBT+ community as cases, and motivations/starting conditions, core strategies, de jure discrimination, and de facto discrimination as questions. One group of students might have motivations and strategies for African Americans, and another group had de jure and de facto discrimination for LGBT+ Americans.

Continue reading “Thinking Comparatively”

And the Data Say…

Hi, and thanks for reading! Chad has graciously invited me to share some activities with y’all focused on bringing research methods ideas into the substantive classroom. The goal of these activities is twofold. First, programs are increasingly adopting objectives (and even school-wide QEPs) about research literacy and data or numeric literacy. The activities I’ll be sharing support those goals.

Second, students have the best chance of success in the research methods classroom when Introduction to Research Methods is not actually their introduction to research methods. One of the reasons students have anxiety and less-than-optimal experiences in their Research Methods courses is the sheer quantity of new material. If we can decrease the novelty by even a small amount, we can improve their experiences in that often-required Methods class. Through these activities, I hope to show you some ways to incorporate social scientific thinking and methods into even introductory courses in ways that require relatively low amounts of additional preparation or even class time.

Students often ask questions about general trends or patterns. Do they usually do that? Is that what normally happens? With a little nudging, you can turn this curiosity into a data-based activity. When you reach a topic where data is readily available, consider a class activity that asks students to hypothesize about relationships between variables or indicators, then test those hypotheses on the spot using a quick graph. For example, my Intro to American Politics textbook spent a bit of time showing graphs of how partisanship has varied over time in Congress and how the current period of hyperpartisanship is different, and how partisanship affects the ability of Congress to get things done. Continue reading “And the Data Say…”