Undergraduates Doing Replication: Strategies for Successful Replication Exercises (Part 3 of 3)

In my last two posts (here and here), I’ve talked through the rationale for undergraduates doing replication and shared a replication assignment of my own. In the final segment of this series, I want to talk a little bit about strategies for developing your own successful replication assignment.

First, start with a highly readable article. Alas, this means that most of what is in the APSR is out. Both Perspectives on Politics and PS: Political Science and Politics have appropriate empirical articles, though, that are typically shorter, more accessible, and less technically sophisticated. The better students can understand the article, the better their chances of success on the assignment. Even if you don’t usually give reading guides, I’d consider doing one (just reading comprehension questions for students to review while reading) for this assignment to help ensure everyone starts off on the right foot with a good understanding of the reading.

Second, talk it up in class. Emphasize that research gets published using only the skills the students currently have, and we’re going to show that to ourselves by replicating published research. This is something they should be proud to be able to do; it’s an achievement.

Third, consider allowing students to work with partners or in trios, even if they turn in separate written work. Working together will give them – especially the women – more confidence about their ability to do the tasks, and that will reduce both the stress on them and the number of anxious questions you’ll get.

Finally, give yourself plenty of time to write the assignment. Be detailed and specific. It will also take you some time to get and clean the data, and possibly write a sketchy codebook in the instructions, so that the assignment is plug-and-play ready when it goes out to students. You’ll want to drop most of the unused variables (especially if there are fixed effects dummies you aren’t using) and similar clutter like that.  There’s a reason I’m posting this blog entry now: writing one of these would be an excellent summer project, a good activity for when you’re stalled on your research or just need to change mental gears for a bit. This will take a few hours, but like all good problem sets, once you’ve written it you can reuse it repeatedly.

Undergraduates Doing Replication: Replication Assignments in Action Part 2 of 3

Last week, I talked about the value of replication exercises for undergraduates and why they might be even greater than for graduate students. The opportunity to combine research and analysis skills with writing skills in a single assignment is almost too good to pass up since it kills several birds (or at least, typical course objectives) with one stone. Today I’ll briefly discuss a replication activity I wrote, and some strategies to help you make your own replication assignments successful.

The assignment I’m sharing today comes from Linda Camp Keith’s “The United Nations International Covenant on Civil and Political Rights: Does It Make a Difference in Human Rights Behavior?” (Journal of Peace Research 36,1 (1999): 95-118). This was a unit project, summing up their studies in bivariate analysis. The replication assignment file can be obtained here. We began as any good researcher would, by getting to know our key dependent and independent variables. We then attempt to replicate her published bivariate results, which is one of the reasons I used this article here. I conclude with question 5, which previews the concepts of control that we’re moving into as they work on this assignment. Finally, in a brief follow-up assignment, students actually add the controls and replicate parts of the multivariate findings. Continue reading

Undergraduates Doing Replication? Why Not!?

When I taught Quantiative Methods last spring, a colleague picked up a paper of mine from the printer and came looking for me. “You’re doing replications with the undergrads?” she asked. “Why?” I looked at her and without thinking simply replied, “Why not?” Replicating studies is considered a best practice of sorts in graduate level methods training. None of the reasons given there – teaching disciplinary norms, emphasizing the importance of transparency in research, etc. – fails to hold in the undergraduate context. If anything, our undergraduates have more need of those objectives than our grad students, who will have them repeatedly reinforced across multiple classes. For most of us who teach undergrads, one methods class is all we’ve got, and we need to make it count.

For most of us, part of the objective of a research methods course is to introduce students to the ways of thinking and doing that characterize social science, and social science research especially. It’s a key point of socialization into the discipline, where they go from being students of politics to being students of political science. Continue reading

Distance Teaching + 1-on-1 = …. Active Learning?

Recently I’ve begun doing some dissertation coaching while in a gap between positions. It’s all done remotely using a free videoconferencing service (more below). But here I am, videocalling with students I’ve never met, trying to step into an ongoing project and guide them out of whatever mess they (think they) are in. It’s an unusual form of teaching, but so far I like it a lot.

The experience has me thinking about what active learning really LOOKS like. Obviously, no simulations are going on here; no games either, or case studies, or other typical discussion materials. So how do you DO active learning when you’re one-on-one, geographically remote, and dealing primarily with the writing process?  Continue reading

Make Your Librarian Love You

I’ve often said that librarians are the most under-utilized resource of any college or university. At one of the schools I’ve taught at, they actually went begging to faculty to be invited in to do things with students. At most of the others, they frequently advertise their services to faculty, hoping that some of us will take them up on their offer.

The usual use of librarians for a research methods course is in teaching students how to find materials for a literature review using library databases. That’s a pretty standard need. But for methods classes that also incorporate qualitative methods, I’d like to suggest a second use for your librarians: teaching a hands-on class on primary source interpretation using materials from the school’s special collections or school archives. Continue reading

Where Did Your Stuff Come From?

Most American students are challenged to understand the extent to which international trade affects their lives, and the way that the US trades with the world. I can (and have) shown statistics about trade and economics in very graphic and immediate form, but numbers in the scale of trillions are hard to conceptualize.

To combat that, I asked students in an introductory international politics class to go on a scavenger hunt. They were tasked to find one item from each of 5 world regions – Europe, Latin America, Sub-Saharan Africa, Middle East & North Africa, and Asia & the Pacific. They had to take a picture of the ‘made in’ indicator (and part of their student ID, to ensure that they didn’t just go grabbing stock photography or Instagram stuff) and post it to the class learning management system’s discussion board. To sweeten the pot, I offered 2 bonus points for unique entries, where no one else posted something from that country. Specialty foods and beverages were excluded (no taking a picture of a bottle of Stella for Belgium).

Students went crazy hunting for stuff. The two bonus points were apparently a huge incentive, with students finding and posting additional items when someone else duplicated “their” country.

Continue reading

How Do We Know What We Know? Active Learning and the Scientific Method

Positivist epistemology can be the topic of a very boring lecture in an introductory social science or methods course. Fortunately, an activity borrowed from an elementary math class can alleviate the boredom by asking students to identify the sources of their own beliefs about unobservable phenomena and to explore questions about what constitutes an explanation.

The basic outline of this activity, which I use during the second class session of most courses I teach, is that unusual objects are concealed in opaque fabric bags. Small groups of students must attempt to describe and identify the object as thoroughly as possible without opening the bag. A recorder observes the group’s discussion and notes what data points they establish, how they negotiate rival hypotheses, and what threshold or pieces of evidence convinced group members about their findings. The process of investigation, discovery, and persuasion is repeated with several different bags, so that each group gets at least 3 objects. We then discuss what they learned about the objects, and as I reveal each item I link it to concepts in positivist epistemology such as what constitutes sufficient evidence, whether we ever can know for sure without being able to observe things, the role of context in defining the meaning of an observed act, and precision and accuracy in measurement.

This activity has been around for a while; I published it in PS: Political Science and Politics in 2006. Since then, it’s been picked up in history, linguistics, and psychology classes. The article goes through sources, ideas, and materials in more detail than I can here. The objects you use can vary, but I strongly recommend that you keep the two bags with film canisters or other containers (one with cotton balls inside, one without), and the bag with a piece of fabric ribbon. These objects allow you to get at issues of precision, accuracy, unobservability and authority most efficiently.

My own set of objects has evolved, and it shifts from course to course depending on the points I want to emphasize. My methods course, for example, emphasizes the difficulty of doing measurement across time and space. To raise this point, one of the bags includes the cake topper from my wedding – a simple engraved acrylic block intended as an executive award or paperweight. Most groups identify that it’s plastic and engraved and probably a paperweight… but without knowing what the engraving says, its context is completely unknown and the object’s significance is vastly misinterpreted. Out of its original context, our identification – coding – of that object was totally incorrect.

I’ve been using this activity for over a decade now and haven’t made any major improvements or changes to the version described in the PS paper. I strongly encourage you to give it a try: let your students’ own curiosity drive their understanding of how researching unobservables works.