Undergraduates Doing Replication: Replication Assignments in Action Part 2 of 3

Last week, I talked about the value of replication exercises for undergraduates and why they might be even greater than for graduate students. The opportunity to combine research and analysis skills with writing skills in a single assignment is almost too good to pass up since it kills several birds (or at least, typical course objectives) with one stone. Today I’ll briefly discuss a replication activity I wrote, and some strategies to help you make your own replication assignments successful.

The assignment I’m sharing today comes from Linda Camp Keith’s “The United Nations International Covenant on Civil and Political Rights: Does It Make a Difference in Human Rights Behavior?” (Journal of Peace Research 36,1 (1999): 95-118). This was a unit project, summing up their studies in bivariate analysis. The replication assignment file can be obtained here. We began as any good researcher would, by getting to know our key dependent and independent variables. We then attempt to replicate her published bivariate results, which is one of the reasons I used this article here. I conclude with question 5, which previews the concepts of control that we’re moving into as they work on this assignment. Finally, in a brief follow-up assignment, students actually add the controls and replicate parts of the multivariate findings. Continue reading

Undergraduates Doing Replication? Why Not!?

When I taught Quantiative Methods last spring, a colleague picked up a paper of mine from the printer and came looking for me. “You’re doing replications with the undergrads?” she asked. “Why?” I looked at her and without thinking simply replied, “Why not?” Replicating studies is considered a best practice of sorts in graduate level methods training. None of the reasons given there – teaching disciplinary norms, emphasizing the importance of transparency in research, etc. – fails to hold in the undergraduate context. If anything, our undergraduates have more need of those objectives than our grad students, who will have them repeatedly reinforced across multiple classes. For most of us who teach undergrads, one methods class is all we’ve got, and we need to make it count.

For most of us, part of the objective of a research methods course is to introduce students to the ways of thinking and doing that characterize social science, and social science research especially. It’s a key point of socialization into the discipline, where they go from being students of politics to being students of political science. Continue reading

Make Your Librarian Love You

I’ve often said that librarians are the most under-utilized resource of any college or university. At one of the schools I’ve taught at, they actually went begging to faculty to be invited in to do things with students. At most of the others, they frequently advertise their services to faculty, hoping that some of us will take them up on their offer.

The usual use of librarians for a research methods course is in teaching students how to find materials for a literature review using library databases. That’s a pretty standard need. But for methods classes that also incorporate qualitative methods, I’d like to suggest a second use for your librarians: teaching a hands-on class on primary source interpretation using materials from the school’s special collections or school archives. Continue reading

A Plea for Poster Sessions

Many research methods classes end with student presentations of their research papers. In the typical format, this requires faculty to clear three days of class time for students and faculty alike to sit there and be bored by badly organized and poorly designed ‘presentations’ of research. Unless students are given significant guidance on what to include and how to organize it, their talks usually omit important elements of the paper. Instructors resent the loss of class time, students resent either the loss of instructional time or the obligation to sit there and appear attentive when they don’t care, and no one really gains much from the whole situation except, perhaps, a small bit of public speaking experience.

I’d like to make a plea instead for poster sessions instead of presentations. In a typical 50- or 75-minute class,  you can run two mini sessions where half the class presents and half is the audience. This costs you one period of instructional time – perhaps two – instead of three or four for standard presentations. Poster sessions are all about audience involvement. Instead of talking about their research once, students will speak about it informally and repeatedly for 20 minutes and respond to more questions from their peers as audience clusters come and go. They learn to speak succinctly and clearly about just the highlights of their research while still having to respond to questions about the details that students glean from the poster. With the aid of a simple poster review sheet, the audience members will engage more with their peers’ research and think more critically about it because their active involvement is a crucial part of poster sessions.

In short, poster sessions are significantly more active than traditional panel-style presentations, and they have clear benefits for presenters, audience members, and faculty alike. Guidance for students (and faculty) on what to put on a poster, and how to convert your traditional powerpoint presentation to a poster, is in chapter 11 of my Empirical Research and Writing: A Political Science Student’s Practical Guide. (Or request a review copy from CQ Press.) It also contains some basic suggestions for how to organize a poster session in a regular classroom without nice big bulletin boards.

How Do We Know What We Know? Active Learning and the Scientific Method

Positivist epistemology can be the topic of a very boring lecture in an introductory social science or methods course. Fortunately, an activity borrowed from an elementary math class can alleviate the boredom by asking students to identify the sources of their own beliefs about unobservable phenomena and to explore questions about what constitutes an explanation.

The basic outline of this activity, which I use during the second class session of most courses I teach, is that unusual objects are concealed in opaque fabric bags. Small groups of students must attempt to describe and identify the object as thoroughly as possible without opening the bag. A recorder observes the group’s discussion and notes what data points they establish, how they negotiate rival hypotheses, and what threshold or pieces of evidence convinced group members about their findings. The process of investigation, discovery, and persuasion is repeated with several different bags, so that each group gets at least 3 objects. We then discuss what they learned about the objects, and as I reveal each item I link it to concepts in positivist epistemology such as what constitutes sufficient evidence, whether we ever can know for sure without being able to observe things, the role of context in defining the meaning of an observed act, and precision and accuracy in measurement.

This activity has been around for a while; I published it in PS: Political Science and Politics in 2006. Since then, it’s been picked up in history, linguistics, and psychology classes. The article goes through sources, ideas, and materials in more detail than I can here. The objects you use can vary, but I strongly recommend that you keep the two bags with film canisters or other containers (one with cotton balls inside, one without), and the bag with a piece of fabric ribbon. These objects allow you to get at issues of precision, accuracy, unobservability and authority most efficiently.

My own set of objects has evolved, and it shifts from course to course depending on the points I want to emphasize. My methods course, for example, emphasizes the difficulty of doing measurement across time and space. To raise this point, one of the bags includes the cake topper from my wedding – a simple engraved acrylic block intended as an executive award or paperweight. Most groups identify that it’s plastic and engraved and probably a paperweight… but without knowing what the engraving says, its context is completely unknown and the object’s significance is vastly misinterpreted. Out of its original context, our identification – coding – of that object was totally incorrect.

I’ve been using this activity for over a decade now and haven’t made any major improvements or changes to the version described in the PS paper. I strongly encourage you to give it a try: let your students’ own curiosity drive their understanding of how researching unobservables works.

Specifications Grading #5: End of Term Report and Reflections

Well, my first experience with specifications grading is almost over, and with the semester drawing to a close, it’s time to reflect on the experiment. Find my first entry on specs grading here, and previous entries on this experiment can be found here, here, here, ad here. But now, here are my top 5 take-aways and lessons learned from specifications grading:

 

#5: Specifications Grading is more work up front, but much less at the end and moving forward

Continue reading