Undergraduates Doing Replication: Replication Assignments in Action Part 2 of 3

Last week, I talked about the value of replication exercises for undergraduates and why they might be even greater than for graduate students. The opportunity to combine research and analysis skills with writing skills in a single assignment is almost too good to pass up since it kills several birds (or at least, typical course objectives) with one stone. Today I’ll briefly discuss a replication activity I wrote, and some strategies to help you make your own replication assignments successful.

The assignment I’m sharing today comes from Linda Camp Keith’s “The United Nations International Covenant on Civil and Political Rights: Does It Make a Difference in Human Rights Behavior?” (Journal of Peace Research 36,1 (1999): 95-118). This was a unit project, summing up their studies in bivariate analysis. The replication assignment file can be obtained here. We began as any good researcher would, by getting to know our key dependent and independent variables. We then attempt to replicate her published bivariate results, which is one of the reasons I used this article here. I conclude with question 5, which previews the concepts of control that we’re moving into as they work on this assignment. Finally, in a brief follow-up assignment, students actually add the controls and replicate parts of the multivariate findings. Continue reading

Undergraduates Doing Replication? Why Not!?

When I taught Quantiative Methods last spring, a colleague picked up a paper of mine from the printer and came looking for me. “You’re doing replications with the undergrads?” she asked. “Why?” I looked at her and without thinking simply replied, “Why not?” Replicating studies is considered a best practice of sorts in graduate level methods training. None of the reasons given there – teaching disciplinary norms, emphasizing the importance of transparency in research, etc. – fails to hold in the undergraduate context. If anything, our undergraduates have more need of those objectives than our grad students, who will have them repeatedly reinforced across multiple classes. For most of us who teach undergrads, one methods class is all we’ve got, and we need to make it count.

For most of us, part of the objective of a research methods course is to introduce students to the ways of thinking and doing that characterize social science, and social science research especially. It’s a key point of socialization into the discipline, where they go from being students of politics to being students of political science. Continue reading

My Students Don’t Read: Responses to a Classic Classroom Problem

All experienced instructors have had this happen to them: You assign an interesting reading that is pivotal to a topic on the syllabus. You emphasize to the students how important it is that they complete that particular reading, as it will be the basis of the next class session’s discussion. Walking into class, you smile, anticipating a smart, informed discussion on a fascinating topic, and ask a basic question to get things going. And then, the silence, and the signs: the blank stares, the eyes that won’t meet yours, the walls and shoes and notebooks that suddenly are the most interesting things in the room. Your smile drops as you realize the horrible truth: none of the students did the reading.

Quickly you realize it’s not entirely true: a small handful of students, the ones you can always rely on, tentatively raise their hands. Others may have skimmed the reading, or tried to do it just as class started. Still others pull it out as you ask the question, trying to do in 30 seconds what they need a concentrated 10 or 30 minutes to do. Despite this, the vast majority of the class simply did not do as instructed.

What’s the dedicated instructor to do?

I have been teaching for more than ten years, and this happened to me twice this semester alone. In one case, only one student out in my intro to IR class had read Thucydides’ short Melian Dialogue that IR teachers the world over use as an introduction to Realism—even though they had weekly reading quizzes on the material. In my intro to American politics course, none of them had read Federalist Paper #84, which outlines the arguments regarding the inclusion of a bill of rights in the Constitution. In the moment when I realized that my students were not prepared for the reading-based discussion I had planned, I had a decision to make: how would I respond to their lack of preparation?

A few options immediately came to mind.

Continue reading

Pedagogical Defense: Avoiding Soul Crushing Writing Assignments

Recently I’ve been working on decoupling/narrowing what I expect in my writing assignments. For those of us who teach 70+ students at a shot and do not have TAs , the prospect of grading their papers is not only daunting…it is SOUL CRUSHING.

Even if their work is well-intentioned with good editing and citation, most undergraduate student work is still under development in nearly every area: structure, readability, sophistication of hypotheses, strength of argument, etc etc etc…

In prior courses I’ve laid out complex rubrics with several categories, points, and lots of very specific feedback. The net result was not only that I hated reading blah papers, but now I had tons of blah feedback to provide which tended to overwhelm and demoralize my students more than help.

This semester I’m trying a different tack with my first-year students: Two developmental criteria per paper ONLY, plus an invitation for creativity. The first criteria is to advance the some aspect of their writing’s quality of thought, the second, to advance one aspect of formatting, the third is to save my soul.

Example: My most recent assignment is an early attempt at synthesizing and discussing the work of more than one author. (Preparatory work for eventual literature reviews) PLUS…and remember this part…I don’t want to have my soul crushed trying to read them all. Note the areas where I’m trying to stop them from killing my soul.

Author Synthesis Assignment (see what I did there?)
Cocktail Party Script: (Soul Crush Avoidance Technique)

Imagine you’re at a cocktail party with three prominent scholars who have published research related to your question. (**Questions and sources were developed and vetted these in a prior class.) Write a script that details the conversation you would have with these authors.

Content: Your script must include…
1. Your question and why it is important
2. Each author’s research and insights and how they pertain to your question. NOTE: Accuracy and specificity get higher grades, vagueness and misinterpretation get lower values.
(Writing Development Emphasis)
3. Potential disagreements and agreements between each member in the party—including yourself.
4. Humor or Drama of some kind. (Soul Crush Avoidance Technique)

Formatting: Your script must focus on …
1. Careful attention to citation frequency, format, and accuracy. (choose any style you like but be consistent) (**Format Development Emphasis)

Dazzle me with your concision and creativity! No more than 6 pages. Focus on citation and accuracy. If you’re all freaked out about margins and font size you’re missing the point. 

I’ll post results next week. Wish me LUCK!

Where Did Your Stuff Come From?

Most American students are challenged to understand the extent to which international trade affects their lives, and the way that the US trades with the world. I can (and have) shown statistics about trade and economics in very graphic and immediate form, but numbers in the scale of trillions are hard to conceptualize.

To combat that, I asked students in an introductory international politics class to go on a scavenger hunt. They were tasked to find one item from each of 5 world regions – Europe, Latin America, Sub-Saharan Africa, Middle East & North Africa, and Asia & the Pacific. They had to take a picture of the ‘made in’ indicator (and part of their student ID, to ensure that they didn’t just go grabbing stock photography or Instagram stuff) and post it to the class learning management system’s discussion board. To sweeten the pot, I offered 2 bonus points for unique entries, where no one else posted something from that country. Specialty foods and beverages were excluded (no taking a picture of a bottle of Stella for Belgium).

Students went crazy hunting for stuff. The two bonus points were apparently a huge incentive, with students finding and posting additional items when someone else duplicated “their” country.

Continue reading

The Power of Unpublished Research: You Be the Reviewer

We want our students to learn to read critically and to interrogate and evaluate what they read. Does the author have the right data? Do the conclusions actually follow from the data? Are other explanations missing from the argument? That’s what we want them to ask themselves. A quick look at students’ notes from reading – if they even took any – reveal a totally different set of information, usually focused on the literature review and sometimes the theory. After all, this is the main textually-based body of an empirical paper, so it’s easiest for them to read.

Beyond steps we can take to teach students to read articles effectively (see my previous post on R&U and the Article Sort activity), I like to engage my intermediate and upper-level courses in an activity we call “You Be the Reviewer.” Students in all of my classes have already done the R&U activity and read (briefly) about the process papers go through to get published. So at some point in the term, I assign an unpublished article manuscript – often from a colleague or a conference paper pulled from the conference archives with author permission – and ask students to write a journal-style review, including a decision of whether the item should be published.

As support for this assignment, I distribute a handout like the one available here. It suggests some questions for students to consider, reminds them to check R&U for more guidance, and gives them a framework for writing a review. Typically, they are asked to post their reviews to the course learning management site and to bring a hard copy to class for reference. The resulting conversations have been far more in-depth and wide-ranging than anything else I’ve tried. At the end of the discussion, we collectively decide on the disposition of the article. Several classes – including a freshman-level intro course – have voted to reject manuscripts, though, as in the real world, R&Rs are the most common response.

While this activity obviously works better with upper-division classes, even lower-level students have enjoyed it and given very piercing feedback. For lower-level classes, qualitative research or very simple quantitative analysis works best. I normally compile the students’ feedback (copying particularly relevant bits from the CMS and pasting into a document) and send it to the author as thanks for sharing the manuscript. In an undergraduate methods class, I once was able to have the author come and give a (previously prepared) conference style presentation to the class on the manuscript they had reviewed. The author also took questions, so that the class had a model presentation to use in preparing their own as well as a chance to ask the author about research design decisions and practice giving useful feedback on research-in-progress before their own peer review process.

I’ve found that using a manuscript – an honest to goodness pre-publication, looks-like-it-was-written-in-Word-then-PDF’d manuscript – gets a far better reaction than published research. Students are reluctant to question or challenge work by ‘experts’ that’s already been vetted and published, but papers are a different matter.

Have you used unpublished research (other than your own) with your students? What was their reaction?

Binary Failure

kitten-fightAn example of the kind of assignment design that I mentioned in my last post about feature creep:

I have assigned three iterations of policy memos in this year’s iteration of my first-year seminar. Directions for the first memo are here. The memos are intended to function as authentic writing exercises — each has a specified purpose, audience, and format. The authenticity is supposed serve as a vehicle for stimulating students’ interest in the topic.

The memos require a small amount of creative problem-solving. First, each student chooses a policy recommendation that he or she prefers. Limited choice is always good because it generates mental investment in the outcome. Second, each student selects from information that I’ve provided to create a rationale for the policy recommendation, but this has to be done within the constraints of the memo’s format. There is a puzzle to solve. Continue reading