Unconventional Films in the Classroom: Into the (Methodological) Woods

Just after the (last) New Year, I emailed my spring Research Methods sections with the usual ‘class starts soon, here’s the syllabus, order your books’ message.  This year’s message contained a strange assignment, due during the second week of class. They were to watch the recent film version of Stephen Sondheim’s Into the Woods and write a one page reaction to the questions, whose fault is it, and why?

For those unfamiliar with the film, Into the Woods is a complex tale braiding together half a dozen well-known fairy tales around the story of the Baker and his Wife, who remain childless thanks to a curse put on the Baker’s house by the Witch. The Baker and his Wife go into the woods to seek the ingredients to lift the curse. The rest of the village, meanwhile, is driven into the same woods by the arrival of a Giant (Jack’s fault, of course). While in the woods, all sorts of calamities befall the group. By intermission, things seem to be mostly repaired so that happily ever after is plausible, but in the second act, thanks to a continuation of behaviors from before – and the realization by some characters that fulfilled wishes aren’t always what you dreamed of – a second giant drives the group into the woods once again. Disasters of various sorts again ensue. But whose fault is it?

Midway through the second act, the characters confront the realities of determining causation in the real world. As the Baker says to Jack, “It’s because of you that there’s a giant in our midst and my wife is dead.” At that point, the fun begins from a methodological and pedagogical perspective. Are those two separate outcomes, or one? Is the causal chain from Jack to those event(s) equally strong? What constitutes a good explanation? The Baker’s Father was the ultimate trigger of the curse, which was placed on them by the Witch’s Mother – is it one of their faults? Neither of those characters even appears in the show – is that convincing? For that matter, what is “it”?? We have to begin by determining the actual dependent variable of interest: the phenomenon we’re explaining. Because of the nature of this particular show’s plot, all the plots are intertwined; isolating a single simple causal story is impossible. (Hello, equifinality and multiple causality!) The Baker’s wife walked off a cliff (in the more recent version), so it’s technically her own fault that she’s dead – or did the shaking of the earthquake cause her to lose her balance and fall? How do you attribute blame convincingly to an earthquake?

Aside from a bit of grumbling at an assignment so early, and the unfortunate guy who watched it in the frat house without knowing that “it was all singing!!,” this activity was generally well received. Forcing students to pick a side in their reaction papers gave us a starting point for discussion as I tallied blame on the board, then went into some of the justifications. Overall, the highly unorthodox use of film in a research methods class, and the even more unorthodox choice of films, was a very good way to start my admittedly unorthodox approach to a required but dreaded methods class.

The Power of Unpublished Research: You Be the Reviewer

We want our students to learn to read critically and to interrogate and evaluate what they read. Does the author have the right data? Do the conclusions actually follow from the data? Are other explanations missing from the argument? That’s what we want them to ask themselves. A quick look at students’ notes from reading – if they even took any – reveal a totally different set of information, usually focused on the literature review and sometimes the theory. After all, this is the main textually-based body of an empirical paper, so it’s easiest for them to read.

Beyond steps we can take to teach students to read articles effectively (see my previous post on R&U and the Article Sort activity), I like to engage my intermediate and upper-level courses in an activity we call “You Be the Reviewer.” Students in all of my classes have already done the R&U activity and read (briefly) about the process papers go through to get published. So at some point in the term, I assign an unpublished article manuscript – often from a colleague or a conference paper pulled from the conference archives with author permission – and ask students to write a journal-style review, including a decision of whether the item should be published.

As support for this assignment, I distribute a handout like the one available here. It suggests some questions for students to consider, reminds them to check R&U for more guidance, and gives them a framework for writing a review. Typically, they are asked to post their reviews to the course learning management site and to bring a hard copy to class for reference. The resulting conversations have been far more in-depth and wide-ranging than anything else I’ve tried. At the end of the discussion, we collectively decide on the disposition of the article. Several classes – including a freshman-level intro course – have voted to reject manuscripts, though, as in the real world, R&Rs are the most common response.

While this activity obviously works better with upper-division classes, even lower-level students have enjoyed it and given very piercing feedback. For lower-level classes, qualitative research or very simple quantitative analysis works best. I normally compile the students’ feedback (copying particularly relevant bits from the CMS and pasting into a document) and send it to the author as thanks for sharing the manuscript. In an undergraduate methods class, I once was able to have the author come and give a (previously prepared) conference style presentation to the class on the manuscript they had reviewed. The author also took questions, so that the class had a model presentation to use in preparing their own as well as a chance to ask the author about research design decisions and practice giving useful feedback on research-in-progress before their own peer review process.

I’ve found that using a manuscript – an honest to goodness pre-publication, looks-like-it-was-written-in-Word-then-PDF’d manuscript – gets a far better reaction than published research. Students are reluctant to question or challenge work by ‘experts’ that’s already been vetted and published, but papers are a different matter.

Have you used unpublished research (other than your own) with your students? What was their reaction?

Being a Research Consumer: The Article Sort

Becoming a competent consumer of political science scholarship is almost always an objective of my courses, especially general education courses intended to expose students to the social scientific way of thinking. To support this objective, a long ways back I wrote a document called “Reading and Understanding Political Science,” which is an undergraduate’s guide to types of scholarship in political science, the parts of an empirical article, and questions to ask oneself while reading quantitative, qualitative, and formal modeling publications. We typically read this for the second day of class, when most are still struggling to obtain textbooks in this new order-by-mail world. After a brief review of the typology and parts, we engage in The Great Article Sort.

To begin, we brainstorm a list of key words and other ways to tell what type of article an item is. Then I pair students off, have them introduce themselves, and distribute 2-3 articles from a pile that I’ve prepared to each pair. Their task is to classify as many articles as they can in 5-8 minutes; extras (and ones that pairs have finished) go in a stack up front for recirculation. The pair with the most correct classifications at the end gets 2 bonus points, so they make two copies of their findings – one to turn in at the conclusion of the sort period, and one to keep for discussion. At the conclusion of the work period, I collect a copy from each group and we review their responses as a class – both what they decided and how they knew. The whole activity, including debrief, takes about 20-25 minutes, depending on how many items they want to discuss.

Preparation for this activity took about 45 minutes and consisted mostly of using JSTOR and the internet to access publications where I knew I could find articles of various types (literature reviews, empirical, op-eds, modeling and other theoretical pieces, etc.) across the various subfields of political science. For longer items, usually I printed only the first 4 pages; printing two pages to a sheet and both sides of the paper meant that they’re still only one piece of paper in the stack. Sometimes I was able to reuse items I had in my personal collection that I no longer needed (e.g., spare copies from something distributed in a previous term). I had about 25 items labeled with letters, and usually two copies of each so that we had enough to go around. This wasn’t enough for a 35-person class. If I were prepping this activity again, I’d aim for 40 items and number them, and be very selective in the debrief discussion.

Thinking Comparatively

Last time, I posted about ways to get students involved in making hypotheses, forming expectations, and testing them with instructor-provided data. Today’s activity leverages technology in a different way, by allowing students to collaborate on data generation and encouraging comparative thinking as a means of drawing conclusions.

I begin by creating and sharing a Google spreadsheet (1) with several questions across the top of columns, and several cases listed one to a row. Then, pairs or small groups of students are assigned one or two cells of the resulting table. For example, my Intro to American Politics class did comparative civil rights struggles, comparing African Americans, women, Hispanic Americans, Asian Americans, and the LGBT+ community as cases, and motivations/starting conditions, core strategies, de jure discrimination, and de facto discrimination as questions. One group of students might have motivations and strategies for African Americans, and another group had de jure and de facto discrimination for LGBT+ Americans.

Continue reading

Specs Grading Update #4: On poor syllabus design, late work, and the woes of grading

It’s time for another specifications grading update! These posts are my attempt to be highly transparent about the benefits and challenges of using this method in my research methods class this semester, with all the trials and tribulations and the reflection and changes they prompt here on display.  Check out Posts 1, 2, and 3 if you haven’t already, or for a deep dive into the ins-and-outs of specifications grading as a system, take a look at our extensive discussions on the subject over the last year.

Today’s topics: requiring so much that I set up my students to fail; dealing with late work; and how all that grading is going. In other words, let’s talk about how even extensive reflection and consideration can result in basic syllabus mistakes that pose unacceptable challenges to students.

Let’s start with this basic question: Why in the world did I require 21 assignments? Yes, pedagogically speaking, this made sense: these assignments, collectively, added up the basic knowledge required of an introductory methods course. They covered topics such as writing research questions and hypotheses to measurement, ethics, sampling, correlation v. causation, and everything else. Back in the summer, I spent a lot of time considering whether to require everything, or allow students to complete some smaller number or percentage of the total.

This student just failed my class due to poor syllabus-making on my part.
This student just failed my class due to poor syllabus-making on my part.

Here’s the problem: if a student completes twenty assignments with a satisfactory or higher score, but misses assignment #21 because they forgot or overslept or had a family emergency, then according to my syllabus they fail the course outright. Sure, the tokens help with this, letting students get a 24 hour extension per token, up to 72 hours—but what if they don’t have enough tokens in the bank, or they completely forget for a week or more? These students FAIL THE COURSE.

Continue reading

Teaching the Scientific Method with Inception

Gokcek 2Today Gigi Gokcek, associate professor of political science at the Dominican University of California, discusses a technique for teaching the scientific method. She can be contacted at gigi.gokcek[at]dominican[dot]edu.

“You need the simplest version of the idea in order for it to grow naturally in your subject’s mind.”  –Eames the Forger

A course on research methods can elicit all kinds of emotions from both students and faculty — ambivalence, anxiety, or even fear. The active learning literature offers two reasons a course on research methods incites this kind of reaction: (1) covering statistics before properly introducing the scientific method, and (2) presenting the research process in a manner that lacks creativity.

Popular movies with puzzling endings are a creative way of teaching students the scientific method. Christopher Nolan’s movie Inception (2010) presents one such example. Inception’s premise revolves around Cobb, a thief who specializes in using dream technology to “extract” corporate secrets from the minds of his victims. Saito, the head of a major corporation, offers Cobb a job to plant an “idea” inside the mind of his main competitor, Fischer. In order to “incept” Fischer with an idea to divide up his father’s empire, Cobb recruits a team of thieves, who use dream technology to enter Fischer’s mind. The first half of the movie introduces the audience to the team, lays out the rules of the dreamscapes, and sets up the plot to “incept” Fischer. During the second half, the team uses a flight from Sydney to Los Angeles to enter Fischer’s mind through the dreamscapes. But the dream world poses challenges to the team as they battle Fischer’s mental defenses against the operation. The movie concludes with Fischer incepted and Cobb returning home to his children. The audience is left wondering if Cobb is still inside a dreamscape or if he has truly awakened and returned home.

A professor using a movie like Inception to teach the scientific method can do the following:

  • First, give the students questions about the movie’s ending: What kinds of evidence suggest that Cobb awakened in the end? What evidence suggests that Cobb is not awake in the end? (Sample student worksheet is here.)
  • Second, students can be asked to identify the relevant factors. This may be achieved by reviewing the plot of the movie for information that is vital to their interpretation of the ending. Students can be informed that each dreamer must have a totem, an inanimate object (like the top Cobb uses) that helps one distinguish the dream world from the dreamer’s own reality.
  • Third, students should be required to formulate testable hypotheses based on the information provided. For example, if the top does not fall then Cobb is dreaming, or if the top falls then Cobb is awake.
  • Fourth, students test their hypotheses by collecting information (data) both while viewing the film and afterwards by searching movie blogs online.
  • Finally, students should present their findings both orally and in writing, including any new questions that have emerged in the process.

Finding creative ways to engage students in the research process will not only help them truly grasp the “science” in political science, but will also create future ambassadors for our discipline who can demonstrate to outsiders the value of what it is that we do. If we can get students to appreciate the scientific method they will not only receive knowledge discovered by others, but also add to it. By using Inception in my methods courses, my students get “incepted” with the “idea” that the scientific process, as an opportunity to expand human knowledge, is to be welcomed rather than dreaded.

And the Data Say…

Hi, and thanks for reading! Chad has graciously invited me to share some activities with y’all focused on bringing research methods ideas into the substantive classroom. The goal of these activities is twofold. First, programs are increasingly adopting objectives (and even school-wide QEPs) about research literacy and data or numeric literacy. The activities I’ll be sharing support those goals.

Second, students have the best chance of success in the research methods classroom when Introduction to Research Methods is not actually their introduction to research methods. One of the reasons students have anxiety and less-than-optimal experiences in their Research Methods courses is the sheer quantity of new material. If we can decrease the novelty by even a small amount, we can improve their experiences in that often-required Methods class. Through these activities, I hope to show you some ways to incorporate social scientific thinking and methods into even introductory courses in ways that require relatively low amounts of additional preparation or even class time.

Students often ask questions about general trends or patterns. Do they usually do that? Is that what normally happens? With a little nudging, you can turn this curiosity into a data-based activity. When you reach a topic where data is readily available, consider a class activity that asks students to hypothesize about relationships between variables or indicators, then test those hypotheses on the spot using a quick graph. For example, my Intro to American Politics textbook spent a bit of time showing graphs of how partisanship has varied over time in Congress and how the current period of hyperpartisanship is different, and how partisanship affects the ability of Congress to get things done. Continue reading