The Results Are In

As I’ve discussed here and here, this past semester I ran a role-playing simulation for undergraduates on international relations in Europe on the eve of World War I. Previously I had tested for the simulation’s possible effects on student learning by comparing essay exam grades from a “regular” class that participated in the simulation to grades from an honors class that had not. Grades from the honors class were higher, and in most cases the differences were statistically significant, but the possibility existed that the higher scores were due to the honors students supposed better academic ability.

This semester I finally was able to teach an honors class again and do a more valid comparison. Some preliminary data crunching indicates that the simulation had no effect on essay assignments and exams, not what I’d expected. However, in a 2009 study, Stroessner,  Beckerman, and Whittaker found that students’ “writing ability, at least extemporaneous writing, was not affected” by Barnard’s Reacting to the Past role-playing simulations.* This makes me wonder what can be changed about history-based role-playing simulations so that students become better able to connect their simulation experiences to course content and demonstrate that connection through their writing.

I know someone reading this is thinking “it’s quite possible that your assignments and exams aren’t measuring what you think they are measuring.” Yes, possible, but logic dictates that if (a) we think it’s important for students to learn how to use theory to create a coherent and persuasive written explanation of an event, and (b) a question asks, for example, “did the international system in Europe prior to World War I best reflect liberal or realist IR theory, and why?” then (c) assignments and exams indicate how well students can accomplish (a). In other words, I’m testing for what I think students should know, and the simulation that I’ve been using doesn’t seem to have any effect on how much knowledge students acquire over a semester.

I will be presenting on this subject at the APSA Teaching and Learning Conference (TLC) in February. I’ve been attending the TLC  since 2007 and it’s been remarkably rewarding — lots of interesting and pedagogically practical information. I encourage you to attend.

*Stroessner, Steven J, Laurie Susser Beckerman, and Alexis Whittaker (2009) “All the World’s a Stage? Consequences of a Role-Playing Pedagogy on Psychological Factors and Writing and Rhetorical Skill in College Undergraduates,” Journal of Educational Psychology 101(3):605-620.

Best of Both Worlds Model at APSA

We make our students work in groups to learn from each other right?

Michael Brintnall has done amazing things for the APSA conference in a way that few appear to realize. There are, in fact, working groups at APSA, and although I was uncertain what might be helpful about attending one…I quickly learned.

“Understanding Terrorist Change” was the working group to which I assigned myself. At first we stood around, admittedly some of us had other papers written, and were somewhat uncertain as to what exactly we needed to do.

Our moderator treated the first meeting like a social. He told us to go out and listen, talk, and report back on the last day about what we saw and heard. Then we ended the formal part of the meeting. And the magic began…..We began to talk, and talk, and talk. What we found in our first meeting, was a vast spectrum of scholars who had expertise or curiosity for some part of thinking about the title.

In our second meeting we managed to synthesize the panels we all observed and in doing so capitalized on the multiple eyes and ears in the rooms.  Rather than having to pick and choose what I might find interesting, we got to distill what was being presented at all the APSA panels concerning terrorism.

From here we talked more.  We all became excited about the collaboration and contributions each member had to offer. We resolved to create an edited volume of papers that celebrated the interdisciplinarity, the vast array of perspectives, and approaches. Moreover we agreed that our moderator would assign us a set of definitions from which to begin our work. (Please do this, it will help create a cohesive project)

In short, something actually productive, collaborative, and important came out of the APSA working group.

More importantly, also in the room were professors of classes on terrorism, seeking to bring home the insights generated from the working group.  They were tasked with developing syllabi that rest on the forefront of of research, rather than exemplifying the coattails of a dusty old literature.

The working group.  What a novel idea…. It seems that teaching has taught us once again, how to learn from one another and it took Michael Britnall to show us that.

Next year, make a working group, or join one. It will make your APSA worth it.

APSA Thoughts

I have a love and hate relationship with APSA, but one thing I like is that it kicks off the year. I always come home with more energy to work.  Tomorrow I’ll be posting about some of the active learning ideas I encountered at panels, but today I’m going to take a time out to talk briefly about a disturbing trends that I saw and would love to discuss in the comments.

Panels are DEPRESSING.  Its rare that I leave a panel feeling happy that I attended instead of just downloading the papers on my own time.  There are plenty of reasons for this, but I think the most prominent one is that our format for exchanging knowledge at conferences is fundamentally flawed.  All the research that we know about how people learn best, and our preferred method is to have a group of individuals talk at the audience and each other for an hour and a half and then (if we are lucky!) allow for questions and dialogue with the audience.  I wish I could say that the teaching and learning sections did better, but one of these panels was the worst offender, with only ten minutes left for questions, and most of those more technical ‘how-do-I-do-this’ type questions instead of genuine discussion.

I much prefer the working group model of ECPR’s joint sessions, round-table style conversations, or the track method at TLC.  I would love to see us just throw out the rulebook, look up from our own papers, and talk to each other.  Perhaps that’s wishful thinking, but I do want to think through some other models that would really allow us to engage with each other and perhaps, even–dare I say it?–teach each other about our findings.

Edited to add: Nina posted about the working group model at APSA which also sounds like a better method and one that could be applied more broadly.