As I’ve discussed here and here, this past semester I ran a role-playing simulation for undergraduates on international relations in Europe on the eve of World War I. Previously I had tested for the simulation’s possible effects on student learning by comparing essay exam grades from a “regular” class that participated in the simulation to grades from an honors class that had not. Grades from the honors class were higher, and in most cases the differences were statistically significant, but the possibility existed that the higher scores were due to the honors students supposed better academic ability.
This semester I finally was able to teach an honors class again and do a more valid comparison. Some preliminary data crunching indicates that the simulation had no effect on essay assignments and exams, not what I’d expected. However, in a 2009 study, Stroessner, Beckerman, and Whittaker found that students’ “writing ability, at least extemporaneous writing, was not affected” by Barnard’s Reacting to the Past role-playing simulations.* This makes me wonder what can be changed about history-based role-playing simulations so that students become better able to connect their simulation experiences to course content and demonstrate that connection through their writing.
I know someone reading this is thinking “it’s quite possible that your assignments and exams aren’t measuring what you think they are measuring.” Yes, possible, but logic dictates that if (a) we think it’s important for students to learn how to use theory to create a coherent and persuasive written explanation of an event, and (b) a question asks, for example, “did the international system in Europe prior to World War I best reflect liberal or realist IR theory, and why?” then (c) assignments and exams indicate how well students can accomplish (a). In other words, I’m testing for what I think students should know, and the simulation that I’ve been using doesn’t seem to have any effect on how much knowledge students acquire over a semester.
I will be presenting on this subject at the APSA Teaching and Learning Conference (TLC) in February. I’ve been attending the TLC since 2007 and it’s been remarkably rewarding — lots of interesting and pedagogically practical information. I encourage you to attend.