Process v. outcome in simulations

Chad's simulations were becoming a bit too realistic...
Chad’s simulations were becoming a bit too realistic…

Very occasionally, different parts of one’s life collide with each other, often after you’ve had a bit too much to drink, which further contributes to the further unrolling of the evening.

Any way.

In this case, no drinking (except of cups of tea) was involved for me this week, when I attended a ‘wargame’ of the British renegotiation of European Union membership. Run by Open Europe, the day had two parts, each intended to cast some light on what might happen in, respectively, the current renegotiation and then in the event of a ‘no’ vote in the referendum.

This was a high-rent production. Fancy City venue, live-streaming of the entire event, plus (most importantly) roles being played by People (former ministers and ambassadors): The British were represented by a former Foreign Secretary, Maclom Rifkind, and a former Chancellor, Norman Lamont. All of this built on a previous event in 2013 and given the number of TV camera crews from across Europe, it was not your usual event.

For all that, it was also a very strange affair.

Continue reading

Simulations are just too amazing

Hell, yeah!
Hell, yeah!

For the past couple of days, I’ve been talking about simulations at an event organised by Peter Bursens and colleagues at the University of Antwerp. If we leave to one side how nice it was to get some many positive comments about this blog from people, then it was a really heartening workshop for more academic reasons.

One of the biggest challenges that users of simulations (and other active learning techniques) face is the lack of a robust evidence base that such pedagogies actually have an educational benefit for students, either at all or above and above ‘conventional approaches’. This workshop was directed precisely at discussing this gap.

Continue reading

Building new layers of student interaction

I’m gonna design you a game that you can’t refuse…

Last week’s good news came on the same day that I was invited to talk to colleagues at the University of Bath about simulations and role plays.

Felia Allum is running a module/course (actually, Bath calls them units, but you know what I mean) about organised crime in Italy and beyond and had won some funding to support the development of a much more active learning approach, using sims. Together with her Faculty’s e-learning development office, Geraldine Jones, they wanted to explore how this might work and to get some feedback on their ideas.

This kind of thing is exactly what I love about my work: getting a specific project into which to input and (hopefully) develop (plus a trip to Bath, which is always good anyway).

As with many of the other projects I’ve had this kind of role in, it’s something that at first (and indeed, second) glance look to be almost impossible: ideas that I would never have come to by myself.

I’ll not talk in much detail about this project – especially because I’m going to get Felia and Geraldine to do that further down the line – but I do what to draw out two really interesting aspects.

The first is their notion that students should design their own games. We know from the literature that teaching is one of the best ways to learn, and in terms of students developing an appreciation of the dynamics involved in the lives of organised criminals, making them create a game is very powerful.

The flip-side, obviously, is that this is rather daunting: I’ll admit I spent most of a morning trying to work how I might do it without much success. Felia and Geraldine’s way around this is rather good, namely giving the students a fairly rigid template of what is needed and what they have to do, coupled to feedback on a draft version, before any actual gameplay.

The second exciting aspect is about how you get students to internalise and understand the pressures that face criminals. The problem is that those pressures are very different to those facing a university student.

It is precisely through the students’ games that they will work towards this internalisation and understanding, but we talked about how an initial nudge might help them on the way.

With that in mind, we looked at starting the module with another game, to set up some tensions – about money, status, loyalty and trust – that might inform what comes later. My thought was that be having a system of credit for each of these factors would set up a clear signalling mechanism to students, which they could in turn build into their own games (thereby helping with the first point about design).

This is something still in development, but once I’ve had a chance to go through the game I pulled together on the journey home, I’ll be posting it up on my simulations website for you all to see.

Even if you’re not interested in organised crime, if you can get to involved in discussions with colleagues about their projects and ideas, then it’s not only good for them, it’s good for you too. I’ve come away from my day in Bath with a whole new set of thoughts and possibilities to explore.

The Best Laid Plans

Cat and MouseA while back I wrote a series of posts on reworking my first-year seminar. My assumption was that this fall’s version would meet three days a week, as happened in the course’s initial iteration. I recently learned that instead it will meet on Tuesdays and Thursdays. Given that much of the course involves student-to-student interaction in the classroom, the new schedule necessitated further changes. To start, I dropped the book that I had originally fit into the last third of the semester, and with it plans for a class-wide Twine project. The course now looks like this:

  • Team-based Twines on the book An Ordinary Man (Rwanda).
  • Simulation exercises on the first four cases in Chasing Chaos (Rwanda, Sudan, Sri Lanka, Sierra Leone).
  • Team-based Twines on the last case in Chasing Chaos (Haiti).

Since this is a course for incoming college students, I added The New Science of Learning and some other meta-cognitive content on skills for academic success. This means that students will have on average three writing assignments on readings per week even though the class only meets twice a week, which I think this is a good thing. Students won’t be able to forget about the course between Thursdays and Tuesdays.

As I discussed in my informal assessment back in January, I had a problematic formulation for the briefing memo that prepared students for each Chasing Chaos simulation. I’ve rewritten the assignment instructions accordingly, and created a new sample memo for students to use as a guide. The effort that I’m putting into the design of this course reflects something about how college works that I’ll discuss in more detail in my next two posts.

Multiple and Parallel Worlds: Simulations in Large Classes

For faculty that teach in large classrooms with several hundred students, lecture can seem like the only possible method of instruction. Discussion, let along simulations and games, are seen as better employed in TA-led sections, where the number of students is much more manageable. This may seem especially true for some of the games we discuss on ALPS, such as Diplomacy, which in its classic version is meant for only 7 players, and even in teams becomes unwieldy with more than 21.

At first glance, not the most appealing environment for gaming and role-play. Source.

There are simulations that can work really well in a large classroom setting. A Model United Nations, for example, run as either a General Assembly or split into committees, can be great with 200 students. The same principle holds for any organizational simulation, such as a Model Congress or Model European Union. The Hobbes Game also works regardless of the number of students.

But in other games, 200 students is about 170 too many. Groups are too large, there are high incentives for free-riding, or lots of dead time where students have to wait for others to act. Running one large game in this kind of environment is likely to be unsuccessful, if the game itself is not easily scalable.

Luckily, there is a trick to managing this problem. It involves the use of multiple or ‘parallel’ worlds, where instead of running a single version of the game, you run multiple simultaneous versions, with students split into several smaller groups and each group plays its own self-contained version of game. If you wanted to play Diplomacy, for example, you could bring in several game boards and run 3 or 4 different games at the same time, with a TA adjudicating moves for each independent game. In a UN Security Council simulation, you can run 2 or 3 different security councils, all working on the same issue.

There are several advantages to this. First, it lets you use games and simulations meant for smaller groups in a manageable way, whether during sections or during the main lecture meeting. Second, it lets students see how the same exact starting point can lead to very different results, and allows them to discuss the reasons for those differences (structure? social? individual ability?). It also creates some neat opportunities that smaller versions of the game may not have. For example, in Diplomacy, you could allow teams representing the same country in different game worlds to talk strategy with each other. Students can then see how the same strategy or goals can have vastly different effects depending on the actions of other teams and players.

So if you teach large classes and want to try out a game or simulation in your class, consider the parallel worlds model. It’s a great way to bring some more active components into the classroom!

For more on this, check out Jason Keiber’s paper, “Dividing Up the Game: From Serial to Parallel
presented at the 2013 APSA TLC.

Puzzle Pieces: Part 3

Jigsaw Puzzle 3As I’ve discussed in Part 1 and Part 2 of this series, I’m redesigning my first-year seminar for its second iteration. In the original version of the course last fall, students individually completed frequent reading responses on the three books I had assigned. Teams of students produced Twines on each of the books. These projects enabled students to interact with each other much more than during in-class discussions of their reading responses, but but their interest seemed to flag as the semester progressed — though the class’s 8:00 a.m. start time was probably also a contributing factor.

Whether the lack of interest was caused by boredom or sleep deprivation, I think my first-semester college students need to engage in a greater variety tasks. But as I mentioned in my previous posts, my choices are constrained by my course outcomes: interaction with peers, practice in developing higher order thinking skills, and exposure to cultural perspectives that are different from their own.

I am keeping the reading response technique; I use it in all my courses and it works well. The team-based Twine on An Ordinary Man seemed to be a success last fall, so I’ll keep that also. The Chasing Chaos book and its associated simulations generated a favorable student response when I first used them in another course, so I’m somewhat confident the same will occur in the first-year seminar. That leaves the third book, The Big Truck That Went By.

The Chasing Chaos simulations might be a different enough experience for students to remain engaged while working on a second Twine in the last part of the course. The problem with this idea is that in the first iteration of the course, teams wrote their Twines on each book’s protagonist. In The Big Truck That Went By, the protagonist is a U.S. journalist, and I don’t think students will benefit much from creating a fictionalized version of what he recounts in his book.

Would it be possible to involve the entire class in the creation of a single Twine on post-earthquake Haiti? Maybe. This is a video version of what I’m thinking of — the player chooses from one of several roles but the storylines derive from an integrated body of content.

The tricky part is figuring out how to build in individual accountability and a means of assessing it. I could leave the organization of responsibilities up to the students as a lesson in project management, but I should probably establish some signposts before students begin so that they have something against which to measure their performance. Or perhaps the whole idea — which I really did think of as I was writing this post — is likely to be an abysmal failure. Thoughts and suggestions are welcome.

Puzzle Pieces: Part 2

Jigsaw Puzzle 2As mentioned in my first post in this series, Chasing Chaos by Jessica Alexander corresponds well to the student outcomes I created for my first-year seminar — in part because of the associated simulations I had created for another course. But my informal assessment of these simulations last fall leads me to think that they need three major adjustments.

First, the negotiation phase for each simulation can be shortened to only one class period. If no team achieves its goal in the allotted time, that’s ok — these are crisis scenarios. Second, I am dropping the reward for a unanimous agreement between teams so that students are less likely to abandon their roles in pursuit of earning the maximum number of possible points. This will create more contentiousness and by default result in a proportion of student teams “losing” what they didn’t have to begin with, but again, I think this is ok.

Third, the briefing memos that I assigned to prepare students for the simulations were too complex. This type of analytic writing exercise is detailed in CATs (“analytic memos,” pages 177-180). As noted in CATs, the technique requires large amounts of time and effort from both students and the instructor, but it serves as a high-quality and realistic skill-building exercise for students. In my case and in contrast to the recommendations of the authors of CATs, I grade the memos as formal assignments  — otherwise students won’t do them.

Because of these three concerns, I have altered the instructions for the briefing memos as follows, and I have inserted information for the Rwanda simulation for the purposes of example:

You are employed by the Humanitarian Information Unit at the U.S. Department of State (HIU). A new President has been inaugurated, and the President’s senior foreign policy advisors seek recommendations on U.S. responses to emerging political and economic conflicts around the world. The HIU has been assigned the task of providing these advisors with recommendations that conform to the mission of the HIU. Your job is to write a briefing memo for your superiors in the following format:

♦ Single-spaced paragraphs with a blank line between each paragraph, 11 or 12 point font, and no longer than two pages.
♦ Correct identification of memo’s author and recipient. The sub-heading of “Recommendation,” followed by a single concise sentence that states your  recommendation.
♦ The sub-heading “Justification,” followed by at least one paragraph explaining why the U.S. government should adopt your recommendation as foreign policy. Background sources should be referenced using in-text citations rather than with footnotes or endnotes—for example, “(Eggers, 87)”.

See the sample briefing memo for guidance.

 Background sources:

♦ Samantha Power, “Bystanders to genocide: why the United States let the Rwandan genocide happen,” Atlantic Monthly 288, 2 September 2001.
♦ Jason K. Stearns, “Congo’s Peace: Miracle or Mirage?” Current History 106(700), May 2007.
♦ Thomas Turner, “Will Rwanda End Its Meddling in Congo?” Current History 112(754), May 2013.
♦ Jeffrey Gettleman, “The Global Elite’s Favorite Strongman,” New York Times, 4 September 2013.

The conflict:

A previously-unknown armed group calling itself the Hutu Liberation Front (HLF) has attacked three Congolese villages near the Rwandan border. The attacks killed the villages’ residents and several Congolese soldiers who were stationed at a checkpoint along a nearby highway that runs between Kinshasha and Kigali. The Rwandan government claims that the HLF is under the direction of Congo’s ruling political party and it has mobilized Rwandan army units for a potential incursion into Congo to fight the HLF. Simultaneously soldiers in Congo’s army who identify themselves as Tutsis have mutinied against their commanders and are leading a rebellion against the country’s elected government. French and U.S. intelligence agencies report that the mutiny may have been encouraged by the Rwandan government.

Puzzle Pieces: Part 1

Jigsaw Puzzle 1In the spirit of sharing, here is the first of a series of examples on the continuous-improvement approach to course redesign.

In the fall semester I will teach for the second time a seminar on disasters and survival to incoming college students. I thought last year’s inaugural version of the seminar was acceptable but that it definitely had room for improvement.

My intended outcomes for the course remain the same and act as my basic design constraint:

  • Create a classroom environment that doesn’t drive down my university’s retention rate, mainly by getting students to interact with each other as much as possible.
  • Foster higher order thinking skills so students become better decision makers (an outcome revealed by the CATS self-assessment for another course that I teach).
  • Encourage awareness of and respect for people who have cultural backgrounds and perspectives that are different from those of the students. (My research on global empathy did not detect any significant improvement on this the first time around.)

Given students’ responses to my online survey at the end of the semester, I knew that assigning different books might strengthen students’ achievement of the third outcome. Students reacted very positively to the book I assigned at the beginning of the semester, An Ordinary Man by Paul Rusesabagina, about the 1994 Rwandan genocide. However, some students said that they found the seminar’s other two books to be dull or confusing. Also, I realized after the books had been ordered that all three were written by and about men. It’s easy for me to be Exhibit A for white male privilege, but I do try to avoid it, especially since enrollment at my university is more than sixty percent female and contains a sizable proportion of first-generation college students.

So the first puzzle was finding two new books about disasters that affected people outside the USA, with at least one of them authored by a woman.

I am not be teaching introduction to IR in the fall, which means I can pull the book Chasing Chaos out of that course and into the first-year seminar. The author of Chasing Chaos is female, and the book is an autobiographical account of her experiences as a humanitarian aid worker in far-flung locations around the world. Check marks for relevance of content and author’s gender. Chasing Chaos has another benefit, referenced in the linked post above, but I’ll discuss that in detail soon in another post.

This left the third book, and I already had a few possibilities in mind: My Life As a Traitor, by Zarah Ghahramani; When Broken Glass Floats, by Chanrithy Him; and Even Silence Has an End, by Ingrid Betancourt. All are autobiographical, written by women, and discuss survival in extreme conditions.

My Life As a Traitor is about Ms. Ghahramani’s incarceration in Tehran’s Evin Prison for political activities that Iran’s government deemed subversive. It’s well-written and the author was a college student at the time the events described in the book took place. I was worried though that it would reinforce students’ pre-existing negative stereotypes about the Middle East — with no prior study of the region, it would be easy for them to attribute the treatment of Ms. Ghahramani to religion rather than to an authoritarian state that attempts to use religion to justify its oppression of citizens. So I removed the book from consideration.

When Broken Glass Floats is about the author’s childhood during Khmer Rouge Cambodia, a subject with which I am extremely familiar. This book would be very easy for me to use to stimulate class discussions. However, I realized that both it and An Ordinary Man are about genocide, and I didn’t want to make genocide the focus of the course. So I ruled this book out.

Even Silence Has an End is about the author’s six years as a hostage of the FARC in the jungles of Colombia, a story that probably a lot of people find riveting. But this book is over five hundred pages long — too much for the last third of the semester. I was not willing to drop An Ordinary Man or Chasing Chaos to fit it in. What to do?

In one of those minor epiphanies that sometimes happen when thinking about teaching, I remembered that 1) Chasing Chaos concludes with a chapter on post-earthquake Haiti, and 2) I use a book about the same topic in a graduate course — The Big Truck That Went By, by Jonathan Katz. Disaster, check. Cultural milieu that is different from that of my students, check. Short enough to fit? Check.

I have identified my three books for the next version of the course, and none obviously conflict with my learning outcome criteria. Now I have a framework on which to hang the rest of the course.

Simulating Stubbornness

Don’t be one (or at least not in a simulation)

We talk a lot on this blog about the beauty of simulations for producing rich and varied environments in which to immerse students. We also talk a lot about how ‘failure’ is usually as valid a learning point as ‘success’ in these games: the inability to reach the nominal objective is a valuable learning moment.

But as I’ve been reading for my other work, I realise there’s a different category of political phenomenon that we have more trouble in capturing: the (pre-planned) impasse.

Let me give you the example that triggered this thought, to illustrate.

As you’ll know – if you live in Europe, at least – Greece’s economy has suffered grievously over the past few years, as a result of the 2008 crisis and (more pertinently here) the strict austerity regime imposed by creditors. The election last month of a radical-left government has since led to efforts to recast that austerity regime into something else.

It would be fair to say that all sides are deeply dug in: the Greeks say the current situation is untenable, the creditors say they won’t wear anything else. As so, after a rather brief meeting last night of finance ministers, there was another failure to agree.

Now my cynical side (yes, I do have one) says that this is all part of the game: you act tough, we act tough, then we reach the settlement we knew we would at the outset, but we can assuage our constituents about it all. It’s a bit the logic of my own austerity game, where heads of government have scope to bluster.

However, from a simulation point of view, it’s actually really tricky to recreate this kind of behaviour. In the real world, actors are not bound to a discrete forum of action, nor to a timetable (although there is nominally one in the Greek case): this means their capacity and willingness to not try for a resolution is greater than normal.

If we were trying to run a game of the Greek situation, then we might have a finance ministers’ meeting, but either we reach an agreement (which might not be very realistic), or we don’t (in which case we have to talk about what happens next). What we probably can’t do is run another meeting, and another, so that we can capture the iterative dynamics of it all, nor the way in which we eventually reach compromise.

In short, politics isn’t built for the convenience of the academic calendar.

So what to do? Three ideas present themselves.

First is to just live with it, and note that a game’s failure to reach agreement is lifelike, but also unrealistic in that real-world actors get many more bites of the cherry.

Second is to try to build in rounds of negotiation into a game. The danger here is that you end up covering the same ground multiple times, and that your players might ‘solve’ the problem early on, in which case you have a bunch of time on your hands (although some crisis could easily be invented to upset things).

Third option is to not try to play out the scenario, but rather to get students to apply their knowledge of negotiation/politics/etc to the real-world situation and suggest ways that the different actors could progress things. Those suggestions might be presented as a negotiation brief or some other forum.

This last idea has some interesting advantages. Most importantly, it forces students to articulate negotiation strategy more explicitly (so you can evaluate it), as well as connecting conventional research with an active project.

Finally, trying this last approach is not site-specific: you can use it with a whole range of political events. I’m sure that a moment’s reflection will bring to mind an example of a political institution or process in your area of interest that isn’t working to its full potential, so use that. Easier than building a full simulation, but still with those elements of active engagement that we like so much.

Modelling systems in simulations

We’re gonna need a bigger seminar room

As I’ve noted before, simulations are really good for integrating students’ understanding of large, complex events: the active approach required forces students to apply their knowledge and adapt it to a wide range of elements and dynamics.

However, one area were this breaks down is in the area of systems, in the sense of larger-scale interactions. The two most obvious examples are the international system in toto and the various stages of a legislative procedure from start to finish, both things we find rather interesting in politics/IR.

The difficulty comes in the sheer volume of stuff that has to be modelled by the simulation: lots of actors and of variables, usually without the requisite time, space or number of players to make it work.

So what to do?

There are a number of ways to address this, each with merits and shortcomings.

The first way is just to pitch in and try to model it all: certainly that’s what sims like Statecraft try to do (it’s also my approach here). You take the chance that you cover all the key variables, you plonk the players in some initial condition and then let them go. As I’ve noted with my game, the breadth of possibilities is both a blessing and a curse: players can do a huge range of things, but this also often inhibits them as they get cramped by indecision. Moreover, it’s possible to get too hung up on the range of variables: what goes in, what stays out, how do they interact with each other? In short, you can do it, but it’s easy to end up with either a very, very complicated game or one that produces some perverse incentives.

The second way is to break down the big thing into some smaller things. For an international system, you might create a world of actors, who then meet in some specific locations (a UN meeting, a security challenge, regional cooperation, etc.) that can be simulated more easily. For comparativists, you might get actors contesting an election, then forming a government, then falling into (and out of) civil war. The appreciation of the larger picture then emerges from the shadow of the past games (and the shadow of the future ones, if you tell students what’s coming), and the reflection about how that shapes action. The tricky part of this is creating a situation where those different elements hang together in a sensible way: this might well be a case of having to try to build up the sims over a number of iterations.

The third way is a more networked approach. I wrote about this a couple of years ago, suggesting a set of legislative documents that different institutions could pick up and advance through a full legislative process. This would let everyone focus on a specific element, while also gaining a sense of a bigger picture/process. The detailing could be very impressive, but the downside is the one that should be evident: namely, it’s a very complicated thing to create and maintain (which is the major reason I never got around to trying to make it happen).

The final way is go go down the stylised route and produce games that capture the spirit, without all the detail (like this). This is certainly the simplest to run and manage, but one necessarily loses the specifics. It’s also worth saying that it’s often quite tricky to capture the key mechanism, not least because it’s often a normative choice about what you think is important to capture: One can say that the European Parliament is about how committees dominate procedure, rather than about how transnational groups form, depending on what you want to tell students.

As always, there’s no right answer in all of this, but as you approach the bigger things, it’s really worth spending some time thinking about how you tackle them.