Managing Transitions

After my summer hiatus, I find myself back in the office, working through the pile of things that has accumulated on my desk in my absence. At one level, it’s a classic case of work-as-usual, in that there’s nothing that I haven’t had to deal with before at some point; but it’s also a key transition point.

In the past month, Surrey has switched over to its new VLE, SurreyLearn, after an extensive purchasing and implementation period. The switch to a completely new platform was predicated on the need for a step-change in functionality and robustness: my various responsibilities in learning & teaching have meant that I’ve been involved in the process from an early stage and I’m aware of the potential that the new system holds.

However, I now find myself asking how I am going to use this new system. On the one hand, I know that I can now do things that were impossible before, notably in integrating a number of other systems and networks into a single space. On the other, I am very dubious about innovating for innovation’s sake. One of the biggest barriers that I encounter when talking to colleagues across the sector is the fear that having an interest in L&T means having to constantly change one’s practice to accommodate the ‘latest thinking’. This concern is a real one, especially in situations where institutions are ambivalent about the place of teaching in their provision. Even here – where that does not apply – there are enough other areas of change that change-for-change’s-sake is not appropriate.

The upshot is that I’m going to wait for now. As the year progresses, I’m going to see when I can add real value-added via the new VLE, both from my own practice and from talking with others. This latter point is an essential one, in my view: Teaching can be a very personal activity, and often the most useful advice comes not from ‘the literature’ but from conversations with colleagues, especially those who have worked through the practicalities of a situation: educational theory is all well and good, but sometimes you just need to have a walk-through.

All of which leads me to a plug for internet resource on simulations that I was working on earlier this summer.  Thanks to the HEA, I was able to push on with a big part of the content. It’s meant to be a practical guide to using simulations and games (and draws on several elements from other ALPS contributors) and I’m very happy to talk about any of it with people and to receive more content.

Taking Student Motivations Into Account

Great session at TLC this year, with lots of interesting papers and ideas for simulations and games, including another wonderful workshop by ALPS’ own Victor Asal.

My own paper (with Nina Kollars) dealt with the issue of student motivation and engagement.  Our previous work was on the first principles of simulation and game design: that is, the instructor’s purpose for the game and its function within a classroom.  This new paper is on the next stage of design: taking student motivations into account.  Students approach our classrooms with different skills and motivations.  On the skill side, some may be expert, experienced gamers, while others may have little knowledge of how to work with the rules of the game; they may also differ in their teamwork abilities.  These differing skills–which may have nothing to do with our material or content–may impact the play of the game and the readiness and ability of students to learn from it.

Just as important is the differing motivations that students bring to the classroom.  This is not a groundbreaking observation by any sense of the word–psychologists have known this for decades. To use their terms, some students may be intrinsically motivated–interested in mastery of the material for its own sake–while others may be extrinsically motivated by a grade or some other reward or fear of punishment.  Such students will approach a game very differently, and our design of games should take these differing motivations into account.  An ungraded simulation or grade may fail to elicit full participation or learning in an extrinsically motivated student, but a game that IS graded may focus the intrinsic learner on external rewards, and perhaps lessen their intrinsic interest (see Deci 1971 and 1972 amongst many others).

Our point is that our knowledge of intrinsic and extrinsic motivations should impact the way we design our games.  We should not assume that a grade–even one of participation–is necessary to engineer participation in our games.  Games that are used in place or supplemental to lecture should be graded no more than attendance at a lecture.  Grades in fact might produce perverse incentives for students in a game, where they respond to the external motivators and not the game as we have built it.  Thus we must carefully consider how grading and other extrinsic motivators might influence the way students interact with our sim, and design accordingly.

The Results Are In

As I’ve discussed here and here, this past semester I ran a role-playing simulation for undergraduates on international relations in Europe on the eve of World War I. Previously I had tested for the simulation’s possible effects on student learning by comparing essay exam grades from a “regular” class that participated in the simulation to grades from an honors class that had not. Grades from the honors class were higher, and in most cases the differences were statistically significant, but the possibility existed that the higher scores were due to the honors students supposed better academic ability.

This semester I finally was able to teach an honors class again and do a more valid comparison. Some preliminary data crunching indicates that the simulation had no effect on essay assignments and exams, not what I’d expected. However, in a 2009 study, Stroessner,  Beckerman, and Whittaker found that students’ “writing ability, at least extemporaneous writing, was not affected” by Barnard’s Reacting to the Past role-playing simulations.* This makes me wonder what can be changed about history-based role-playing simulations so that students become better able to connect their simulation experiences to course content and demonstrate that connection through their writing.

I know someone reading this is thinking “it’s quite possible that your assignments and exams aren’t measuring what you think they are measuring.” Yes, possible, but logic dictates that if (a) we think it’s important for students to learn how to use theory to create a coherent and persuasive written explanation of an event, and (b) a question asks, for example, “did the international system in Europe prior to World War I best reflect liberal or realist IR theory, and why?” then (c) assignments and exams indicate how well students can accomplish (a). In other words, I’m testing for what I think students should know, and the simulation that I’ve been using doesn’t seem to have any effect on how much knowledge students acquire over a semester.

I will be presenting on this subject at the APSA Teaching and Learning Conference (TLC) in February. I’ve been attending the TLC  since 2007 and it’s been remarkably rewarding — lots of interesting and pedagogically practical information. I encourage you to attend.

*Stroessner, Steven J, Laurie Susser Beckerman, and Alexis Whittaker (2009) “All the World’s a Stage? Consequences of a Role-Playing Pedagogy on Psychological Factors and Writing and Rhetorical Skill in College Undergraduates,” Journal of Educational Psychology 101(3):605-620.

Getting some help in

Yesterday evening after class I had a long discussion with a student about the validity of different people’s opinions: the student felt that my opinion was worth more than that of his peers, because I was the teacher and “had more experience.”  This led into a rather “post-modern” (to quote my wife) discussion about the lack of objective truth and the value of all opinions.

I mention this because last week I took a group up to London to play a European Parliament simulation, at the EP’s UK office.  This is a development of a game I’ve played with students in Brussels, all organised by the EP.  Over the past years, I’ve found this has been an excellent opportunity to relate my class-based work to a more applied and specific context.  In addition, it has highlighted the value of expertise: Gergely Polner at the EP has been able to bring his extensive institutional knowledge to bear on the design of the game and its subject material.  Moreover, his network of contacts meant we were able to get a jury that included both EP officials and representatives from political groups, to give an insight that goes far beyond that which I personally could provide.  Coupled to some of the insights that the students gave into how particular issues might play out, everyone learnt something from the event.

The suggestion here is not that you beat a path to your local EP representation (helpful though they are), but to consider getting institutional buy-in from the organisations you might simulate: press offices abound; you or your colleagues have contacts.  It’s a relatively low-cost way for everyone to gain something useful (expertise, profile) and it underlines my ‘post-modern’ point about the value that different people can bring to the table.

Statecraft: A Simulation for International Relations

I’ve been meaning to write about Statecraft for some time. I was an alpha tester for the simulation last year and thus am very familiar with the team at Digital World Construction, the sim, and the development process.

Statecraft is “an immersive simulation that allows students to experience the challenges, opportunities, and complexities of international relations in a very vivid, intense, and personal way.”  Its based on a long standing pen-and-paper sim run by Dr. Jonathan Keller of James Madison University.

If you haven’t heard about it, check out the link above.  Basically students will play as teams representing fictional countries on a set map.  They have to manage their resources and wealth as well as various political factions in their countries, international organizations, military and diplomatic incursions from other countries, and natural disasters.  Its great fun for the students, and active learning at its best.  There’s a grading system incorporated into the sim, so the burden on the instructor is minimal (beyond, of course, providing the context via the concepts and theories of IR).  There is a fee of $25/student/course for the game, but if you consider the game as a ‘text’, that’s actually quite cheap compared to mainstream textbooks and readers. The sim can be used for classes as small as 7 or as large as needed.

If you’ve been thinking about using a sim in your class and don’t know how to go about creating one of your own, this would be a great choice.  My students loved it when they played, and that was before the onslaught of new changes which have really streamlined the gameplay and made the entire game more user-friendly.  If you are teaching intro IR in the spring, consider adding this to your syllabus.

20 Minute Legislature

I’m a big fan of the twenty minute simulation.  One of the big hurdles to using sims in the classroom is the perception that they take a lot of preparation on the part of the professor and a lot of class time to do properly. But sims do not need to be a big production in order to be successful.

Take for example my 20 Minute Legislature, an adaptation of an exercise found in the endlessly useful Instructor’s Manual for Ginsberg, Lowi, and Weir’s We the People.  I should note that I don’t use their text, but this is the best guide I’ve found in terms of suggesting discussion questions and activities.  Its a great go to reference, and deserves its own post.

Basically I give students five categories for spending a $100 billion national budget: Education, Transportation, Health and Welfare, Environmental Protection, and Defense.  They must budget each of these areas, and cannot give equal amounts to each.  They have 2 minutes to do so as individuals, and then based on their ranking of each area, I assign them to ‘committees’ which then have three minutes to come up with a committee budget for the legislature to consider.  Proposed budgets get placed on the board.  The students then have five minutes to persuade others to vote for their budget bill, making whatever promises and exchanges they please.  The final vote is held, and then we debrief for the last 7 minutes or so.

I’ve tried this a couple of times with good success.  I served as the Rules Committee (and the President) and mandated that there were no amendments allowed, that no one could vote for their own bill (in unequally sized groups this was important), and that only one bill could pass.  I also determined the order the bills were discussed.  I have played with the incentives involved, offering extra credit to the winning bill’s team and to the individual who’s personal budget most closely matches the budget that passes.  These rules and incentives do matter–the last time I tried this game, no bills passed.

Ultimately whether someone wins or loses is irrelevant.  The game is a quick and interactive way for students to understand the tradeoffs in budgeting, the role of committees in the legislative process, log rolling, and the influence of self-interest (as the authors predict, education always receives the most money).  Students can see that the process is messy and that what they consider to be good bills still need votes in order to pass. As a bonus, you can also compare their budgets with the actual discretionary budget of the US.

The exercise provides a great reference point throughout discussions of Congress and general discussions of politics as “who gets what, when and how” (as the authors note).  It works particularly well at the start of the lesson on Congress, and while I used it in a class of 25 could be adapted for larger and smaller groups relatively easily.

More on a Blog-based Simulation

This semester marks the second time I’m running my Europe1914 simulation in an introductory international relations course. I first taught this course to honors students in Fall 2008. In Fall 2009, I ran the simulation, but in a non-honors section. I had hoped that the simulation would be associated with better student performance on exams, but the data didn’t bear this out, probably because of the difference in academic abilities among students in the two groups.

In 2009, I asked students to rate themselves on how confident they were about being able to meet their goals at four different points in the simulation, before and after simulation sessions in class. Students’ confidence dropped markedly between the first and second assessments and then rebounded somewhat in the third and fourth assessments. The before and after ratings converged at the last assessment:

Student Confidence Over Time

I also asked how much control students thought they had over their success in the simulation (possible responses ranged from “I control my destiny in the simulation” to “I do not control my destiny in the simulation at all”). The results were similar – a sharp decline between the first and second assessments, followed by a rebound and convergence between the before and after scores.

My findings from the 2008/2009 comparison will appear in a 2012 issue of Journal of Political Science Education. If you’d like a copy of the Teaching and Learning Conference paper that the article is based on, please contact me.

This semester I’m teaching an honors section again, so I’m hoping to be avoid the apples-oranges problem by comparing exam scores from this semester with those from Fall 2008.

Even if my current students enjoy the simulation as much as the Fall 2009 students did, I’m questioning whether the exercise is worth the time and effort. In addition to the in-class time that the simulation eats up, I have to monitor the blogs (my inbox explodes), and students have technical problems that I can’t solve. It would be a lot more convenient if web apps like “Angry Birds” existed for instructional simulations.

If exam scores and other indicators show that the simulation has a beneficial effect on student performance, then I might continue to use this simulation. If there’s no demonstrable benefit, then I probably will not.

Blog-based Simulations

Two years ago I created a role-play simulation for an undergraduate international relations course. Though the simulation includes an in-class component, much of the action occurs on this blog. Feel free to borrow what I’ve created — just please credit me and my employer in the process. A few thoughts on using simulations like this:

Students are increasingly unfamiliar with blogging. Not only do I need to include a training session in how to use the blog for the simulation, I need to discuss the underlying premise of blogging itself. Though students may be regularly reading blog-style publications, social networking and mobile device apps have eclipsed blogs in their collective unconscious.

A blog should have the capability of delivering real-time updates to students’ preferred means of communication. While I do not necessarily need to know that Zachary replied to Kaitlyn’s latest post with “U rock grrl ha ha,” a torrent of messages appearing on students’ smartphones helps keep the simulation at the top of their screens and at the forefront of their minds outside of class.

The instructor must emphasize to students that any communication conducted outside of the blog will not be graded and, if done in lieu of the blog, will harm a student’s grade. This goes for texting, email, and face-to-face meetings. I tell students that I’ve created the blog to be their online workspace, and it’s their responsibility to use it.

Last item, which applies generally to all team-oriented simulations: individual writing assignments prevent free riders. Student who do a task initially on their own will be less likely to think “group project” when doing the same task later on with others.