The Results Are In

As I’ve discussed here and here, this past semester I ran a role-playing simulation for undergraduates on international relations in Europe on the eve of World War I. Previously I had tested for the simulation’s possible effects on student learning by comparing essay exam grades from a “regular” class that participated in the simulation to grades from an honors class that had not. Grades from the honors class were higher, and in most cases the differences were statistically significant, but the possibility existed that the higher scores were due to the honors students supposed better academic ability.

This semester I finally was able to teach an honors class again and do a more valid comparison. Some preliminary data crunching indicates that the simulation had no effect on essay assignments and exams, not what I’d expected. However, in a 2009 study, Stroessner,  Beckerman, and Whittaker found that students’ “writing ability, at least extemporaneous writing, was not affected” by Barnard’s Reacting to the Past role-playing simulations.* This makes me wonder what can be changed about history-based role-playing simulations so that students become better able to connect their simulation experiences to course content and demonstrate that connection through their writing.

I know someone reading this is thinking “it’s quite possible that your assignments and exams aren’t measuring what you think they are measuring.” Yes, possible, but logic dictates that if (a) we think it’s important for students to learn how to use theory to create a coherent and persuasive written explanation of an event, and (b) a question asks, for example, “did the international system in Europe prior to World War I best reflect liberal or realist IR theory, and why?” then (c) assignments and exams indicate how well students can accomplish (a). In other words, I’m testing for what I think students should know, and the simulation that I’ve been using doesn’t seem to have any effect on how much knowledge students acquire over a semester.

I will be presenting on this subject at the APSA Teaching and Learning Conference (TLC) in February. I’ve been attending the TLC  since 2007 and it’s been remarkably rewarding — lots of interesting and pedagogically practical information. I encourage you to attend.

*Stroessner, Steven J, Laurie Susser Beckerman, and Alexis Whittaker (2009) “All the World’s a Stage? Consequences of a Role-Playing Pedagogy on Psychological Factors and Writing and Rhetorical Skill in College Undergraduates,” Journal of Educational Psychology 101(3):605-620.

Connections Exercise

This is the last week of classes, and I’m trying to get students to think about how what has happened inside the classroom can be used to understand a world of which they remain mostly ignorant. So I’ve come up with the following small group exercise, which I’ll probably give them about twenty minutes to complete:

Write a short narrative (not a bulleted list) that explains the connection between the following people, places, and things:

Free Trade Area of the Americas
Foreign Assistance Act Of 1961

 Correctly connecting eight of the above items earns one point toward your final grade for each person in your group. For nine, two points. For all of them, three points.

Since my students might find this post on Google, I’m not going to post the answer until later this week (please don’t reveal the answer if you know it).

The Pedagogical Utility of Half Credit

I structure most of my courses around what I call reading responses — short writing assignments on journal articles that students complete outside of class. These assignments fulfill several objectives:

  • Students read, think, and write outside of class, making them better able to actively engage with ideas during class. As a consequence, I don’t have to lecture as much and the dynamic of the class moves away from me delivering facts while students try to memorize them.
  • Students are able to exercise a limited degree of choice, which increases their sense of ownership and investment in the course (I typically have about fifteen assignments but students are only expected to complete ten of them).
  • Students have repeated opportunities to practice constructing written arguments that contain evidence from “experts in the field.”

It’s easy for students to earn full credit on these assignments. They must be:

  • turned in on time (before class, via course management software),
  • present an argument that addresses the question,
  • contain specific, properly cited references to the assigned reading.

I do not allow rewrites on these assignments because the requirements are very simple and the responses are the basis for class discussion on the days that they are due. Invariably there are some students who cease including cited references to readings in their responses about three-quarters of the way through the semester. I suppose the practice of completing these assignments becomes so routine to them that they get lazy about how they construct their arguments. In these cases the students automatically earn half credit. Receiving a score of 50 percent on an assignment usually wakes them up and gets them back on track fairly quickly.

Active Learning About Plagiarism

All my syllabi contain the usual policy statement about plagiarism — refer to the catalog for the university’s definition, don’t do it, if you do do it you might fail the course. And as is the norm for information that’s in a course syllabus, the statement often gets ignored, and I end up meeting budding plagiarists in my office for face to face discussions.

I’ve started requiring that any student who wants to “discuss” a plagiarized paper and the the grade (of zero) that it has received bring a document (typed, paper) to the meeting. The document has to contain, in the student’s own words, how the contents of his or her paper relate to the definition of plagiarism contained in the catalog. In other words, the student has to argue effectively that the paper was not plagiarized or admit to the plagiarism.

This method prevents the usual weepy sniffling or feigned indignation that occurs at such meetings. The student is forced to reflect on his or her actions, and the consequences thereof, before he or she enters my office. And I get a written confession.

Another benefit is that the process makes me less central to the situation and the conflict becomes more impersonal. When students read the university’s definition of plagiarism and begin writing about it, they see that they have violated a university policy, which I, as a faculty member, am simply upholding.

Using Rubrics As Teaching Tools

In the early years of my teaching career, I adopted rubrics to speed up grading of student writing, but I’d see the same mistakes, from the same students, on paper after paper throughout the semester. The content of the rubric would leave as little an impression on students’ minds as the inked comments on their papers. And there were the usual end-of-semester complaints of “Why didn’t I get a good grade on this?”

Now I have students create their own rubrics (sort of). About a week before the first major writing assignment is due, I distribute a few short writing samples to students. Each sample is a modified anonymous passage written by students in prior semesters. Each passage contains a variety of writing errors — vague or hard to find thesis statement, illogical organization, run-on sentences, spelling mistakes, etc.  I usually hand out five or six different samples. Students read and write comments on the samples and then form groups with other students who have the same one.

While in groups, students compare notes and discuss how the passage they’ve examined can be improved. In the final stage of the exercise, one student from each group reports on the group’s findings to the rest of the class. I jot down notes and ask questions.

I then tell the class that I will create a rubric based on what they have identified as indicators of good and bad writing. Students are therefore responsible for following their own recommendations.

Students make similar comments every semester, so I don’t have to change my rubric much, if at all.

Blog-based Simulations

Two years ago I created a role-play simulation for an undergraduate international relations course. Though the simulation includes an in-class component, much of the action occurs on this blog. Feel free to borrow what I’ve created — just please credit me and my employer in the process. A few thoughts on using simulations like this:

Students are increasingly unfamiliar with blogging. Not only do I need to include a training session in how to use the blog for the simulation, I need to discuss the underlying premise of blogging itself. Though students may be regularly reading blog-style publications, social networking and mobile device apps have eclipsed blogs in their collective unconscious.

A blog should have the capability of delivering real-time updates to students’ preferred means of communication. While I do not necessarily need to know that Zachary replied to Kaitlyn’s latest post with “U rock grrl ha ha,” a torrent of messages appearing on students’ smartphones helps keep the simulation at the top of their screens and at the forefront of their minds outside of class.

The instructor must emphasize to students that any communication conducted outside of the blog will not be graded and, if done in lieu of the blog, will harm a student’s grade. This goes for texting, email, and face-to-face meetings. I tell students that I’ve created the blog to be their online workspace, and it’s their responsibility to use it.

Last item, which applies generally to all team-oriented simulations: individual writing assignments prevent free riders. Student who do a task initially on their own will be less likely to think “group project” when doing the same task later on with others.

More on Solving the Reading Problem

Many of us are familiar with the think-pair-share exercise in which a class is given a question and, after a short period of time to think about it, students pair up to discuss their answers. Two downsides to think-pair-share: students will state opinions instead of referencing readings unless questions are carefully worded, and students are not writing.

I run an exercise that combines elements of think-pair-share and Amanda’s weekly critique method.  Each reading assignment is accompanied by an argumentative (why rather than what or when) question. Students answer a question before its corresponding reading is due to be covered in class. Answers must incorporate specifics from the reading assignment, are limited to one-half to a full page in length, and contain proper citations. Grading is on a 0 to 2 point basis and is very quick. Online submission of the answers prevents the “my printer didn’t work” excuse. Typically I create a dozen questions sprinkled throughout the semester and students have to answer ten of them for 20-30 percent of their final grade.

In class, I divide students into groups of four. Each student quickly reports to his or her group on the written answer he or she submitted before class. Each group then has a few minutes to reach a consensus that it can present orally to the rest of the class. At the beginning of the semester, I specify different roles for each student in a group — taking notes, keeping time, presenting the consensus — but students learn the routine very quickly. I’ll randomly select a few groups to present answers that often conflict with each other, which launches discussion for the entire class.

Benefits of this method: students are forced to intellectually engage with the readings outside of class, before hearing me lecture, and they have repeated opportunities to practice constructing evidence-based arguments. I get to lecture less, students participate meaningfully, and class is more interesting for everyone.