Team Champ

As promised, here is the prompt for the collaborative portion of the forecasting project in my upcoming Middle East course. There are two of these team deliverables — a draft report due after students have submitted the first three of their individual CHAMP assignments, and a final version (shown below) due at the end of the semester. In terms of contribution to course grade, the draft and final versions together are worth only a third of what the five individual assignments are worth. Also, a portion of the course grade will come from teammate evaluations.

Your team is interviewing for a job with Eurasia Group. The interview process requires that your team submit a jointly-written report on your team’s Forecasting Project topic using the CHAMP framework:

  • Comparisons
  • Historical Trends
  • Average Opinion
  • Mathematical Models
  • Predictable Biases

Your team’s final report should apply all five components of the CHAMP framework in a forecast that is no more than five pages of double-spaced 11- or 12-point font text. Do not use quotations of sources in the report. Reference source material using footnotes. See the list of syllabus readings for proper bibliographic format. Footnotes can be in 10-point font. 

Only one member of your team needs to submit the document for this assignment.

Your team’s work will be assessed using the rubric below.

Now I just need to create rubrics for the project’s individual and team assignments . . .

Assign Like a CHAMP

As promised in my last post, here is an example of iterating so that students repeatedly practice the same skills.

As I’ve previously mentioned, I’m putting a forecasting project into my fall semester Middle East course. The project’s constituent assignments will be based on the CHAMP system recommended by people like Phil Tetlock. A brief description of CHAMP is at the end of this Financial Times article by the economist Tim Harford.

My prompt for the first CHAMP assignment reads:

You are interviewing for a job with Eurasia Group. The interview process requires that you submit a forecast on your team’s Forecasting Project topic. The forecast needs to use the CHAMP framework:

  • Comparisons
  • Historical Trends
  • Average Opinion
  • Mathematical Models
  • Predictable Biases

In a one-page, double-spaced, 11- or 12-point font document, answer these questions for the Comparisons portion of your forecast:

What other cases are comparable to this situation?
How do they indicate what will happen this time?

My guiding questions for the other CHAMP assignments are:

Historical Trends

What individuals, groups, and institutions played key roles in similar events in the past?
How are these “power players” likely to influence the current situation?

Average Opinion

What are the experts predicting about this situation?
What is the view that lies in the middle of their assessments? 

Mathematical Models

Are there mathematical models or empirical measures that can be used to gain insight into this situation?
What do these models or measures indicate?

Predictable Biases

How has your thinking been affected by emotion and personal preference?
How have you adjusted your analysis to account for these biases?

I’ll talk about the team-based aspects of this project in a future post.

American Autogolpe

A brief post this week about the televised hearings of the U.S. House of Representatives’ January 6 committee.*

Republic

I teach democracy from a comparative perspective, a challenge when students have had the ideology of American exceptionalism drilled into them since birth.

When watching the second installment of the hearings, it occurred to me that they could serve as a reality check for students who tend to see “democracy” as a purely American phenomenon and whose culminating undergraduate achievement is a legalistic rehash of a 19th century Supreme Court opinion on the U.S. constitution’s Establishment Clause.

In my opinion, a much more meaningful exercise would be for students to research forms of democracy and threats to it globally. A class could be divided into teams with each team analyzing a different country in relation to the USA. Testimony from the hearings could be used to identify pivotal events that might or might not parallel what has happened in, for example, Venezuela.**

It just so happens that there are plenty of people who already thought of this kind of project — the folks at Democratic Erosion. Check out their sample syllabus for a semester-long course.

* full name: Select Committee to Investigate the January 6 Attack on the United States Capitol

** with readings such as Javier Corrales, “Authoritarian Survival: Why Maduro Hasn’t Fallen,” and Milan W. Svolik, “Polarization versus Democracy,” which appeared in Journal of Democracy in 2020 and 2019, respectively.

Perusall 4

Another reflection on last semester’s comparative politics course . . .

I noticed a loose association between final course grades and students’ Perusall activity, so the cost-benefit of engaging or not engaging with Perusall assignments ought to be transparent to students.* Another plus: because Perusall scores student activity automatically with an AI algorithm, the assignments are basically “set and forget” on my end. This aspect was very convenient when I didn’t have the time or inclination to read all of the students’ annotations on certain assignments.

I’m so pleased with how Perusall functions that I’m going to incorporate it into my fall semester undergraduate courses.

Previous posts on Perusall:

Perusall

Perusall 2

Perusall 3

*With only twelve students in the course by the end of the semester, I’m not going to bother to try to calculate correlation coefficients.

Creating Wicked Students 3

Time to reflect on the previous semester’s successes and failures:

I might be on to something with the Wicked Problems that I created for my comparative politics course. Previous posts on the subject are here and here. A brief synopsis of the activity: in class, teams of students have to quickly determine and present a possible solution to an unstructured, authentic problem. I put four of these exercises into the course:

  • Political risk consultants recommend to Volkswagen executives which of two sub-Saharan African states is most suitable for establishing a new automobile manufacturing site and sales network.
  • Defense Intelligence Agency analysts identify which of three Latin American U.S. allies is most susceptible to a Russian GRU election disinformation campaign.
  • The United States Institute for Peace delivers a conference speech on constitutional design for leaders of Libya’s major political parties that compares constitutionally-established institutions of government across four states.
  • Members of Iran’s Mujahedin-e-Khalq create a strategy for overthrowing the Islamic Republic by examining revolutionary movements in four other states.

Students found the exercises engaging. My exams included a question that asked students to reflect on what they learned about their problem-solving ability from each Wicked Problem, and their answers indicated a reasonable degree of meta-cognition.

But it was obvious that students failed to use the methods of comparison that I repeatedly demonstrated during class discussions. I expected students to organize their cases and variables into a simple table, like I had, but they didn’t. So, for example, instead of something like this:

BotswanaNigeria
Ethnically heterogeneousNoYes
Prior civil warNoYes
Major oil exporterNoYes
High level of political riskNoYes

students presented the equivalent of this:

Nigeria has a large population and represents a larger automobile market than Rwanda, so Volkswagen should site its new operation in Nigeria.

I suppose the solution is to require that students create their presentations by filling in a blank table, which will force them to select cases and variables in a logical manner.

The Marshmallow Tower Game

Along the lines of my last post, I’ve tweaked another game that I have used previously — the marshmallow challenge. My goal was to illustrate how economic development can be considered a collective action problem in which trust plays a key role. Here are the rules of the game:

  • Each team has 18 minutes to build a tower topped by a marshmallow using the materials provided.
  • The members of the team that builds the tallest tower earn 25 points each.
  • A “Red” player secretly placed on your team gets 25 points if their real team wins.
  • If a team correctly identifies its Red player, each team member wins 25 points. Only one guess per team.

The debriefing discussion included my brief description of Rousseau’s stag hunt scenario, and these questions:

  • If one considers the height of a tower as an indicator of a society’s level of economic development, why did some societies (teams) develop more quickly than others?
  • Did cultural values promote trust among team members?
  • What was in each person’s best interest? Were these interests achieved?
  • How did having a Red on your team affect your team’s behavior?
  • Who do you think the Reds were? Why?
  • How does it feel to be accused of being a Red?

At the very end of the discussion, I revealed that there were no Red players.

The class had ten students that I divided into three teams. One team’s tower collapsed when time expired, but none of the teams exhibited a high degree of dysfunction due to suspicions about the identity of its Red player. As usual, I think the game would work better in a class with more students.

The Bandit Game

In an attempt to rectify the failure of my previous classroom game on ethnic heterogeneity, democracy and dictatorship, I created another game that included a loss aversion component. I intended the game to demonstrate the concepts found in Mancur Olson’s 1993 article, “Dictatorship, Democracy, and Development” (The American Political Science Review 87, 3). Here are the rules for game’s initial version:

  • Each person gets a playing card and 4 chips.
  • The class is divided into small groups.
  • The person with the highest card value in each group is a bandit.
  • The game has five rounds.
  • Each group’s bandit confiscates 1, 2, 3, or 4 chips each round from every other group member. This decision is made by the bandit. The bandit has to confiscate at least 1 chip from each group member each round, assuming the group member has a chip.
  • After round 1, 2, 3, and 4, each non-bandit gets 1 additional chip if they have ended the round with > 0 chips.
  • The person in each group with the most chips after round 5 earns points equivalent to the number of chips in their possession.

Version 2 of the game has the same rules as Version 1, plus:

  • A bandit can switch to a different group after each of rounds 1-4. The bandit with a higher value card turns another group’s bandit into an ordinary person.
  • The new bandit takes the eliminated bandit’s chips and can keep them or distribute some or all of them in any manner to members of their new group.

Version 3 has the same rules as Versions 1 and 2, plus:

  • Members of a group can eliminate a bandit if (a) they have card suits different from the bandit’s suit, and (b) the combined value of their cards exceeds the value of the bandit’s card. If a bandit is eliminated, the bandit’s chips are distributed equally among the challengers.
  • A bandit can retain control if (a) group members with cards of the same suit as the bandit’s decide to ally with the bandit and (b) the combined value of cards of this suit exceeds that of the bandit’s challengers.

Before play started, I stacked the deck with cards from only three suits because of the small class size — thirteen students are registered for the course, but only eleven showed up. I divided these eleven students into three groups.

For all versions of the game, all bandits confiscated the same number of chips from their group’s members in each round, even though the rules did not specify that they had to do this. In Version 1, one bandit confiscated all the chips from every group member in one round, which ended that group’s game play for the remaining rounds — demonstrating that it’s better for a stationary bandit to extract only a portion of wealth from the populace at any given time. During Version 2, no bandit changed groups, and in Version 3, no one tried to eliminate a bandit.

This game worked better than the last one, but it still needs a much larger number of participants for it to function as intended.

When a Game Fails

An inadvertent update to a 2015 post on the perils of small classes:

I recently ran a game in two classes that I had hoped would demonstrate the effects of ethnic heterogeneity in dictatorships and democracies. The basic mechanics of the game:

The class is split into groups. Each person gets a playing card. Card suit represents ethnicity, though I didn’t tell students this. A card’s numeric value equates to the power level of the person holding it. If someone in a group has a face card, then the group is a dictatorship. The person in the group with the highest value face card is the dictator, who makes all decisions. If no one in the group has a face card, then the group is a democracy, with decisions made by majority vote. The numeric values of the cards don’t matter.

The game is played in multiple rounds, with a greater number of points at stake in each round — I used five rounds, worth 3, 5, 7, 10, and 15 points, respectively. These points count toward the final course grade. In every round, each group allocates its points to its members according to the rules above. If anyone in a group is dissatisfied with how the points were distributed, the person can recruit a cluster of allies who have cards of the same suit to challenge the distribution. In a dictatorship, the challenge succeeds if the cluster’s combined power level exceeds that formed by the dictator’s allies. In a democracy, the challenge succeeds if the cluster’s total power level exceeds that of the rest of the group. When there is a successful challenge, the group has to distribute its points in a different way. Each round had a time limit of just a few minutes, and if a group failed to successfully allocate its points before a round ended, the group’s points for that round disappeared.

Continue reading “When a Game Fails”

Creating Wicked Students 2

As promised last week, here is an example of a wicked problem I’ve given to my comparative politics class.

  • You are an employee of the The Scowcroft Group.
  • Volkswagen wants to expand into a new African market.
  • Setting up production facilities and distribution channels will take three years.
  • Which sub-Saharan African country should Volkswagen choose to expand into?
  • Your task is to compare risk to political stability for two sub-Saharan African nation-states, and choose the one with the least risk.
  • Use ≥ 1 quantitative and ≥ 1 non-quantitative indicator.
  • Present your recommendation to Volkswagen’s CEO and board of directors.
  • You have 15 minutes to create a 3 minute presentation.

I show the instructions, small teams of students work on the problem, and each team presents its solution. I grade the presentations using this rubric:

Perusall 3

I decided to survey my comparative politics class on their opinions about Perusall after the first exam. Of a total of thirteen students, only eight were in class on the day of the survey, so the results are in no way statistically representative. But here they are anyway. Each survey item was on a five-point scale, with 1 equal to “strongly disagree” and 5 as “strongly agree.”

Ave Score
Reading other people’s annotations helps me understand assigned readings.4.1
The university should continue to offer Perusall as an option for undergraduate courses.3.2
I find Perusall difficult to use.2.4
I’m more likely to read assigned journal articles that are on Perusall.3.3
Perusall helped me complete reading responses.3.6
Perusall helped me study for the exam.3.4

No obvious warning signs in the results. And my main objective in using Perusall — to increase students’ understanding of assigned readings — was the statement with which they most strongly agreed.

The class has scored on average 80% on Perusall assignments so far. In my opinion, this is a sign that Perusall’s assessment algorithm fairly evaluates the quality of students’ interaction with assigned readings. Since the marking process involves no effort on my part, it’s win-win situation. I’m now thinking of how I can incorporate Perusall into other courses.

Other posts in this series:

Perusall

Perusall 2