APSA Montreal Travel Requirements

Non-Canadians planning to attend the APSA annual meeting in Montreal next month are subject to some additional entry requirements related to the coronavirus pandemic. First and foremost, you have to meet the criteria for “fully vaccinated.” Visitors must provide evidence of vaccination status at their point of entry using the Canadian government’s ArriveCAN app. Full instructions on this process are here.

For people in the USA who plan on driving to Montreal: in the past, cellular reception has been spotty near the border, so I plan on having paper copies of the ArriveCAN information as a back-up.

Perusall 6

A brief note about Perusall course settings:

I put Perusall assignments into an online graduate course that I’m currently teaching. For the course’s first two weeks, students’ assignment scores were not syncing with the Canvas gradebook, nor were they visible to students in Perusall, until after the assignment deadline had passed. I had to manually release scores for each assignment. Perusall was not functioning as it had with my undergraduate courses in the spring semester, when assignment scores were always visible to students and were updated continuously in real time.

I eventually found the cause of the problem. I had not selected when to release scores to students in the settings page of the instructor’s dashboard:

Either this setting’s default had changed after the spring semester from “immediately, as students submit work” to one of the other options, or I had forgotten that I needed to change it when I was building the course on Perusall. Either way, the problem was easily solved. To this absent-minded professor, it was another demonstration of how easy Perusall is to use.

Previous posts in this series:

Perusall

Perusall 2

Perusall 3

Perusall 4

Perusall 5

Team Champ

As promised, here is the prompt for the collaborative portion of the forecasting project in my upcoming Middle East course. There are two of these team deliverables — a draft report due after students have submitted the first three of their individual CHAMP assignments, and a final version (shown below) due at the end of the semester. In terms of contribution to course grade, the draft and final versions together are worth only a third of what the five individual assignments are worth. Also, a portion of the course grade will come from teammate evaluations.

Your team is interviewing for a job with Eurasia Group. The interview process requires that your team submit a jointly-written report on your team’s Forecasting Project topic using the CHAMP framework:

  • Comparisons
  • Historical Trends
  • Average Opinion
  • Mathematical Models
  • Predictable Biases

Your team’s final report should apply all five components of the CHAMP framework in a forecast that is no more than five pages of double-spaced 11- or 12-point font text. Do not use quotations of sources in the report. Reference source material using footnotes. See the list of syllabus readings for proper bibliographic format. Footnotes can be in 10-point font. 

Only one member of your team needs to submit the document for this assignment.

Your team’s work will be assessed using the rubric below.

Now I just need to create rubrics for the project’s individual and team assignments . . .

Another Note on Graduate Education

Two recent articles about grant-funded efforts to reform doctoral training caught my attention. One appeared in Inside Higher Ed and the other in The Chronicle. The latter article is paywalled, so here is a summary if you can’t access it:

  1. Professional internships need to be part of doctoral training because they provide examples of non-academic career paths. Yet the internships created through the grant were in arts and humanities non-profit organizations and in higher education — industries with few jobs and low salaries.
  2. Graduate department curricula remained focused on producing subject matter experts, through traditional disciplinary coursework and production of a dissertation.
  3. Initiatives died when the grant ended. There was no institutional buy-in.

I’ve written before about the need to change doctoral programs; for example, here and here. But I don’t have much hope that this change is going to occur quickly enough. Institutionally, graduate education in the humanities and social sciences is mainly organized to replicate itself, leaving it increasingly disconnected from present reality.

Assign Like a CHAMP

As promised in my last post, here is an example of iterating so that students repeatedly practice the same skills.

As I’ve previously mentioned, I’m putting a forecasting project into my fall semester Middle East course. The project’s constituent assignments will be based on the CHAMP system recommended by people like Phil Tetlock. A brief description of CHAMP is at the end of this Financial Times article by the economist Tim Harford.

My prompt for the first CHAMP assignment reads:

You are interviewing for a job with Eurasia Group. The interview process requires that you submit a forecast on your team’s Forecasting Project topic. The forecast needs to use the CHAMP framework:

  • Comparisons
  • Historical Trends
  • Average Opinion
  • Mathematical Models
  • Predictable Biases

In a one-page, double-spaced, 11- or 12-point font document, answer these questions for the Comparisons portion of your forecast:

What other cases are comparable to this situation?
How do they indicate what will happen this time?

My guiding questions for the other CHAMP assignments are:

Historical Trends

What individuals, groups, and institutions played key roles in similar events in the past?
How are these “power players” likely to influence the current situation?

Average Opinion

What are the experts predicting about this situation?
What is the view that lies in the middle of their assessments? 

Mathematical Models

Are there mathematical models or empirical measures that can be used to gain insight into this situation?
What do these models or measures indicate?

Predictable Biases

How has your thinking been affected by emotion and personal preference?
How have you adjusted your analysis to account for these biases?

I’ll talk about the team-based aspects of this project in a future post.

Identity Theft: An Example of Cognitive Load Theory

What is the connection between identity theft and designing for cognitive load?

A close colleague recently discovered her inbox filling with spam at a rate of over 1,000 messages every twelve hours. Buried in the avalanche was a single email from a bank congratulating her on a new account that she hadn’t opened. Upon contacting the bank, she discovered that someone had obtained her social security number, opened the account, and linked it to second account under yet a different name.

This is not germane to my learning.

The strategy is known as “email bombing” — flood someone’s email with obvious spam on the expectation that they won’t notice the one message signaling identity theft. In other words, criminals are maximizing extraneous load to decrease learning.

My university returned to in-person undergraduate instruction last year, but I’m focusing on reducing extraneous load for the fall semester as if I were still teaching these courses online. This means dropping some non-essential content so that students get more opportunities to practice applying a minimum amount of knowledge to achieve a specific learning outcome. For example, in the course that includes game design, students will have five rather than four opportunities to evaluate existing games before finishing the construction of their own game. In the course where students will try to predict the future, they will practice the different elements of good forecasting multiple times. More on this in my next post.

Perusall 5

I’ve begun integrating Perusall into my online, asynchronous graduate international relations courses. First up is a course in our master’s degree program that starts next month. I’ve chosen to start with this one because I typically assign an analysis of a peer-reviewed journal article in lieu of a midterm exam, and the questions in my Perusall assignments for undergraduates mirror my instructions for the article analysis. Regular Perusall assignments will give them opportunities to develop skills they will need for the article analysis.

While practice improves performance generally, in this case I see it as particularly important. A growing proportion of our M.A. students are undergrads who have opted for a fifth-year master’s degree. They begin taking graduate courses in their fourth year of college. My four-person department only has about ten political science majors per year, but given the organization of the department’s curriculum, I encounter only about half of these majors in the classroom prior to their graduation. This means a wide variation in content knowledge and writing ability among the majors who enter the five-year program and first pop up in my M.A. courses. Making the situation even more complicated: the two-year M.A. students are often mid-career military officers who have first-hand international experience and are very academically talented.

These courses are seven weeks long. Previously I assigned an extensive list of readings, two writing prompts, and discussion board participation each week. I’ve replaced one of the writing prompts with two Perusall assignments in each week. I’m hoping that this change will help build a sense of community among the students, which is more difficult to achieve in an asynchronous online environment than it is in a physical classroom. At minimum the use of Perusall should cause students to notice the superior skills of some of their classmates and stimulate them to increase their own efforts.

American Autogolpe

A brief post this week about the televised hearings of the U.S. House of Representatives’ January 6 committee.*

Republic

I teach democracy from a comparative perspective, a challenge when students have had the ideology of American exceptionalism drilled into them since birth.

When watching the second installment of the hearings, it occurred to me that they could serve as a reality check for students who tend to see “democracy” as a purely American phenomenon and whose culminating undergraduate achievement is a legalistic rehash of a 19th century Supreme Court opinion on the U.S. constitution’s Establishment Clause.

In my opinion, a much more meaningful exercise would be for students to research forms of democracy and threats to it globally. A class could be divided into teams with each team analyzing a different country in relation to the USA. Testimony from the hearings could be used to identify pivotal events that might or might not parallel what has happened in, for example, Venezuela.**

It just so happens that there are plenty of people who already thought of this kind of project — the folks at Democratic Erosion. Check out their sample syllabus for a semester-long course.

* full name: Select Committee to Investigate the January 6 Attack on the United States Capitol

** with readings such as Javier Corrales, “Authoritarian Survival: Why Maduro Hasn’t Fallen,” and Milan W. Svolik, “Polarization versus Democracy,” which appeared in Journal of Democracy in 2020 and 2019, respectively.

A Tempting Experiment

You’re not trained for that

I recently listened to this episode of the Hidden Brain podcast, on using audible clickers to train humans how to throw a frisbee and perform surgery. Clickers seem to be very effective in part because they substitute for other, possibly emotion-laden reactions from the trainer.

I wondered how I might use clickers as a teaching tool, and had an email conversation with a psychology colleague who specializes in behavioral training. Here is the gist of the conversation:

Clickers provide immediate positive feedback for a specific, discrete action within a complex chain of behaviors, without the need to interrupt the chain as it unfolds.

Any process that is composed of multiple, discrete behaviors is amenable to clicker training, as long as the process can be observed by the teacher and the clicks can be delivered within a second or two of observing the targeted response. An activity like writing is probably not suitable for clickers, because the writing process can’t easily be separated into a series of precisely-defined behaviors, and it would require that the teacher continuously observe the student perform the writing task from start to finish.

However, the technique could be applied to something like class presentations — with clicks delivered when students complete important components of the presentation, such as using terminology correctly, answering an important question, speaking at an adequate volume, or making eye contact with the audience.

Clicks should initially be delivered each time the targeted behavior is displayed, but then systematically delivered less often as the behavior becomes more frequent. Likewise, they should only be used to indicate support for a desirable behavior rather than to signal disapproval of an undesirable behavior.

Last, teachers should obtain consent or buy-in from students about the process and its goals.

Can Students Predict the Future?

I like each of my undergraduate courses to have at least one authentic skill-based deliverable; for example, historical timelines or ArcGIS storymaps. I’m teaching the Middle East in the fall semester, and, having recently read books like Superforecasting by Tetlock and Noise by Kahneman, Sibony, and Sunstein, I hit upon students trying to predict events in the region.

I’ll have students assign percentage probabilities to a list of potential events and give them opportunities to periodically adjust these probabilities. At the end of the semester, students will earn points if the events have happened and lose points if they haven’t, with point amounts weighted against the associated percentages. My formula:

((Sum of the percentage probabilities student chose for each of his or her correct forecasts) – (Sum of the percentage probabilities student chose for each of his or her incorrect forecasts ))*50.

A hypothetical example:

% ProbabilityEventOutcome
100The sun rises in the eastern skyCorrect
30The capital of Turkey changes from Ankara to IstanbulIncorrect
70Barham Sali remains president of IraqCorrect
50Iran fires an ICBM with a nuclear warhead at Saudi ArabiaIncorrect

The points earned by this student would be: ((1.0 + 0.7) – (0.3 + 0.5)) * 50 = 45.

The negative effect of incorrect predictions should prevent students from trying to game the process by blindly assigning 100% probability to every event on the list. But they won’t actually suffer from incorrect predictions, since the scale for the final course grade tops out at 950 and more than 1,000 points will be available from all assignments, quizzes, and exams. If an event on the list happens before the semester ends, I will simply use each student’s most recent forecast to calculate point values.

Since forecasting accuracy improves when estimates are averaged, I will show the class the aggregate results of students’ forecasts each time they are updated. I’m also going to divide the class into teams, with the members each team taking the lead — through additional research and class presentations — on becoming more expert on one topic from the list. Potential events that I might put on the list:

  • The value of the Turkish lira against the U.S. dollar decreases to less than 22:1.
  • In Iran, the Assembly of Experts names a new Supreme Leader.
  • An anti-government protest in Cairo results in at least twenty demonstrators arrested, injured, and/or killed.
  • The president or prime minister of Lebanon is assassinated.
  • Turkey ends its occupation of Syrian territory.

I’m still working out more of the details on this idea, but I think it could be interesting for me and the students.