Continuing to review my fall semester . . .
The forecasting project might have helped students learn Middle East politics and history. I’d rate it as a success on that front. As to whether their decision making skills have improved from using the CHAMP method, who knows?
At five different points in the semester, students forecasted the likelihood of these events occurring by December 9:
- The value of the Turkish lira against the U.S. dollar decreases to less than 22:1.
- In Iran, the Assembly of Experts names a new Supreme Leader.
- An anti-government protest in Cairo results in at least twenty demonstrators arrested, injured, and/or killed.
- The president or prime minister of Lebanon is assassinated.
- Turkey ends its occupation of Syrian territory.
None of these events happened before the deadline, but that was ok given my purposes for the project. Here are the class’s predictions, with average percentage probability on the y-axis:
I need to tweak some of the project’s components. For example, the prompt for the last individual assignment — assess how your forecasts have been affected by cognitive biases — included this statement:
“People like Daniel Kahneman, Charles Wheelan, Tim Harford, Gerd Gigerenzer, and Nassim Taleb have written about cognitive biases and how to counter their effects.”
A few students did not discuss cognitive biases at all. Others clearly did a bad job of Googling “cognitive biases” and what the above individuals have written about them. In the future I’ll need to assign a specific reading on the topic. I see this as another manifestation of student inability or unwillingness to find information that I don’t put right in front of them.
Similarly, I either need to discard the in-class team presentations or formally assess them. Overall, they were of poor quality. Students need an explicit, rigid template for constructing presentations, and students will follow the template only if the presentations are graded. Asking students to give informal, ungraded presentations simply doesn’t work. Given that this country has raised a generation of children who frequently suffer from anxiety disorders, I might need to institute a rule that credit for presentations only goes to the students who deliver them, with the condition that each member of a team can present if they so choose. I already design my courses to provide students with “multiple paths to success,” so optional-yet-graded presentations are not much of a complication for me.
I administered my own course evaluation at the end of the semester. Here are the results — from 20 out a class of 22 students — for questions with a scale from “strongly agree” (5) to “strongly disagree” (1):
- The forecasting project improved my ability to analyze political events in the Middle East – 3.9
- I am now better able to use forecasting as a decision making tool in my own life – 3.7
- More courses should include training in decision making skills like forecasting – 3.4
I would like the average scores on the second and third items to be higher.
Final comment: the last two reading response assignments before the final exam asked students to respond to “Will Lebanon/Syria still be a single sovereign state in 2030?” I did not realize until the last week of classes that these questions dovetail perfectly with the forecasting project, and that I should somehow integrate the CHAMP method and reading responses so that students get more opportunities to hone their decision making skills.