Creating Wicked Students 3

Time to reflect on the previous semester’s successes and failures:

I might be on to something with the Wicked Problems that I created for my comparative politics course. Previous posts on the subject are here and here. A brief synopsis of the activity: in class, teams of students have to quickly determine and present a possible solution to an unstructured, authentic problem. I put four of these exercises into the course:

  • Political risk consultants recommend to Volkswagen executives which of two sub-Saharan African states is most suitable for establishing a new automobile manufacturing site and sales network.
  • Defense Intelligence Agency analysts identify which of three Latin American U.S. allies is most susceptible to a Russian GRU election disinformation campaign.
  • The United States Institute for Peace delivers a conference speech on constitutional design for leaders of Libya’s major political parties that compares constitutionally-established institutions of government across four states.
  • Members of Iran’s Mujahedin-e-Khalq create a strategy for overthrowing the Islamic Republic by examining revolutionary movements in four other states.

Students found the exercises engaging. My exams included a question that asked students to reflect on what they learned about their problem-solving ability from each Wicked Problem, and their answers indicated a reasonable degree of meta-cognition.

But it was obvious that students failed to use the methods of comparison that I repeatedly demonstrated during class discussions. I expected students to organize their cases and variables into a simple table, like I had, but they didn’t. So, for example, instead of something like this:

BotswanaNigeria
Ethnically heterogeneousNoYes
Prior civil warNoYes
Major oil exporterNoYes
High level of political riskNoYes

students presented the equivalent of this:

Nigeria has a large population and represents a larger automobile market than Rwanda, so Volkswagen should site its new operation in Nigeria.

I suppose the solution is to require that students create their presentations by filling in a blank table, which will force them to select cases and variables in a logical manner.

TLC @ APSA

This year’s Teaching and Learning Conference will be held on Saturday, September 17, as part of the APSA’s annual meeting in Montreal. Full details on the program and registration process are here. The early bird rate for conference registration ends on July 11.

Scientific Teaching: A Review

A colleague who was cleaning out his office gave me a copy of Scientific Teaching by Jo Handelsman, Sarah Miller, and Christine Pfund (W.H. Freeman and Co., 2008). Intrigued by the title, I gave it a quick read. The book contains some general information on active learning and presents a template for organizing faculty development workshops on topics like assessment, but it was not the guide to effective teaching that I had expected. The book does not discuss empirically-backed research on how people learn. At all.

Instead, Handelsman, Miller, and Pfund favorably discuss learning styles, a zombie educational concept that refuses to die. They heavily reference Learning Styles and Pedagogy in Post-16 Learning: A Systematic and Critical Review by Frank Coffield, David Moseley, Elaine Hall, and Kathryn Ecclestone (Learning and  Skills  Research  Centre, 2004) as support for their argument. In the process, they fundamentally mischaracterize the report’s findings.

For example, on page 9, they write that Coffield et al. (2004) “identified over 70 unique approaches to learning styles . . [that] range from models that explain learning styles as innate . . . ‘flexibly stable’ or . . . that contribute to learning efficacy.” Coffield et al. (2004) state very clearly that these are claims made by those who advocate for the concept of learning styles, not that evidence exists for those claims. In fact, when Coffield et al. (2004) examined thirteen commonly used learning-style inventories, they found that twelve did not meet one or more basic criteria for internal consistency, test-retest reliability, construct validity, and predictive validity. They conclude that the field of learning styles ‘‘is bedeviled by vested interests because some of the leading developers of learning style instruments have themselves conducted the research into the psychometric properties of their own tests, which they are simultaneously offering for sale in the marketplace . . . After more than 30 years of research, no consensus has been reached about the most effective instrument for measuring learning styles and no agreement about the most appropriate pedagogical interventions” (p. 137).

The lack of evidence for the existence learning styles was also discussed in detail by Harold Pashler,  Mark McDaniel, Doug  Rohrer, and Robert Bjork in ‘‘Learning Styles: Concepts and Evidence’’ (Psychological  Science in the Public Interest  9, 3 [2008]). They note in this article that adjusting teaching  techniques  to students’ expressed preferences for particular forms of instruction (i.e., learning styles) does not correlate to observable cognitive or skill aptitudes, and that only a handful of published studies citing the existence of learning  styles had conducted valid experimental tests. The lack of evidence for learning styles was also discussed in this 2009 interview with the cognitive psychologist Daniel Willingham.

In sum, Scientific Teaching‘s reliance on a concept that was widely discredited both before and soon after its publication renders it misleading and, therefore, useless.

The Marshmallow Tower Game

Along the lines of my last post, I’ve tweaked another game that I have used previously — the marshmallow challenge. My goal was to illustrate how economic development can be considered a collective action problem in which trust plays a key role. Here are the rules of the game:

  • Each team has 18 minutes to build a tower topped by a marshmallow using the materials provided.
  • The members of the team that builds the tallest tower earn 25 points each.
  • A “Red” player secretly placed on your team gets 25 points if their real team wins.
  • If a team correctly identifies its Red player, each team member wins 25 points. Only one guess per team.

The debriefing discussion included my brief description of Rousseau’s stag hunt scenario, and these questions:

  • If one considers the height of a tower as an indicator of a society’s level of economic development, why did some societies (teams) develop more quickly than others?
  • Did cultural values promote trust among team members?
  • What was in each person’s best interest? Were these interests achieved?
  • How did having a Red on your team affect your team’s behavior?
  • Who do you think the Reds were? Why?
  • How does it feel to be accused of being a Red?

At the very end of the discussion, I revealed that there were no Red players.

The class had ten students that I divided into three teams. One team’s tower collapsed when time expired, but none of the teams exhibited a high degree of dysfunction due to suspicions about the identity of its Red player. As usual, I think the game would work better in a class with more students.

The Bandit Game

In an attempt to rectify the failure of my previous classroom game on ethnic heterogeneity, democracy and dictatorship, I created another game that included a loss aversion component. I intended the game to demonstrate the concepts found in Mancur Olson’s 1993 article, “Dictatorship, Democracy, and Development” (The American Political Science Review 87, 3). Here are the rules for game’s initial version:

  • Each person gets a playing card and 4 chips.
  • The class is divided into small groups.
  • The person with the highest card value in each group is a bandit.
  • The game has five rounds.
  • Each group’s bandit confiscates 1, 2, 3, or 4 chips each round from every other group member. This decision is made by the bandit. The bandit has to confiscate at least 1 chip from each group member each round, assuming the group member has a chip.
  • After round 1, 2, 3, and 4, each non-bandit gets 1 additional chip if they have ended the round with > 0 chips.
  • The person in each group with the most chips after round 5 earns points equivalent to the number of chips in their possession.

Version 2 of the game has the same rules as Version 1, plus:

  • A bandit can switch to a different group after each of rounds 1-4. The bandit with a higher value card turns another group’s bandit into an ordinary person.
  • The new bandit takes the eliminated bandit’s chips and can keep them or distribute some or all of them in any manner to members of their new group.

Version 3 has the same rules as Versions 1 and 2, plus:

  • Members of a group can eliminate a bandit if (a) they have card suits different from the bandit’s suit, and (b) the combined value of their cards exceeds the value of the bandit’s card. If a bandit is eliminated, the bandit’s chips are distributed equally among the challengers.
  • A bandit can retain control if (a) group members with cards of the same suit as the bandit’s decide to ally with the bandit and (b) the combined value of cards of this suit exceeds that of the bandit’s challengers.

Before play started, I stacked the deck with cards from only three suits because of the small class size — thirteen students are registered for the course, but only eleven showed up. I divided these eleven students into three groups.

For all versions of the game, all bandits confiscated the same number of chips from their group’s members in each round, even though the rules did not specify that they had to do this. In Version 1, one bandit confiscated all the chips from every group member in one round, which ended that group’s game play for the remaining rounds — demonstrating that it’s better for a stationary bandit to extract only a portion of wealth from the populace at any given time. During Version 2, no bandit changed groups, and in Version 3, no one tried to eliminate a bandit.

This game worked better than the last one, but it still needs a much larger number of participants for it to function as intended.

Serendipity in Research Methods

Sometimes it is easier to demonstrate real-world relevance than others.

Last week students in my research methods course read Charles Wheelan, Naked Statistics, Ch. 12, and Ashley A. Smith, “Students Taking More Credit Courses and Introductory Math Faring Well,” Inside Higher Ed, 7 December 2018.

They then had to answer this question: What mistakes are Nevada officials making with data about community college students?

As written, the Inside Higher Ed story describes people who should know better falling victim to omitted variable bias and confusing correlation with causation. Although I might be making similar mistakes in evaluating in-class discussion about the assignment, I think that students found it more interesting than most because the assignment was about other students.

Soon afterward, two similar items came across my radar:

Students prefer mixing and matching online with on-campus courses.

Common premises about college students are wrong.

I shared these with my students, as additional examples of analyzing (or not) data about their peers.

Study Abroad As Active Learning

Alternative title for this post: What I Did Over Spring Break.

Photo credit:
Chad Raymond

One of the benefits of having a joint appointment in an interdisciplinary department is being able to participate in field research initiatives. Two weeks ago, I was in Belize as a co-leader for a tropical biology course examining the effects of agricultural development on biodiversity. This was a great opportunity to observe and teach about intersections between economic development, environmental sustainability, and public policy — while getting one’s hands dirty among the scorpions, bats, snakes, and octopi.

A few random observations:

Undergraduate political science programs with curricula that exclude experiential learning in favor of scholasticism are really doing their students a disservice. Philosophical treatises are no longer the sole repository of knowledge.

American provincialism did manifest itself, but only occasionally and, in my opinion, innocuously. For example, as this blog’s non-U.S. audience probably already knows, my people are generally at a loss in non-English language environments. But despite English being the official language of Belize, it is English with a Belizean accent. And in everyday conversation among locals, Belizean Creole is used — often mixed with Spanish. Occasionally I asked students, “Did you understand that?” and their answer was “no.” I would then point out that American English is not the only form of English, and that they needed to train themselves for that reality. The few among us competent in Spanish definitely had an advantage when we encountered people who did not speak any version of English. For readers in the U.S., does your political science department require proficiency in a second language? It should. Politics are global.

I was impressed by the students’ initiative and willingness to try to independently solve problems on the fly while doing their research. They were also relatively unperturbed by the conditions. Maybe this is due to self-selection for this specific study abroad program, but whatever the reason, it made my job much easier.

In-country logistics ran like clockwork, mainly due to the talented local program organizer. It was only when returning to the USA, a Third World country, that we ran into problems — a flight delay, a mad scramble through Miami airport, a missed connecting flight, an interminable rebooking process, not all of us finding seats on the last flight of the day, and a portion of our group forced to overnight in an airport hotel. But everyone did return home in the end. Lesson to students: be prepared in case things don’t go according to plan.

When a Game Fails

An inadvertent update to a 2015 post on the perils of small classes:

I recently ran a game in two classes that I had hoped would demonstrate the effects of ethnic heterogeneity in dictatorships and democracies. The basic mechanics of the game:

The class is split into groups. Each person gets a playing card. Card suit represents ethnicity, though I didn’t tell students this. A card’s numeric value equates to the power level of the person holding it. If someone in a group has a face card, then the group is a dictatorship. The person in the group with the highest value face card is the dictator, who makes all decisions. If no one in the group has a face card, then the group is a democracy, with decisions made by majority vote. The numeric values of the cards don’t matter.

The game is played in multiple rounds, with a greater number of points at stake in each round — I used five rounds, worth 3, 5, 7, 10, and 15 points, respectively. These points count toward the final course grade. In every round, each group allocates its points to its members according to the rules above. If anyone in a group is dissatisfied with how the points were distributed, the person can recruit a cluster of allies who have cards of the same suit to challenge the distribution. In a dictatorship, the challenge succeeds if the cluster’s combined power level exceeds that formed by the dictator’s allies. In a democracy, the challenge succeeds if the cluster’s total power level exceeds that of the rest of the group. When there is a successful challenge, the group has to distribute its points in a different way. Each round had a time limit of just a few minutes, and if a group failed to successfully allocate its points before a round ended, the group’s points for that round disappeared.

Continue reading “When a Game Fails”

Creating Wicked Students 2

As promised last week, here is an example of a wicked problem I’ve given to my comparative politics class.

  • You are an employee of the The Scowcroft Group.
  • Volkswagen wants to expand into a new African market.
  • Setting up production facilities and distribution channels will take three years.
  • Which sub-Saharan African country should Volkswagen choose to expand into?
  • Your task is to compare risk to political stability for two sub-Saharan African nation-states, and choose the one with the least risk.
  • Use ≥ 1 quantitative and ≥ 1 non-quantitative indicator.
  • Present your recommendation to Volkswagen’s CEO and board of directors.
  • You have 15 minutes to create a 3 minute presentation.

I show the instructions, small teams of students work on the problem, and each team presents its solution. I grade the presentations using this rubric:

Creating Wicked Students

I recently read Creating Wicked Students : Designing Courses for a Complex World by Paul Hanstedt (Stylus Publishing, 2018). Hanstedt argues, I think correctly, that:

“most of our students are not like us . . . most professors are highly autonomous in their learning, interested in high levels of abstraction, and intrinsically motivated when it comes to their fields. This is not the case for most of our students. Some of them simply view our classes as a hoop they have to jump through. Others don’t understand what all the fuss is about, why these ideas are so much more important than, say, real life; and others just struggle. They may have come to college without the tools we had or without the preparation to master high levels of thinking and reams of content. Almost all of them have been shaped by a testing culture that puts an emphasis on content mastery over conceptual thinking” (p. 44).

Students therefore need to be “forced to take responsibility for their own learning . . . There is really only one way for people to gain authority: They must assume it, repeatedly and often , . . That sense that one is capable of engaging in complex problem-solving can only come from solving complex problems” (p. 65-66).

These complex problems are unstructured, require the transfer of knowledge from one context to another, and are authentic — making them very different from the academic tasks that students typically encounter.

I found the book thought provoking and decided to try including some wicked problems in my comparative politics course. I’ll put an example in my next post.