Teammate Evaluations, Revisited

Students in my courses do a lot of team-based projects. In an attempt to prevent free riders, I have teammates evaluate each other. This semester I have again modified the evaluation system a bit, with an unexpected result.fusball-table

For the past three semesters I’ve been using Google Forms to distribute and tabulate anonymous surveys in which students rank themselves and their teammates. The instructions on this survey:

Evaluate the performance of everyone on your team, including yourself, by ranking all members of your team with a different number. Each number can be used only once, otherwise your responses will be discarded. Assign the number 1 to the person who made the most valuable contribution to the project, the number 2 to the person who made the second most valuable contribution, etc. Only enter information for your team. Leave questions for other teams blank.

My explanation to students of how rankings translated into points — in this case, up to 40:

This component of your course grade is determined by how your team ranks your contribution to its performance. The formula for calculating your score is:

(1/∑) * N * 40

where ∑ is the total numerical score you receive on the evaluations and N is the total number of people on your team who voted.

The above method was a straightforward calculation for me. For students, not so much. If everyone on a team ranked the same person as the most valuable, then that person would earn 40 points. But that never happened, and often students earned no more than half of the maximum points possible. Students wondered why, when the outcome was a mathematical artifact of the formula.

This semester I am using a different formula. I sum the scores for each member of a team. The student with the lowest sum (since “1” is “most valuable” on the ranking scale) earns 40 points. Second most valuable student earns 30 points. And so on. A few teams have five members, so the student ranked as least valuable earns only 5 points.

Here is the interesting part: for the first project, members of one team figured out that if they coordinated their rankings, the sums for every member of the team would be identical. I decided to reward their team-like behavior by giving each student the maximum of 40 points. I sent each of them this email:

It looks like your team figured out the puzzle for the teammate evaluation. Take a look at this TED Talk on super chickens.”

I shuffle the composition of teams a bit for each project, and for the second round, word spread about what had happened. Three teams successfully coordinated their teammate rankings, and I sent all of these students the same email. A fourth team failed because one of its members didn’t complete the survey. A fifth team, judging by the survey responses of its members, had not heard how to game the system. For the third project, I anticipate that the members of all of the teams will try to coordinate their rankings.

If I was teaching a course on political theory, I would relate this process to Rousseau’s stag hunt.