Simon Fink: Testing the scalability of an EU simulation

Following the Mainz workshop, we are pleased to welcome a guest contributor, Simon Fink of the University of Bamberg. This piece was originally published on Simon’s blog.

Simulations are thought to be “open”. Participants are not re-enacting a predefined script, but are masters of the negotiations. To be sure, participants are not completely free in their actions, but within constraints, they should be able to greatly influence the negotiation outcomes.

But is it true? Could it be the case that simulations have hidden biases, for example, depending on the number of participants, or the duration of the simulation? We cannot see these biases if we look at only a few simulations.

Luckily, I have data from a large number of simulations I did with Eurosoc. At the workshop “EU simulations: Scholarly reflection and research on an innovative teaching methodology“ at Mainz, I presented a paper that uses these data to test whether our EU simulation is truly “open”.

The data contain information about the setting of the simulation (How long did the simulation go, how many participants did we have…?), and about the outcome of the negotiations. As the whole simulation is based on political science data, the negotiation outcomes are expressed in spatial models. That is, we leave the realm of anecdotes about single simulations and have a nice large-n dataset about settings and outcomes.

If you are interested in the whole analysis: Here’s the paper.

A negative (and therefore good) result as a short preview: The duration of the simulation does not influence the content of the negotiated directive.

The independent variable is duration of the simulation: One or two days.

The dependent variable is the content of the directive measured as the mean issue position. The short version (you can find the long version in the paper): We negotiate the chocolate directive, which can be divided into four issues. For each issue, 0 represents the strictest position, 100 represents the most liberal position. Mean issue position is simply the sum of all positions divided by 4. Thus, a directive with a mean issue position of 79 is more liberal than a directive scored 20.

Question: Does the duration of the simulation systematically influence the strictness of the negotiated directive? If yes, this would be bad for the simulation as a teaching tool. What if shorter simulations always produce the same kind of outcomes, different from longer simulations? In that case, shorter simulations are not equivalent to longer simulations.

Answer: There only little relation between the duration and the negotiation outcome of the simulation. The median issue position is 51 for one-day-simulations and 47.5 for two-day-simulations. The only effect is that the variation of outcomes is smaller for two-day-simulations. My interpretation is that longer simulations tend to weed out the more extreme outcomes, as participants have more time for negotiations.

But overall, this result is good news: One-day-simulations have a pattern of outcomes similar to  two-day-simulations. In an analogy to databases or methods, one could say: The simulation scales quite well.