Students in my courses do a lot of team-based projects. In an attempt to prevent free riders, I have teammates evaluate each other. This semester I have again modified the evaluation system a bit, with an unexpected result.
For the past three semesters I’ve been using Google Forms to distribute and tabulate anonymous surveys in which students rank themselves and their teammates. The instructions on this survey:
Evaluate the performance of everyone on your team, including yourself, by ranking all members of your team with a different number. Each number can be used only once, otherwise your responses will be discarded. Assign the number 1 to the person who made the most valuable contribution to the project, the number 2 to the person who made the second most valuable contribution, etc. Only enter information for your team. Leave questions for other teams blank.
My explanation to students of how rankings translated into points — in this case, up to 40: Continue reading →
This semester I have formally incorporated a civic engagement project into one of my courses — students have partnered with local business owners to learn about the production and consumption of global commodities. I am new to community partnerships, and I have already learned some useful lessons about communication, expectations, and pedagogical design:
As I wrote in my post on theperils of small classes, this past semester I used Google Forms to create a digital ballot forpresentation competitions. Theballot worked well — students could hide their votes from other students, and tabulating the results only took a few seconds on my part.
The success of the ballot led me to adopt Google Forms for end-of-semesterteammate evaluations. This turned out to be a much simpler method of incorporating individual accountability into collaborative projects than the worksheets I had used previously in afirst-year seminarand acapstone course. No time wasted in class while students complete evaluation forms, and no entering of numerical data into a spreadsheet. I create one form for the whole class, which I send out by embedding the link in one email. Google does the rest for me.
Some students failed to follow directions, but that happens regardless of whether an evaluation is on paper or electronic. As stated on the forms I created, I just deleted those responses from the results.
Some of my courses this semester have such small enrollments that team-based projects and classroom activities have become problematic.
Balloting in presentation competitions is one minor example that I have apparently fixed. I regularly hold classroom competitions in which teams of students present on a topic and then select the winning team by voting with Monopoly money (seehere,here, andherefor more details). When voting, students come to the front of the room and place their money on placards with team numbers written on them — the only rule being that students cannot vote for their own team.
In the first two competitions in one course I noticed little variation in the sums of money received by different teams, even though some presentations were clearly superior to others. I asked the class about this unlikely result. I found out that in such a small class where everyone has interacted in some fashion with everyone else, students did not want to risk publicly embarrassing badly-performing peers or be judged as cruel, so they distributed their money evenly across teams regardless of presentation quality. My solution? Create a ballot on Google Forms through which students can anonymously select the best-performing team. After the last presentation of the day, I email students the link and they submit their responses. Students’ votes now better reflect the quality of the presentations. I also find that it’s easier to tally results with a spreadsheet on Google than to count paper money.
I don’t know a similarly-easy solution to other size-related problems. I recently used an ICONS simulation for the first time (more on that experience in a future post) in a class with only eleven students. Six nation-states comprised the simulation’s actors, so five states were represented by two students and one state by just a single student. If the latter student had been absent on either of the days I ran the simulation, or if two students from the same team had been absent, the exercise would have been wrecked.
In the same course, I had organized a significant portion of classroom time around a sequence of eight topics. For each topic, a student team was supposed to give a mini-lecture and then lead the rest of the class through an activity for some active learning-based peer-to-peer instruction. Then in the subsequent class, students take a quiz on the topic to reinforce the previously-presented material. With only eleven students, I could form only four teams, which left me with the task of delivering the content for half of the topics in the syllabus — defeating the main purpose of the exercise for fifty percent of the time.
Given that my students have priorities in life that often conflict with attending class, I now believe that teams should contain at least four or five students. With only two or three students, there is too high a probability that most or all members of the team will be absent on any particular day. This in fact happened yesterday: only a single member of two different teams attended class.
But if I increase the size of teams in a class with ten to twelve students, that means only two or three teams. Such a small number of teams is too few to expose students to multiple perspectives on a particular topic by means of competitive presentations. It is also too few to adequately incorporate peer instruction across an entire semester, unless the teams that exist are responsible for teaching multiple topics, and this increases the risk that students will be exposed several times to a terribly-performing team.
Why not return individually-completed projects and presentations? I prefer to avoid this option, because it gives students the message that collaboration is unimportant. As I’ve said before, group work matters. For example, individual writing assignments mean that I am the only person who ever reads what students write, which diminishes theauthenticity of assignmentsand leads to lower quality work. And while it may be possible to squeeze as many as six individually-delivered presentations into a 50- or 75-minute time block, a class with six or fewer students doesn’t meet the minimum enrollment requirement at my university. It probably doesn’t at yours either.
The Best X in Town is a project I developed five years ago to find a way of teaching methods that attempted to be fun and interesting for students that were intimidated and bored by the subject material. I presented a poster on it at APSA in 2011, and have been meaning to write it up for the blog for quite a while. I’ve run it five or six times now, in classes of 13-20 that mostly contain political science and international relations majors. Without a doubt it is one of the big reasons that methods is now one of my favorite classes that I teach.
I had three main goals for the project:
Find something with individual and group components that let the students practice several components of the research process;
Base the project around a question that required little advanced knowledge by the students but whose answer they might find interesting.
Provide a theme that helps structure our in-class discussions as well as homework assignments and the final paper.
Basically I wanted a semester-long project that could structure both in-class activities as well as homework assignments and the final course paper that would motivate students to learn methods. I also wanted the project to have nothing to do with politics–partially because methods has so many applications outside of the field, but also to prevent students from being intimidated by doing a project on something they know very little about. In a group setting, this can shut down conversation as students are afraid of saying something wrong. This is already an issue in methods, where they tend to be intimidated from participating anyway, and I did not want to exacerbate it. So I wanted something that would rely on knowledge they already had in their day-to-day lives. Finally, I wanted something to help tie the class together that was not just an individual project. Having a common reference point to discuss various methodological concepts makes it easier for the students to work together to learn.
Enter the Boardwalk Cafe, a local restaurant with a sign in its window proclaiming its status as the ‘Best Breakfast in Town’. Nowhere was listed a source of that accolade–no restaurant review or poll or anything else; it was just a claim by the owners. And it was the perfect claim to serve as the basis for the project–who says they have the best breakfast in town? According to what criteria?
Thus, the project entails answering the question ‘Where is the Best Breakfast* in Town?”
Here are the stages:
Literature Review-as a homework assignment, students have to find 4-6 sources (usually non-scholarly; we discuss the drawbacks there) that will help us define the parameters of one of our three concepts: best, breakfast, and/or town. This might be critic reviews of restaurants, annual polls, studies done on other food items or other towns, etc. They turn in an annotated bibliography of these sources. I compile the citations and make them available to the whole class, as they will need 10 sources in their final papers when they write an actual ‘literature’ review.
Measurement-after we review conceptual and operational definitions, students are divided into 3 groups, one for each of the three concepts, and have to generate operational and conceptual definitions for them. The entire research team ultimately has to agree on those definitions. We do this entirely in class.
Research Design-Next they need to decide how to carry out the research. The two main options are either to do a survey of town residents, or to do observational work collecting data themselves on each establishment. So far, each class I’ve done this with has gone for the second option, at least in part because its less time intensive and has the bonus of letting them sample the food themselves. For this reason I include a note in my syllabus that says that the course may require the outlay of up to $20 each for the project. This method also requires no IRB, as only the students themselves are involved; if you want the survey option to be possible, you may want to check the IRB rules at your institution. This is also a class discussion.
Sample and Population-The next stage is to come up with a sampling frame to find the population of restaurants that meet our definitions, (and/or the people if they are doing a traditional survey) and then do the legwork to carry it out. This has entailed driving around town, using google maps or Yelp, license records, the phone book, etc. They then have to decide whether we are studying the entire population (possible with a narrow definition of town, inconceivable if they do the entire metropolitan area) or a sample, and if the latter, how to sample. They also have to decide how to assign restaurants to individuals if they are going the observational route. This is part discussion, part homework assignment.
Instrument Design-in their three groups they design the instrument to use for either the survey or observations. This is where they practice their skills in survey design. At this point both X/Breakfast and Town are fully defined, so everyone is working on the criteria to establish ‘best’. I usually split them into groups by criteria (most classes have 4-6 criteria for ‘best’ such as taste, atmosphere, service, cost, value, etc), and then each group has a homework assignment to write items on their criteria; we then discuss and edit in class.
Gathering Data–I usually give them 2 weeks to gather the data, including fall/spring break. As a group they have to establish any parameters of the visit–how long they should have to stay, what they have to order, whether others (classmates or friends) are allowed to join them, etc. Some groups establish lots of parameters and reliability checks, others do barely any. The parameters are either an in-class or online discussion; the data gathering itself is done on their own time (although I set up class so they have oral exams during one of the weeks, allowing them to use their ‘off’ day to do their restaurant visits).
Coding–Once I get the data back, I have my research assistant code the data into Excel. You can teach the students to do this as well, but I want a very quick turnaround so I don’t bother.
Data Analysis-I teach the students how to use Excel (choose any stats program you like; Excel is on all the computers in the lab and most students have it, plus learning the basics is a useful skill outside of political science) and basic descriptive analysis and comparison of means.
Final Project–on the 2nd to last day of class, the students turn in a 10-12 page paper that does two things. First, it has to answer the question of Where is the Best Breakfast in Town, including the question, a literature review, a description of the methods, and the full data analysis and answer. The second part requires the student to act as a journal editor and evaluate the paper and project. They have to critique every step of our research process and decide whether they would accept our project for publication, reject it, or require revisions. This second part is crucial, because it allows them to reflect on how decisions made early on (in definitions, sample, or instrument design) affected later stages and in particular, the data they had available to analyze. It also lets students vent about decisions they disagreed with, and lets me test to see how well they’ve grasped the concepts we’ve been learning.
Reward-on the last day of class, I take them out for breakfast to the winning restaurant, and we give a certificate to the manager and take a photo. Its a nice cap to the entire project.
If you want to try this project, feel free to contact me for any of my materials, syllabus, or just to answer questions. I’d love to hear about any other attempts or experiences.
*I now call it the Best X in Town and let the students select the X. It can be parks or movie theaters or cafes or pizza–basically anything that has a bit of ambiguity to it and that is doable given our time and resources.
We talk a lot on this blog about the beauty of simulations for producing rich and varied environments in which to immerse students. We also talk a lot about how ‘failure’ is usually as valid a learning point as ‘success’ in these games: the inability to reach the nominal objective is a valuable learning moment.
But as I’ve been reading for my other work, I realise there’s a different category of political phenomenon that we have more trouble in capturing: the (pre-planned) impasse.
Let me give you the example that triggered this thought, to illustrate.
As you’ll know – if you live in Europe, at least – Greece’s economy has suffered grievously over the past few years, as a result of the 2008 crisis and (more pertinently here) the strict austerity regime imposed by creditors. The election last month of a radical-left government has since led to efforts to recast that austerity regime into something else.
It would be fair to say that all sides are deeply dug in: the Greeks say the current situation is untenable, the creditors say they won’t wear anything else. As so, after a rather brief meeting last night of finance ministers, there was another failure to agree.
Now my cynical side (yes, I do have one) says that this is all part of the game: you act tough, we act tough, then we reach the settlement we knew we would at the outset, but we can assuage our constituents about it all. It’s a bit the logic of my own austerity game, where heads of government have scope to bluster.
However, from a simulation point of view, it’s actually really tricky to recreate this kind of behaviour. In the real world, actors are not bound to a discrete forum of action, nor to a timetable (although there is nominally one in the Greek case): this means their capacity and willingness to not try for a resolution is greater than normal.
If we were trying to run a game of the Greek situation, then we might have a finance ministers’ meeting, but either we reach an agreement (which might not be very realistic), or we don’t (in which case we have to talk about what happens next). What we probably can’t do is run another meeting, and another, so that we can capture the iterative dynamics of it all, nor the way in which we eventually reach compromise.
In short, politics isn’t built for the convenience of the academic calendar.
So what to do? Three ideas present themselves.
First is to just live with it, and note that a game’s failure to reach agreement is lifelike, but also unrealistic in that real-world actors get many more bites of the cherry.
Second is to try to build in rounds of negotiation into a game. The danger here is that you end up covering the same ground multiple times, and that your players might ‘solve’ the problem early on, in which case you have a bunch of time on your hands (although some crisis could easily be invented to upset things).
Third option is to not try to play out the scenario, but rather to get students to apply their knowledge of negotiation/politics/etc to the real-world situation and suggest ways that the different actors could progress things. Those suggestions might be presented as a negotiation brief or some other forum.
This last idea has some interesting advantages. Most importantly, it forces students to articulate negotiation strategy more explicitly (so you can evaluate it), as well as connecting conventional research with an active project.
Finally, trying this last approach is not site-specific: you can use it with a whole range of political events. I’m sure that a moment’s reflection will bring to mind an example of a political institution or process in your area of interest that isn’t working to its full potential, so use that. Easier than building a full simulation, but still with those elements of active engagement that we like so much.
The debate about the future of the EU has been dominated for decades by the dilemma whether the process of European integration should lead to the creation of a sovereign state of Europe or result in a Europe of sovereign states. As recently as 2005 the Belgian politician Paul Magnette published a study under the title: What is the European Union?1 In a rare twist of logic the author refrained from answering the question he posed in the title of his book. Seven years onwards Rosas and Armati concluded their widely acclaimed study on EU Constitutional Law by expressing serious doubt as to whether the EU can ever be defined.2
Friedrich Nietzsche should be remembered for giving his successors a sound piece of advice. The German philosopher suggested that, if you encounter a problem which can not be solved, by no means whatsoever, you should turn in into a game.3 Treading in his footsteps I decided to turn the unidentifiable and unsolvable European Union into a board game, called Eurocracy.
On the board, the EU member states are portrayed according to their size with 1 to 4 cities. Players represent political parties competing for power. They embark on a continuous election campaign through the EU. They obtain Members of the European Parliament (MEPs) by winning municipal elections. Other parties landing on the same city may challenge them in fresh elections, which are democratically decided by the throw of the dice. The may also deal and swap cities in order to get entire countries. The first player to command a qualified majority in the European Parliament, wins the game and is proclaimed President of the European Union…..on condition of sufficient support in the European Parliament.
Eurocracy is played with six persons per game. When played in the class room, the winners on each table are invited to engage in a Speech Battle. They have to deliver a speech in which the propose their priorities on taking office. During this part of the event, the other pupils assume the role of the European Parliament and decide which winner will be the ultimate EU-President.
After play, the teacher/trainer/professor invites the participants to engage in a debate about the EU. Is the EU a state or an organisation of states? What are the characteristics of the Union? Are there differences with other regional organisations and, if so, can you name these differences.
By asking and further questioning, the group arrives at the point where the problem must be faced as to whether the EU fits or does not fit in the Westphalian system of International Relations. This moment constitutes the turning point. It depends on the time available how many minutes or hours are spent on the Westphalian system. It forms the basis of the Organisation of the United Nations and underlies the distribution of international justice. Moreover, it sets the rules for the global financial markets. Currencies function on the basis of the Westphalian paradigm.
However, when the distinguishing qualities of the EU are compared with those of states on the one hand and those of organisations of states on the other hand, it becomes instantly clear that the Union is neither a state nor an international organisation. The implication of this conclusion is that the Westphalian paradigm has lost explanatory value with respect to the EU. As it is a general academic rule that obsolete paradigms have to be replaced, the Westphalian paradigm in international relations should also be substituted with a new one. Since the European Union exists of both citizens and member states, the diplomatic perspective of international relations should be replaced with the civic approach of democracy and the rule of law.
From this point of view, the common feature of all EU member states is their respect for democracy and the rule of law. The famous Copenhagen-criteria for the accession of new member states stress that they should fulfil stringent criteria of human rights and democracy. As the essence of democracy is that the citizens are the authors of the laws, which they themselves must respect, it would appear that this principle has also bearing on the governance of the EU. This line of thought leads to the discovery of the Law of Democratic Integration. Whereas the Westphalian system prescribed that the EU should either become a sovereign state of Europe or establish itself as a Europe of sovereign state, the Law of Democratic Integration holds that, if two or more democratic states decide to share the exercise of sovereignty in a number fields with a view to attain common goals, the organisation they create for this purpose should also be democratic.
At this juncture it becomes possible to draft a Citizens’ Definition of the EU. Beyond the Westphalian paradigm the European Union may be defined as a democratic polity of citizens and member states. The distinctive quality of this polity is that the citizens are entitled to participate in the national democracies of their countries and in the common democracy of the Union.
In conclusion, it may be suggested that the term ‘common democracy’ is of primordial importance for the EU. Whereas the term is void of meaning in the old Westphalian paradigm, it is obvious from the new perspective that the EU is evolving from a common market to a common democracy.