As I mentioned in my last post about changes to my globalization course, my original plan of assigning an ethnography in conjunction with a project for a community partner no longer seemed likely to serve its intended purpose, so I removed it mid-semester. As a replacement, I have assigned students the task of creating infographics, first individually, and then in teams of four. I will turn over the latter products to the community partner as one of the deliverables from the project. Directions for the individual assignment are as follows: Continue reading “Changing a Course on Globalization, Part 5”
Changing a Course on Globalization, Part 4
The semester is half over, and it has become apparent that I need to make some on-the-spot changes to my globalization course. The first change is quite minor: students have added or dropped the course, necessitating an edit to my Canvas LMS survey for the Project Contribution Award. As I mentioned previously, the mechanics of this procedure would be extremely time-consuming with a large class.
The second change is much more . . . extensive. As part of a foundation grant, the class is formally partnered with a local non-profit organization, Aquidneck Community Table (ACT). Students are collecting and analyzing data on the food consumption patterns of local residents by means of face-to-face interviews and supermarket receipts. Course assignments related to this project include a food ethnography and a single class-wide report for ACT.
The food ethnography is essentially the same as the ethnography of consumption that I have used in past iterations of the course (discussed here and in the bullet points here). My instructions for the ethnography this semester: Continue reading “Changing a Course on Globalization, Part 4”
Changing a Course on Globalization, Part 1
The first of what promises to be a semester-long series on granular-level changes to my globalization course, in which students will be partnering with a local non-profit organization:
Historically my students have been reluctant to evaluate their performance in relation to that of their peers, whether via Monopoly money, rubrics, or anonymous online rank-order surveys. Given that team-based projects account for twenty to twenty-five percent of the final grade in several of my courses, I do think it’s important to ensure some degree of individual accountability in any formally-assessed collaboration between students. No one likes being forced to work with free riders, especially in a course like this one, where students will be conducting research outside of scheduled class time by gathering information from (gasp!) members of the local community. As I have explained to students before — without much success — I can’t be aware of how hard or how well every single one of them has worked with their teammates on a group project, because I’m not always present when the work happens.
So, I racked my brain for a new method of peer evaluation, and came up with the Project Contribution Award:
Please select four people from class, other than yourself, whom you think each deserves 40 points for their outstanding contributions to the project. I will tally the results and the individuals with the most votes will win the award.
Students will submit their choices via an ungraded survey on the Canvas LMS.
The class has only twenty students, so the Project Contribution Award translates into one-fifth of them winning an additional 40 points in a course with a 1,200 point grading scale, a three percent bonus for being perceived by classmates as having performed well on the project. Although I will be dividing students into teams that will take on different responsibilities, the quality of the final product for the community partner will be dependent in some fashion on everyone’s contribution. So I’m hoping that the one in five possibility of earning the award is a sufficient incentive. If not, I can increase the weight of the award in future semesters, or assign one award to each team instead of having multiple awards for the whole class.
A mechanical note: constructing this kind of survey on Canvas requires repeatedly copying and pasting the names of all the students on the class roster. In my case, the survey has four multiple choice questions, each with the same set of twenty names, so the process only took a few minutes. But the inability to generate multiple iterations of a survey or quiz question with a “copy” command probably reduces the utility of Canvas’s quiz feature for courses with large enrollments. No one wants to paste each student’s name over and over again for a 200-student class. In this scenario, I would probably use Google Forms, which does allow the copying of questions. However, I would not be able to just send a link to the survey to students, because responses would be anonymous (allowing students to vote for themselves). Everyone in the class would need Gmail accounts. If your university uses Gmail as its email client, that’s great, but if not, this option requires some extra work on the instructor’s part getting students to create Google accounts with easily-identifiable usernames.
Links to all posts in this series:
Making Students Make: Classroom Game Design Part 1
In my 2016 first-year seminar, I had teams of students build games, something that originated with a vaguely-defined classroom exercise that I had created on the spur of the moment in class the year before. I’m going to include game design in the course again this fall, but with a few tweaks. Here is an overview of what’s going to happen:
Teams of students will go through three iterations of game design. An individually-written policy memo serves as a preparatory assignment for each round. The respective contexts of the games are the flight of a refugee from a location in South Sudan, the construction and operation of an NGO-managed camp for Rohingya refugees who have fled to Thailand, and the resettlement of a group of Afghan refugees in a relatively small community in the USA. Complete instructions for all of these game design exercises are at TeachersPayTeachers. Teams design their games in class over a few days and then they beta test each other’s games, evaluating them against a rubric. Points from the rubric get added to each student’s grade. Continue reading “Making Students Make: Classroom Game Design Part 1”
Another End of Semester Reflection
A few thoughts about the semester that is about to end:
First, the “open notebook” in-class quizzes did cause many more students to take notes than in the past. However, creating the quizzes — with most of the questions based on prior class discussion — was a pain. Same for printing and grading them. For three undergraduate courses with small enrollments, the task wasn’t very complex, nor did it require a lot of effort, but it did become yet one more thing I had to organize throughout the semester. Also I gave fewer quizzes than I originally anticipated, which forced me to alter their weights in the final course grade. I’m now thinking that I could deliver the quizzes more easily and frequently through our Canvas LMS — they would be machine graded. But I would still need to invest in designing questions and building each quiz throughout the semester. A more rigid, pre-planned system for class discussions would allow me to generate all the quizzes before the semester starts, but I really don’t want to do this because it would move me back in the direction of lecturing. Continue reading “Another End of Semester Reflection”
Teammate Evaluations, Revisited
Students in my courses do a lot of team-based projects. In an attempt to prevent free riders, I have teammates evaluate each other. This semester I have again modified the evaluation system a bit, with an unexpected result.
For the past three semesters I’ve been using Google Forms to distribute and tabulate anonymous surveys in which students rank themselves and their teammates. The instructions on this survey:
Evaluate the performance of everyone on your team, including yourself, by ranking all members of your team with a different number. Each number can be used only once, otherwise your responses will be discarded. Assign the number 1 to the person who made the most valuable contribution to the project, the number 2 to the person who made the second most valuable contribution, etc. Only enter information for your team. Leave questions for other teams blank.
My explanation to students of how rankings translated into points — in this case, up to 40: Continue reading “Teammate Evaluations, Revisited”
Learning Lessons About Civic Engagement
This semester I have formally incorporated a civic engagement project into one of my courses — students have partnered with local business owners to learn about the production and consumption of global commodities. I am new to community partnerships, and I have already learned some useful lessons about communication, expectations, and pedagogical design:
Google Forms
As I wrote in my post on the perils of small classes, this past semester I used Google Forms to create a digital ballot for presentation competitions. The ballot worked well — students could hide their votes from other students, and tabulating the results only took a few seconds on my part.
The success of the ballot led me to adopt Google Forms for end-of-semester teammate evaluations. This turned out to be a much simpler method of incorporating individual accountability into collaborative projects than the worksheets I had used previously in a first-year seminar and a capstone course. No time wasted in class while students complete evaluation forms, and no entering of numerical data into a spreadsheet. I create one form for the whole class, which I send out by embedding the link in one email. Google does the rest for me.
Some students failed to follow directions, but that happens regardless of whether an evaluation is on paper or electronic. As stated on the forms I created, I just deleted those responses from the results.
Does Size Matter? The Perils of Small Classes
Some of my courses this semester have such small enrollments that team-based projects and classroom activities have become problematic.
Balloting in presentation competitions is one minor example that I have apparently fixed. I regularly hold classroom competitions in which teams of students present on a topic and then select the winning team by voting with Monopoly money (see here, here, and here for more details). When voting, students come to the front of the room and place their money on placards with team numbers written on them — the only rule being that students cannot vote for their own team.
In the first two competitions in one course I noticed little variation in the sums of money received by different teams, even though some presentations were clearly superior to others. I asked the class about this unlikely result. I found out that in such a small class where everyone has interacted in some fashion with everyone else, students did not want to risk publicly embarrassing badly-performing peers or be judged as cruel, so they distributed their money evenly across teams regardless of presentation quality. My solution? Create a ballot on Google Forms through which students can anonymously select the best-performing team. After the last presentation of the day, I email students the link and they submit their responses. Students’ votes now better reflect the quality of the presentations. I also find that it’s easier to tally results with a spreadsheet on Google than to count paper money.
I don’t know a similarly-easy solution to other size-related problems. I recently used an ICONS simulation for the first time (more on that experience in a future post) in a class with only eleven students. Six nation-states comprised the simulation’s actors, so five states were represented by two students and one state by just a single student. If the latter student had been absent on either of the days I ran the simulation, or if two students from the same team had been absent, the exercise would have been wrecked.
In the same course, I had organized a significant portion of classroom time around a sequence of eight topics. For each topic, a student team was supposed to give a mini-lecture and then lead the rest of the class through an activity for some active learning-based peer-to-peer instruction. Then in the subsequent class, students take a quiz on the topic to reinforce the previously-presented material. With only eleven students, I could form only four teams, which left me with the task of delivering the content for half of the topics in the syllabus — defeating the main purpose of the exercise for fifty percent of the time.
Given that my students have priorities in life that often conflict with attending class, I now believe that teams should contain at least four or five students. With only two or three students, there is too high a probability that most or all members of the team will be absent on any particular day. This in fact happened yesterday: only a single member of two different teams attended class.
But if I increase the size of teams in a class with ten to twelve students, that means only two or three teams. Such a small number of teams is too few to expose students to multiple perspectives on a particular topic by means of competitive presentations. It is also too few to adequately incorporate peer instruction across an entire semester, unless the teams that exist are responsible for teaching multiple topics, and this increases the risk that students will be exposed several times to a terribly-performing team.
Why not return individually-completed projects and presentations? I prefer to avoid this option, because it gives students the message that collaboration is unimportant. As I’ve said before, group work matters. For example, individual writing assignments mean that I am the only person who ever reads what students write, which diminishes the authenticity of assignments and leads to lower quality work. And while it may be possible to squeeze as many as six individually-delivered presentations into a 50- or 75-minute time block, a class with six or fewer students doesn’t meet the minimum enrollment requirement at my university. It probably doesn’t at yours either.
The Best X in Town: A Project for Teaching Research Methods
The Best X in Town is a project I developed five years ago to find a way of teaching methods that attempted to be fun and interesting for students that were intimidated and bored by the subject material. I presented a poster on it at APSA in 2011, and have been meaning to write it up for the blog for quite a while. I’ve run it five or six times now, in classes of 13-20 that mostly contain political science and international relations majors. Without a doubt it is one of the big reasons that methods is now one of my favorite classes that I teach.
I had three main goals for the project:
- Find something with individual and group components that let the students practice several components of the research process;
-
Base the project around a question that required little advanced knowledge by the students but whose answer they might find interesting.
-
Provide a theme that helps structure our in-class discussions as well as homework assignments and the final paper.
Basically I wanted a semester-long project that could structure both in-class activities as well as homework assignments and the final course paper that would motivate students to learn methods. I also wanted the project to have nothing to do with politics–partially because methods has so many applications outside of the field, but also to prevent students from being intimidated by doing a project on something they know very little about. In a group setting, this can shut down conversation as students are afraid of saying something wrong. This is already an issue in methods, where they tend to be intimidated from participating anyway, and I did not want to exacerbate it. So I wanted something that would rely on knowledge they already had in their day-to-day lives. Finally, I wanted something to help tie the class together that was not just an individual project. Having a common reference point to discuss various methodological concepts makes it easier for the students to work together to learn.
Enter the Boardwalk Cafe, a local restaurant with a sign in its window proclaiming its status as the ‘Best Breakfast in Town’. Nowhere was listed a source of that accolade–no restaurant review or poll or anything else; it was just a claim by the owners. And it was the perfect claim to serve as the basis for the project–who says they have the best breakfast in town? According to what criteria?
Thus, the project entails answering the question ‘Where is the Best Breakfast* in Town?”
Here are the stages:
- Literature Review-as a homework assignment, students have to find 4-6 sources (usually non-scholarly; we discuss the drawbacks there) that will help us define the parameters of one of our three concepts: best, breakfast, and/or town. This might be critic reviews of restaurants, annual polls, studies done on other food items or other towns, etc. They turn in an annotated bibliography of these sources. I compile the citations and make them available to the whole class, as they will need 10 sources in their final papers when they write an actual ‘literature’ review.
-
Measurement-after we review conceptual and operational definitions, students are divided into 3 groups, one for each of the three concepts, and have to generate operational and conceptual definitions for them. The entire research team ultimately has to agree on those definitions. We do this entirely in class.
-
Research Design-Next they need to decide how to carry out the research. The two main options are either to do a survey of town residents, or to do observational work collecting data themselves on each establishment. So far, each class I’ve done this with has gone for the second option, at least in part because its less time intensive and has the bonus of letting them sample the food themselves. For this reason I include a note in my syllabus that says that the course may require the outlay of up to $20 each for the project. This method also requires no IRB, as only the students themselves are involved; if you want the survey option to be possible, you may want to check the IRB rules at your institution. This is also a class discussion.
-
Sample and Population-The next stage is to come up with a sampling frame to find the population of restaurants that meet our definitions, (and/or the people if they are doing a traditional survey) and then do the legwork to carry it out. This has entailed driving around town, using google maps or Yelp, license records, the phone book, etc. They then have to decide whether we are studying the entire population (possible with a narrow definition of town, inconceivable if they do the entire metropolitan area) or a sample, and if the latter, how to sample. They also have to decide how to assign restaurants to individuals if they are going the observational route. This is part discussion, part homework assignment.
-
Instrument Design-in their three groups they design the instrument to use for either the survey or observations. This is where they practice their skills in survey design. At this point both X/Breakfast and Town are fully defined, so everyone is working on the criteria to establish ‘best’. I usually split them into groups by criteria (most classes have 4-6 criteria for ‘best’ such as taste, atmosphere, service, cost, value, etc), and then each group has a homework assignment to write items on their criteria; we then discuss and edit in class.
-
Gathering Data–I usually give them 2 weeks to gather the data, including fall/spring break. As a group they have to establish any parameters of the visit–how long they should have to stay, what they have to order, whether others (classmates or friends) are allowed to join them, etc. Some groups establish lots of parameters and reliability checks, others do barely any. The parameters are either an in-class or online discussion; the data gathering itself is done on their own time (although I set up class so they have oral exams during one of the weeks, allowing them to use their ‘off’ day to do their restaurant visits).
-
Coding–Once I get the data back, I have my research assistant code the data into Excel. You can teach the students to do this as well, but I want a very quick turnaround so I don’t bother.
-
Data Analysis-I teach the students how to use Excel (choose any stats program you like; Excel is on all the computers in the lab and most students have it, plus learning the basics is a useful skill outside of political science) and basic descriptive analysis and comparison of means.
-
Final Project–on the 2nd to last day of class, the students turn in a 10-12 page paper that does two things. First, it has to answer the question of Where is the Best Breakfast in Town, including the question, a literature review, a description of the methods, and the full data analysis and answer. The second part requires the student to act as a journal editor and evaluate the paper and project. They have to critique every step of our research process and decide whether they would accept our project for publication, reject it, or require revisions. This second part is crucial, because it allows them to reflect on how decisions made early on (in definitions, sample, or instrument design) affected later stages and in particular, the data they had available to analyze. It also lets students vent about decisions they disagreed with, and lets me test to see how well they’ve grasped the concepts we’ve been learning.
-
Reward-on the last day of class, I take them out for breakfast to the winning restaurant, and we give a certificate to the manager and take a photo. Its a nice cap to the entire project.
If you want to try this project, feel free to contact me for any of my materials, syllabus, or just to answer questions. I’d love to hear about any other attempts or experiences.
*I now call it the Best X in Town and let the students select the X. It can be parks or movie theaters or cafes or pizza–basically anything that has a bit of ambiguity to it and that is doable given our time and resources.