Possible Improvement To Team Research Projects

A follow-up to my recent post about increasing the quality of students’ final products from collaborative research projects:

In my Spring 2021 research methods course, I gave students this outline to follow when writing their team’s research reports. I’ve revised the outline for Spring 2022. Each part in the new outline will get graded separately, with a summative grade for the entire report at the end of the semester.

I’m also thinking of being much more specific about the report’s layout, and grading the reports accordingly — similar to what has worked well with student presentations. I can envision the following criteria:

No more than two pages per part, which would limit the final report to eight pages.

Each part must include at least one data visualization — a chart or graph.

No photographic images.

A Lesson Learned About Team Research Projects

Looking at student performance in the 2020-2021 academic year, I see evidence that team research projects due at the end of the semester can’t be scaffolded solely around individually-graded assignments completed throughout the semester. For example, in my Middle East politics course, each student shared four individually-completed assignments with their teammates for use in their team’s historical timeline. In my research methods course, there were ten individual assignments that teammates were supposed to share with each other as drafts of sections of team research reports. While this approach does decrease free riding and encourage collaboration, it apparently does not ensure high quality research in the final product. Four of the five timelines that teams created in the Middle East course lacked mention of significant events. None of the four teams in the research methods course collected information from coffee farmers, processors, or distributors in Central America, despite my instructions to do so, nor did the final reports resemble the industry exemplars I had provided.

It seems that in students’ minds, my formative assessment of their individual work is totally unconnected to the summative assessment of their collaborative work. I probably need to break the team project into discrete, graded chunks, with each chunk layered on top of some of the individual assignments. Teams can use the feedback they receive on each successive chunk of the project to improve the quality of the final product.

My Latest Iteration of Teammate Evaluations

A brief note about end-of-semester teammate evaluations:

I again used Google Forms to distribute a survey for students to evaluate each other’s contributions to team projects, but I changed how I calculated this component of the course grade. Each student had twelve points to distribute across all team members, including themselves. The more valuable a person’s contribution to the team project, the more points that person was supposed to get. People who made equivalent contributions could have been awarded the same number of points, and if a person was judged as having made no contribution at all, he or she could have been given zero points.

When the Google Form closed, I computed an average number of points received for each student. I then divided this mean score by twelve and multiplied it by fifty (the teammate evaluation was worth 50 out of 1,000 points in the course). I used this formula because teams were larger than in previous semesters, and I assumed a few members of each team would do the heavy lifting with the rest doing little or no work. If the resulting number was fifty or higher, a student earned the full fifty points toward his or her course grade. If the result was below ten, the student earned nothing. For any number in between, I rounded to the nearest ten.

This past semester, I had a total of thirty-seven undergraduate students in two courses. Only thirty completed the evaluation. Four of the thirty completed the survey incorrectly — the scores they distributed across team members did not sum to twelve. I deleted their responses, as I had specified in email and in the Google Form’s directions.

In sum, approximately thirty percent of my students did not perform a simple task that could have benefited their own course grades.

As I speculated at the end of the Fall 2020 semester, I was able to label the teammate evaluation as being worth zero points on Canvas. Maybe that partially explains why no students have (so far) complained about this portion of the course grade.

Changing a Comparative Politics Course, Part 4

My previous post explained how students will complete the template that identifies possible causes of either increased democracy or increased authoritarianism in two nation-states from 2000 to 2020. The next step in this project is for students to work in teams to produce qualitative comparative analyses. Here are my instructions for this collaborative assignment:

Continue reading “Changing a Comparative Politics Course, Part 4”

Fall 2020: Looking Backward and Forward, Part 3

One last post about successes and failures from the previous semester: last summer a colleague pointed me toward Knight Lab Timeline JS, and, inspired by Matthew Wilson’s work on crowd-sourcing and self-instruction, I decided to include a timeline project in my undergraduate course on the Middle East. Setting up the project was relatively simple:

Students were already divided into teams for breakout discussions, presentations, and note-taking; I used the same teams for the timelines. I chose five Middle Eastern countries that featured prominently in assigned readings — Egypt, Iran, Iraq, Lebanon, and Saudi Arabia — and created corresponding files in Google Sheets using the spreadsheet template provide by Knight Lab. I gave each team access to its Google Sheet.

Students completed five graded individual assignments that were designed to prevent free riders and guarantee that teams were working on the project throughout the semester rather than only in a frenzied rush at the end. Here are the instructions for the assignment and its associated rubric:

Continue reading “Fall 2020: Looking Backward and Forward, Part 3”

Storytelling with GIS

Another example of why it’s good to consult with librarians:

For the last several years in my globalization course, I’ve had student teams create and deliver presentations on their commodity chain analyses and ethnographies of consumption. Generally students build Powerpoint files for these assignments; occasionally someone uses Prezi. Simple rubrics make grading this work very easy. But the end products aren’t going to make recent graduates stand out from the competition when interviewing with prospective employers. It’s also difficult to convey the content of the entire project in a single presentation without showing a mind-numbing number of slides. Enter the storymap . . .

One of our librarians, a specialist in digital scholarship whom I’ll be working with next semester, introduced me to the digital storytelling tool from Esri,* a.k.a. the Environmental Systems Research Institute, which allows a person to create a multi-media presentation with ArcGIS. Rather than describe what this looks like, I’ll show you:

My task now is to reconfigure the project’s different assignments so that students complete their own storymaps by the end of the course, and figure out how to evaluate them.

*I have no financial interest in this company.

Changing a Course on Globalization, Part 1

The first of what promises to be a semester-long series on granular-level changes to my globalization course, in which students will be partnering with a local non-profit organization:

Historically my students have been reluctant to evaluate their performance in relation to that of their peers, whether via Monopoly money, rubrics, or anonymous online rank-order surveys. Given that team-based projects account for twenty to twenty-five percent of the final grade in several of my courses, I do think it’s important to ensure some degree of individual accountability in any formally-assessed collaboration between students. No one likes being forced to work with free riders, especially in a course like this one, where students will be conducting research outside of scheduled class time by gathering information from (gasp!) members of the local community. As I have explained to students before — without much success — I can’t be aware of how hard or how well every single one of them has worked with their teammates on a group project, because I’m not always present when the work happens.

So, I racked my brain for a new method of peer evaluation, and came up with the Project Contribution Award:

Please select four people from class, other than yourself, whom you think each deserves 40 points for their outstanding contributions to the project. I will tally the results and the individuals with the most votes will win the award.

Students will submit their choices via an ungraded survey on the Canvas LMS.

The class has only twenty students, so the Project Contribution Award translates into one-fifth of them winning an additional 40 points in a course with a 1,200 point grading scale, a three percent bonus for being perceived by classmates as having performed well on the project. Although I will be dividing students into teams that will take on different responsibilities, the quality of the final product for the community partner will be dependent in some fashion on everyone’s contribution. So I’m hoping that the one in five possibility of earning the award is a sufficient incentive. If not, I can increase the weight of the award in future semesters, or assign one award to each team instead of having multiple awards for the whole class.

A mechanical note: constructing this kind of survey on Canvas requires repeatedly copying and pasting the names of all the students on the class roster. In my case, the survey has four multiple choice questions, each with the same set of twenty names, so the process only took a few minutes. But the inability to generate multiple iterations of a survey or quiz question with a “copy” command probably reduces the utility of Canvas’s quiz feature for courses with large enrollments. No one wants to paste each student’s name over and over again for a 200-student class. In this scenario, I would probably use Google Forms, which does allow the copying of questions. However, I would not be able to just send a link to the survey to students, because responses would be anonymous (allowing students to vote for themselves). Everyone in the class would need Gmail accounts. If your university uses Gmail as its email client, that’s great, but if not, this option requires some extra work on the instructor’s part getting students to create Google accounts with easily-identifiable usernames.

Links to all posts in this series:

Changing a Course on Globalization, Part 1

Changing a Course on Globalization, Part 2

Changing a Course on Globalization, Part 3

Changing a Course on Globalization, Part 4

Changing a Course on Globalization, Part 5

Changing a Course on Globalization, Part 6

Changing Course on Globalization, Part 7

 

 

Making Students Make: Classroom Game Design Part 1

credit: Chad Raymond

In my 2016 first-year seminar, I had teams of students build games, something that originated with a vaguely-defined classroom exercise that I had created on the spur of the moment in class the year before. I’m going to include game design in the course again this fall, but with a few tweaks. Here is an overview of what’s going to happen:

Teams of students will go through three iterations of game design. An individually-written policy memo serves as a preparatory assignment for each round. The respective contexts of the games are the flight of a refugee from a location in South Sudan, the construction and operation of an NGO-managed camp for Rohingya refugees who have fled to Thailand, and the resettlement of a group of Afghan refugees in a relatively small community in the USA. Complete instructions for all of these game design exercises are at TeachersPayTeachers. Teams design their games in class over a few days and then they beta test each other’s games, evaluating them against a rubric. Points from the rubric get added to each student’s grade. Continue reading “Making Students Make: Classroom Game Design Part 1”