The Best X in Town: A Project for Teaching Research Methods

The Best X in Town is a project I developed five years ago to find a way of teaching methods that attempted to be fun and interesting for students that were intimidated and bored by the subject material. I presented a poster on it at APSA in 2011, and have been meaning to write it up for the blog for quite a while.  I’ve run it five or six times now, in classes of 13-20 that mostly contain political science and international relations majors.  Without a doubt it is one of the big reasons that methods is now one of my favorite classes that I teach.

I had three main goals for the project:

1. Find something with individual and group components that let the students practice several components of the research process;

2.  Base the project around a question that required little advanced knowledge by the students but whose answer they might find interesting.

3. Provide a theme that helps structure our in-class discussions as well as homework assignments and the final paper.

Basically I wanted a semester-long project that could structure both in-class activities as well as homework assignments and the final course paper that would motivate students to learn methods.  I also wanted the project to have nothing to do with politics–partially because methods has so many applications outside of the field, but also to prevent students from being intimidated by doing a project on something they know very little about. In a group setting, this can shut down conversation as students are afraid of saying something wrong.  This is already an issue in methods, where they tend to be intimidated from participating anyway, and I did not want to exacerbate it. So I wanted something that would rely on knowledge they already had in their day-to-day lives.  Finally, I wanted something to help tie the class together that was not just an individual project.  Having a common reference point to discuss various methodological concepts makes it easier for the students to work together to learn.

Enter the Boardwalk Cafe, a local restaurant with a sign in its window proclaiming its status as the ‘Best Breakfast in Town’. Nowhere was listed a source of that accolade–no restaurant review or poll or anything else; it was just a claim by the owners.  And it was the perfect claim to serve as the basis for the project–who says they have the best breakfast in town?  According to what criteria?

Thus, the project entails answering the question ‘Where is the Best Breakfast* in Town?”

Here are the stages:

1. Literature Review-as a homework assignment, students have to find 4-6 sources (usually non-scholarly; we discuss the drawbacks there) that will help us define the parameters of one of our three concepts: best, breakfast, and/or town.  This might be critic reviews of restaurants, annual polls, studies done on other food items or other towns, etc.  They turn in an annotated bibliography of these sources.  I compile the citations and make them available to the whole class, as they will need 10 sources in their final papers when they write an actual ‘literature’ review.

2. Measurement-after we review conceptual and operational definitions, students are divided into 3 groups, one for each of the three concepts, and have to generate operational and conceptual definitions for them.  The entire research team ultimately has to agree on those definitions.  We do this entirely in class.

3. Research Design-Next they need to decide how to carry out the research.  The two main options are either to do a survey of town residents, or to do observational work collecting data themselves on each establishment.  So far, each class I’ve done this with has gone for the second option, at least in part because its less time intensive and has the bonus of letting them sample the food themselves. For this reason I include a note in my syllabus that says that the course may require the outlay of up to $20 each for the project.  This method also requires no IRB, as only the students themselves are involved; if you want the survey option to be possible, you may want to check the IRB rules at your institution.  This is also a class discussion.

4. Sample and Population-The next stage is to  come up with a sampling frame to find the population of restaurants that meet our definitions, (and/or the people if they are doing a traditional survey) and then do the legwork to carry it out.  This has entailed driving around town, using google maps or Yelp, license records, the phone book, etc.  They then have to decide whether we are studying the entire population (possible with a narrow definition of town, inconceivable if they do the entire metropolitan area) or a sample, and if the latter, how to sample.  They also have to decide how to assign restaurants to individuals if they are going the observational route. This is part discussion, part homework assignment.

5. Instrument Design-in their three groups they design the instrument to use for either the survey or observations.  This is where they practice their skills in survey design.  At this point both X/Breakfast and Town are fully defined, so everyone is working on the criteria to establish ‘best’. I usually split them into groups by criteria (most classes have 4-6 criteria for ‘best’ such as taste, atmosphere, service, cost, value, etc), and then each group has a homework assignment to write items on their criteria; we then discuss and edit in class.

6. Gathering Data–I usually give them 2 weeks to gather the data, including fall/spring break. As a group they have to establish any parameters of the visit–how long they should have to stay, what they have to order, whether others (classmates or friends) are allowed to join them, etc.  Some groups establish lots of parameters and reliability checks, others do barely any. The parameters are either an in-class or online discussion; the data gathering itself is done on their own time (although I set up class so they have oral exams during one of the weeks, allowing them to use their ‘off’ day to do their restaurant visits).

7. Coding–Once I get the data back, I have my research assistant code the data into Excel.  You can teach the students to do this as well, but I want a very quick turnaround so I don’t bother.

8. Data Analysis-I teach the students how to use Excel (choose any stats program you like; Excel is on all the computers in the lab and most students have it, plus learning the basics is a useful skill outside of political science) and basic descriptive analysis and comparison of means.

9. Final Project–on the 2nd to last day of class, the students turn in a 10-12 page paper that does two things.  First, it has to answer the question of Where is the Best Breakfast in Town, including the question, a literature review, a description of the methods, and the full data analysis and answer.  The second part requires the student to act as a journal editor and evaluate the paper and project.  They have to critique every step of our research process and decide whether they would accept our project for publication, reject it, or require revisions.  This second part is crucial, because it allows them to reflect on how decisions made early on (in definitions, sample, or instrument design) affected later stages and in particular, the data they had available to analyze.  It also lets students vent about decisions they disagreed with, and lets me test to see how well they’ve grasped the concepts we’ve been learning.

10. Reward-on the last day of class, I take them out for breakfast to the winning restaurant, and we give a certificate to the manager and take a photo.  Its a nice cap to the entire project.

If you want to try this project, feel free to contact me for any of my materials, syllabus, or just to answer questions. I’d love to hear about any other attempts or experiences.

*I now call it the Best X in Town and let the students select the X.  It can be parks or movie theaters or cafes or pizza–basically anything that has a bit of ambiguity to it and that is doable given our time and resources.