Today I want to discuss the Methods Silo Effect: the belief that a single methods class or sequence is sufficient to teach our students the skills of a political scientist. Following this course, no more instruction in research skills should be needed, and students should be able to employ these skills without additional practice.
This is a dangerous assumption.
First, a confession: I LOVE teaching methods. People don’t believe me when I say that, but it’s true. There is something so freeing about teaching a skill, rather than content, and ties into my longstanding interest of using the non-political to teach the political. Want to teach students about how to evaluate evidence and look for contradictions? Great: use Zendo. Need a way of showing students the difference between a random source and The Literature? There is a scene in episode 1:3 of Sherlock where Watson notes some interesting details about a piece of evidence, and Sherlock’s reply notes how Watson ‘missed almost everything of importance”. It is therefore somewhat easy for me to suggest that methods training be incorporated more widely into our content courses.
In isolation, a single methods course or sequence is simply not enough to really build the research skills of our students. First, one class must by nature be a hodge-podge that includes at a minimum research design, ethics, stats, qualitative approaches, philosophy of social science, writing, professionalization, and basic research skills. Choices must be made about where to go in-depth and where to make cuts, and it is unlikely that students will be truly proficient in all these areas at the end of a course. While other methods and skills courses may be available, this is not necessarily a panacea: if required, they pose opportunity costs at the expense of content courses; if not required, students are unlikely to take them. Regardless, there are logistical issues for smaller departments in terms of finding faculty to offer these classes.
Second, skill retention is an issue. We all have seen the student who recognizes a concept or name, but cannot define or identify it despite having previously done well on a test on the subject. Methods in particular depends on reinforcement and active use of the concepts, and three or four activities and assignments on the same area is not necessarily sufficient for students to have truly internalized the skill. As with a foreign language, constant exposure and use is needed for the skill to really develop.
Third, restricting methods to a single class makes students question the very value of taking the class. They tend to come in suspicious or scared of methods as it is, and if they only encounter the ‘math stuff’ in the one class, they may not fully appreciate the role of methods in political science and thus end with a distorted view of the discipline.
My particular concern is less on stats and more on the basic research skills that we expect all of students to have. And I worry that it is in this area that we are most in danger of failing our students when we fall prey to the methods silo effect.
Yesterday I attended a session on research skills at my university’s Global Citizenship Project Summer Collaboratory, a venue for discussing the implications of our new general education program. The facilitator’s passed out a handout on information literacy from the Association of College and Research Libraries that said the following (paraphrased):
A competent researcher is able to:
- Determine that additional information is needed (ie, their own opinion is insufficient)
- Access the necessary information (find sources effectively)
- Evaluation information and sources critically
- Reflect on the information and potentially reevaluate existing opinions
- Use the information effectively in making an argument
- Cite appropriately to avoid plagiarism
The facilitators then asked us to consider at what stage in the process do students tend to get into trouble. Most people cited the second and third areas as the most troublesome spots. My answer: ever single stage.
Sobering, but true. I’ve seen every step go wrong. I get papers that are full of first-person opinion with not a single external source, as it never occurs to the student despite explicit instructions that outside opinions are necessary to make their point. Others have “sources,” but rely on Wikipedia and the first five results from google. Few papers are brave enough to include potentially contradictory evidence; more common is what I call the 2 AM Problem, where students discover a piece of evidence that contradicts their thesis several hours before the paper is due, and rather than struggling with it or revising their work, they pretend it never existed. Then we have the papers that are strings of endless quotes with no original argument, or where the evidence and the thesis don’t quite match. And then we finally have the papers that commit none of these sins, but instead lack footnotes, citations, or have a bibliography of simple URLs. I suspect we all have our horror stories.
And yet how many of us go forward with assigning research papers, trusting that students either have the above skills already or will somehow manifest them before turning in the work for our course, only to be disappointed in the final product? I have certainly been guilty of this. At best I require a one-page topic description and annotated bibliography and offer to read rough drafts; many instructors go further and require rough drafts and peer-editing. Such measures are crucial, but they are often designed as assessments rather than continued skill-training exercises, or are more aimed at defeating the average student’s poor time management skills than reinforcing skills. Noble goals, but still, not enough.
Just as we require students to learn the content in our classes, if we wish to assign research papers, then we must be willing to teach students how to do them, step by step, and give them a chance to practice the skills before they are assessed on them. We must escape from a methods silo mentality: A single methods class (or even a sequence) will not do; these skills must be practiced and reinforced throughout the major.
I have a few exercises already on these skills and I will post a few of them in the coming weeks. But I want to invite comments pointing to ideas and assignments that readers have found useful in helping students not only break the research process down into manageable chunks, but to actually practice the different stages before being graded on a final effort.
Quick comment before I write up a post on a similar topic, specific to your sentence “Such measures are crucial, but they are often designed as assessments rather than continued skill-training exercises . . .” I have used mandatory rough drafts, scaffolding of writing assignments, etc. in the past, and the problem I always run into is that if it’s not formally and independently assessed as part of the final grade, the vast majority of students refuse to put effort into it.
Great question. Absolutely, student motivation is definitely a problem. There are a couple of things we can do. First, we can make the assignment an in-class exercise, “flipping the classroom” so that we coach them through the exercise while we have them on hand. This is appropriate for some exercises but not others, of course, but is a way to get them to work without grades as a motivator (or using it as part of their participation grade). For example, have them write (solo, in pairs, whatever) a 20 word summary of an article or debate and put them on the board–this gives them practice in writing thesis statements, concise phrasing, and the public nature motivates them to take it seriously.
Another way is simply to incorporate methods topics in discussions of readings (though getting them to read is an issue in and of itself), asking students about the nature of the source, who the audience is, whether it would be appropriate to cite in a paper and why/why not, the hypothesis and variables, etc. Asking the questions reminds them of these concepts, reinforces their importance, and gets them thinking methodologically. A third way is to offer the mother of all incentives, extra credit, which is particularly useful in getting them to take games and simulations seriously in class, but could be adapted to motivate these sorts of exercises. (A variation on this is to actually build your course around the idea that EVERYTHING is extra credit, according to a video game model,, but that’s another post)
Finally, we certainly CAN offer conventional grades for the assignments. This is fodder for another post, but the problem i see is a design issue: we need to be clear in our minds about the purpose of the assignment and whether it is primarily formative or summative (Nina Kollars and I have a forthcoming article in JPSE about this). A formative assignment can still be graded, but it should probably have lower stakes than a summative or evaluative assignment. Alternatively, it can be high stakes, but students would be allowed to revise their work or do another version of the exercise until they ‘master’ the material. I think that scaffolding processes are wonderful, and that they definitely serve a formative purpose, but that in the best cases the design includes an awareness of that formative purpose, and that the assignment is not simply a waystation along the way to the final paper where components are completed in week 6 rather than 16.’
I use Chippendale, Jim Davis’s great cross-tabs generator, to incorporate research questions in many classes, starting with the basic intro and US government classes. Chip is so easy to use that there are virtually no learning barriers for the program itself. That means you can spend most of your time working with trying to get the students to actually learn how to pose and answer simple research questions using easily understood (at first) techniques. You can see how this works and find an on-line version at http://www.zetadata.com/. The program is no longer supported actively, but most of what you need is right there. You can also look at SSDAN for more datasets, albeit somewhat short on political science topics.