Redesigning methods teaching: parallel workshops for interdisciplinary learning

This guest post comes from Dr Viviane Gravey and Dr Heather Johnson, both of Queen’s University Belfast

Research methods are crucial, particularly in Graduate learning, but methods modules are often the most unpopular with students and staff alike.

This makes methods modules prime candidates for either offloading onto temporary staff, or confining to designated ‘methods heavy’ positions for often isolated staff. This shunting of methods teaching onto precarious staff communicates unspoken but negative messages to students about the importance of this training, while consistently lower-than-average student evaluations (regardless of actual teaching excellence) negatively impact the profiles of vulnerable colleagues. 

At a time where we see silly op-eds calling for a Deliveroo approach to higher education (students deciding what they want to learn at MA level, and taught by temporary providers hired ‘on demand’), methods module would be first on the chopping block. Yet these unloved offerings provide, or at least should provide, the building blocks for that much-loved rite of passage: independent research and the MA dissertation. Beyond the dissertation, a deep engagement with methods is needed to better understand where we position ourselves in our respective fields, and so provide critical insights into both the mainstream and its critics.

Redesigning how we teach methods is far from a new topic on ALPS, with examples from using games to make students’ introduction to methods less frightening, to a series of posts on flipping the methods classroom.

This post draws on our own experiences, alongside reflections from EUROTLC discussions on curriculum design. Usual caveats apply: this is not a silver bullet. It depends on our local conditions and is still very much a work in progress. But at a time where the pandemic is forcing a rethink in how, what, and even where we teach, our stranded, workshop-based module can offer a useful starting point.

Context and problem

Following an administrative merger in 2016 we are a bigger school, with a growing number of MA students across 11 programs in Politics/International Relations, Anthropology, and History. Many students have backgrounds in other disciplines, and a growing proportion come from overseas. Some programmes are interdisciplinary, some more discipline-specific, with significant variation in student numbers from 6 up to 80+.    

Teaching different methods modules for each pathway is impractical, and while the merger offers opportunities for interdisciplinarity, combining methods teaching raises three dilemmas. First, should we aim for depth and specialization, or breadth and variety? Second, could we agree core teaching across the disciplinary boundaries?  Finally, how might we achieve student-led learning that encourages exploration and recognizes diverse backgrounds?

An innovative stranded, workshop-based module

Core or optional, breadth or depth? Instead of choosing we opted for both, via two simple design choices (a) ditching the one week/one topic model in favour of parallel workshops and (b) designing ‘strands’ to organise these workshops. Instead of covering 10 to 12 topics in as many weeks, we offer a wide range of parallel workshops, limited only by staff and room availability (and our collective imagination). Last year we offered 40 workshops, delivered in 8 weeks, taught by a team of over 25 colleagues according to their expertise, for close to 200 students. This also served to engage staff at all levels and in all areas of the School, centralizing rather than isolating methods teaching in the curriculum.

Workshops are organized across 6 strands (see examples in Figure 1) – from epistemology to case studies, whereby colleagues walk students through their own research design in a recent project.  These strands are populated according to the demands of our different MA programs, and also reflect the best practices of RCUK graduate training by exposing students to philosophy of science, and to both quantitative and qualitative methods.  They seek to enable flexibility for students according to their prior experience, with workshops that build upon one another in complexity and with different entry points.  A good example is the quantitative methods strand, which offers both basic training for primarily qualitative-focused researchers, alongside both beginner and advanced workshops for students who wish to specialize.

Figure 1

Students can, in effect, design their own path through the module: guided by their own interests and goals, they must take at least 9 workshops, including at least one from each strand.  Each individual program has designated compulsory workshops that students must include in their schedule in order to meet any specialization requirements. Thus, students have the opportunity to specialize, for example, by comparing different approaches to research interviews (5 workshops), or to explore new methods or move beyond their disciplinary boundaries.

Students are assessed on an applied methods portfolio of two items such as a short essay on epistemology, a data analysis exercise, or a practice interview or observation – and a research design proposal, bringing together content from the entire module (literature review, research questions, methods choices, ethical considerations). This proposal can be linked to the MA dissertation, and students are encouraged to treat it as preparation for their own independent research, working with their dissertation supervisors where possible.

Where next?

Reflecting on the first two years of this module, the welcome increase in student choice came at three costs – which we working to offset.

First, we need to ensure we do not ask students to run before they can walk: some students have no background in either methods or epistemological debates, and the kind of writing required in research design is often different than in a traditional essay.  As general training in writing skills is offered elsewhere in the university, this is difficult to address.  Nevertheless, we can both develop more ‘nuts and bolts’ workshops, and also sign-post students early on to outside support.

Second, the workshop model plays havoc with student timetables and our room-booking. Students can have different teaching loads week on week, and our commitment to (relatively) small class sizes means that we often need to add duplicate sessions to accommodate workshop popularity. This lack of certainty does not impact our student population equally – students working alongside their studies, those with caring responsibilities, or those living far from campus, will see their choices limited in practice. Providing more sessions online via asynchronous means will solve some, although not all, of these difficulties. We can also commit to publishing the timetable of workshops before term begins to facilitate student planning. 

Third, while the teaching load is shared, such a large and complex module comes with a commensurate administrative load for the course convenor. While some of that burden can be front-loaded in preparing the online learning environment (e.g. online workshop registration), the administrative load will remain large and often invisible.

Methods in a time of coronavirus

How teaching will happen in September remains uncertain. Nevertheless, we can focus on a number of ‘no regrets’ options.

First, we can ‘flip’ lectures, with pre-recorded, asynchronous introductions to different methods, and focusing any in-person class time on application. This would also allow students to discover a wider range of methods, and provide long term resources for their dissertation.

Second, it will be important to provide some dedicated training towards online research methods and ways to adapt traditional methods to social distancing. 

Finally, we can draw on external sources to broaden workshop options and resources. There is a wealth of methods teaching resources online – for example podcast series such as the UK National Centre for Research Methods podcast, or the Give Methods a Chance series.

In these trying times, it is time for universities to collaborate – where better than on methods teaching?

Looking Backward and Forward

Expanding on my last post on failures from this semester:

From where I stand, information literacy skills are important, because they help one identify and demolish specious claims made by authority figures. An assignment that, for example, forces students to locate three peer-reviewed journal articles is practice in finding credible information. It also allows students to determine whether a topic is suitable for a semester-long research project.

To me, these outcomes are both beneficial and rather obvious. But from the students’ perspective, the assignment could simply be yet another meaningless hoop to jump through on the way to getting another A+ on a transcript. Given the sources many students cited in the different stages of their storymap projects, it looks like too many of them customarily take the latter approach to research.

Therefore, in future courses that involve research projects, I should create assignments that are limited to the task of locating scholarly sources and place those assignments at the beginning of the semester. I should demonstrate why this skill is useful outside of the classroom.

I’ve noticed a similar problem with student writing — really basic errors that indicate a lack of proofreading. I don’t expend more effort evaluating a student’s work than the student did creating it. But I do know that sloppy writing indicates sloppy thinking and that the former advertises one’s propensity for the latter to the rest of the world. Again, I should demonstrate early in the semester why it’s important to proofread one’s work before it reaches an audience. My favorite example? The missing Oxford comma that cost a dairy company US$5 million.

I’m also seeing, from the last few journal article worksheets students are submitting, that many still do not have a clear understanding of how evidence-based arguments are constructed in academic literature. An author typically poses a research hypothesis or question at the beginning of a journal article and concludes with the same hypothesis or question reworded as declarative statement. I.e., “Why is the sky blue?” in the introduction with “The sky is blue because . . . ” as the conclusion. Yet on worksheets some students are writing that the hypothesis is about one thing while the conclusion is about some other thing. So again, students need practice in understanding the components of a written argument in scholarly literature, and that practice needs to happen early in the semester.

In principle I’m talking about scaffolding. But many of my assignments are attempts at getting students to builds several different skills simultaneously. I think I need to disentangle my goals for these assignments so that they target only one skill at a time.

The Article Summary

(Photo credit: Joanne H. Lee, Santa Clara University)

Today we have a guest post about teaching the research process by Anne Baker, assistant professor of political science at Santa Clara University. She can be reached at aebaker [at] scu [dot] edu.

Getting students to use academic articles for research papers can be a challenge. In my experience, many students, even those in upper-level courses, are not familiar with search engines such as JSTOR, Lexus Nexus, or Political Science Complete. And if students do happen to use Google Scholar, they frequently rely on excerpts from sources instead of entire articles that they might not have access to. So, what can be done to replace these habits with better practices?

In my advanced writing course on the presidency, I have developed a class activity which provides students with skills they will need if they are going to successfully locate and utilize academic references for their research papers. First, I want them to be able to use the library’s website to access search engines. Second, I want them to understand that research is an iterative process. Sometimes you don’t find what you need for a variety of reasons and you should be able to determine what those reasons are—whether its human error, the need for a wider search net, or that no one has written on the topic (this last possibility always surprises the Google generation). Third, students need to become acquainted with the literature on the presidency, including the subfield’s primary journal, by discovering how research practices in political science have changed overtime, even in a subfield which remains largely qualitative.

I have students work in pairs and I provide them with two search terms related to the institution of the presidency (e.g. signing statements, executive orders, oath of office). I pick the search terms carefully knowing that some topics have no scholarship and represent dead ends and others have later but not earlier scholarship or vice versa. The first step of the activity provides instructions about how to first locate JSTOR on the library’s website and then how to access Presidential Studies Quarterly using JSTOR’s advanced search options. Helpfully, for the purposes of this activity, JSTOR only has copies of the journal until 2000. To access later copies, students have to use the Wiley database, which students have to figure out how to find.

For each search term, I have students locate one article published in the last few years and then another for 1995-2000—a total of four articles. Next, students identify the research question and method the authors used, noting whether it is qualitative or quantitative, the sources of data regardless of method, the type of analysis (e.g. text, interviews, statistical), and the date of publication. After they have their four articles and perform this analysis, I ask them to compare the results of both searches. Finally, we have a class discussion in which we explore road blocks and challenges encountered and review how the field has changed over time.

I have found that this activity makes students more likely to cite academic articles in their final research papers and use them more effectively to support their arguments. Students also exhibit a much better understanding of the subfield and are more likely to use the other search engines that they encountered while on the library’s website. And they learn that research takes time and requires shifting your strategies to find the information you need.

GIYF, apparently

Long story

This academic year I’ve encountered a problem that wasn’t previously one I’d seen.

On several occasions, students have been in touch to say they can’t access materials, or links in reading lists are broken.

That’s bad, on my part, but a quick type into Google pulls up the correct link and/or material.

Most of me assumes that search engines are a staple of modern life, so I struggle to understand why one wouldn’t just check on one if you couldn’t find what you wanted.

But a bit of me also worries that this is symptomatic of some kind of learned helplessness: by being in the habit of just being given stuff, one loses the ability to find stuff for yourself.

I know we go around the spoon-feeding debate in education pretty regularly, but this seems like an odd case, given that we could expect using search engines to be a much more pervasive thing.

I need to follow this up with students and I’ll come back with any feedback, but I’d also welcome your thoughts too, both on causes and solutions.

Call for Book Chapter Proposals: Teaching Political Methodology

Today we have a call for proposals from Jeffrey Bernstein at Eastern Michigan University.

I am working with Edward Elgar Publishing to produce an edited volume, tentatively entitled “Teaching Political Methodology,” that will focus on teaching this subject at the undergraduate level. Such a collection, I believe, will fill a hole in the literature.  Most of our departments offer such a class; however, it usually proves to be a hard course to teach. I’m excited about the possibility of a book that articulates rationales for what this course should look like, and for how it can be done well.

The publishers are looking for fairly thin (200-250 page) book, most likely with around twelve contributors.  The volume will likely consist of two parts.  Section One will focus more on the larger, theoretical questions involved in teaching research methods to political science undergraduates.  Why do we see this as an important topic for students to learn?  Do we want to approach the course as teaching mostly research design, statistical analysis, or programming and using Big Data? How much should we focus on qualitative versus quantitative tools?  While quantitative methods have traditionally dominated, scholars have noted the limitations and biases in both the questions asked and the tools used to answer these questions.  To what extent should our courses reflect this? 

Section Two will focus less on the theoretical and more on the applied.  Once we have determined the sort of methods course we want to teach, how do we do it effectively?  What are the best means to get across the central lessons from methods classes?  What does it look like when students achieve our learning goals?  Papers for this section should move beyond assertions of what we should be doing, or what we believe will work, and present evidence of student learning drawn from their work.  They should include things such as sample assignments to help other instructors build on successful approaches to the subject.

If you are interested in contributing to this collection, please email me as soon as possible at jbernstei@emich.edu with a summary of the idea you are proposing, as well as a CV.  The proposal deadline is May 1. Completed chapters will be due to me by May 31, 2020; this extended time frame will allow people to develop ideas for teaching these classes and test these approaches against data during the 2019-2020 academic year.

Information Literacy as Research Methods IV


R M Media Ltd, CC BY-SA license

As I mentioned in my first post in this series, my interdisciplinary methods course includes a research proposal assignment consisting of:

  • An introduction containing a research question, hypothesis, rationale, and context.
  • A one-paragraph abstract.
  • Two-page discussion of the design of the proposed research project, the types of data that will be collected, how the data will be analyzed, and how this process will test the hypothesis and provide an answer to the research question.
  • A bibliography of references.

The proposal is intended to prepare students for an actual research project that they will design, conduct, and report on before graduating. I’ve created three smaller practice assignments that scaffold different aspects of the final proposal. The first uses research on Bolivia; here are the instructions:

  1. Read the rubric.
  2. Read either a chapter from Jim Schultz and Melissa Crane Draper, eds, Dignity and Defiance: Stories from Bolivia’s Challenge to Globalization, UC Press, 2008, or Isabel M. Scarborough, “Two Generations of Bolivian Female Vendors,” Ethnology 49, 2 (Spring 2010): 87-104.
  3. Read:
    Writing a Good Research Question
    What Makes a Good Research Question
    Constructing Hypotheses in Quantitative Research
    Forming a Good Hypothesis for Scientific Research
    Annotated Bibliography Guidelines

Write and submit the following as a single document:

  • A research question about the local effect of globalization in Bolivia.
  • A hypothesis derived from your research question.
  • A one-paragraph annotated bibliography entry for the item you read about Bolivia. Include the source’s complete bibliographic information.
Continue reading “Information Literacy as Research Methods IV”

Information Literacy as Research Methods III


R M Media Ltd, CC BY-SA license

An important component of both statistical and information literacy is the ability to recognize the difference between correlation and causation. Teaching this skill is made even more difficult by cognitive biases that lead to errors in probabilistic thinking.* So I decided to hit my students over the head with Chapter 4 from Charles Wheelan’s Naked Statistics and, from Tyler Vigen’s Spurious Correlations website, an image of the 99.26% correlation between the divorce rate in Maine and margarine consumption.

The assignment asked students to submit a written response to this question:

Why are these two variables so highly correlated? Does divorce cause margarine consumption or does margarine consumption cause divorce? Why?

All the students who completed the assignment answered the question correctly: neither one causes the other. In class, students identified several possible intervening variables, including:

  • People eat margarine and margarine-laced products as an emotional comfort food when relationships end.
  • Divorce leads to a greater number of households, with each household purchasing its own tub of margarine.

Students’ ideas led in turn to a discussion of how to appropriately measure these variables and construct new hypotheses.

*An excellent overview of this topic is Jack A. Hope and Ivan W. Kelly, “Common Difficulties with Probabilistic Reasoning,” The Mathematics Teacher 76, 8 (November 1983): 565-570.

Links to all posts in this series about information literacy:

Information Literacy as Research Methods II


R M Media Ltd, CC BY-SA license

Another post about the methods course that I’m now teaching. Chapter 3 of Naked Statistics is about deceptive description. So here is the accompanying assignment . . .

Many high school seniors are interested in attending Southwest America State University for college. Before 2015, applicants to this university had to submit high school transcripts that include average GPA scores, SAT scores, and an essay. In 2015, the application process changed; applicants had to submit high school transcripts with average GPA scores and two essays, while submission of SAT scores became optional. In 2019, the university claimed that the academic quality of its students had increased since 2011 given this pattern in the average SAT score of each year’s incoming class:

  • 2011 – 990
  • 2012 – 1130
  • 2013 – 1090
  • 2014 – 1150
  • 2015 – 1160
  • 2016 – 1185
  • 2017 – 1170
  • 2018 – 1190

Is this claim deceptive? Why?

Continue reading “Information Literacy as Research Methods II”

Information Literacy as Research Methods I

Image by R M Media Ltd under a Creative Commons Attribution-ShareAlike license
R M Media Ltd, CC BY-SA license

One of the joys of being department chair is creating a curriculum map for information literacy learning outcomes — as part of a five-year program review for a department that is only two years old. Since I’m teaching research methods, a requirement for students in all three of the department’s interdisciplinary majors, I decided to make information literacy a focus of the course. I designed several brief assignments based on chapters in Charles Wheelan’s Naked Statistics that pertain to evaluating information sources for authority, reliability, and relevance. These tasks in turn complement, in my mind at least, two larger assignments: Amanda’s Best Breakfast in Town project and writing a research proposal.

I thought I’d post some of those assignments here on the blog along with an assessment of how well students did on them. First topic on the list is hypothesis construction:

Given the availability of mobile phone coverage in Ghana, Nigeria, and Tanzania, how can we infer which country is the most violent? Why? (Generate a hypothesis about a relationship between mobile phone coverage and violence.)

Students did a good job thinking of possible causal relationships between mobile phone use and violence. Class discussion included ways to operationalize the concepts of violence, wealth, and happiness, which we did with some quick internet research. Students did not find an association between homicide rate and the amounts of mobile phone coverage in Ghana, Nigeria, and Tanzania, which then led to the topic of sample size. The assignment seemed to work as I had intended.

Continue reading “Information Literacy as Research Methods I”

Badges and recognizing success

Badges are not exactly a new concept, but like many ideas that come from games, they are still not widely known in the world of higher education and learning.  As many of us know from playing sports or participating in Scouts-style programs as children, badges are physical marks of achievement given to those that show competency at a particular skill or for doing well in a competition. In video games like World of Warcraft, meeting a particular goal is called an ‘achievement’.  Each achievement you earn flashes across the public chat, allowing for recognition, and some of them come with special items or titles in the game.  Defeat the Lich King, and you not only get to bask in the glory of victory, but you earn the title ‘Kingslayer’. While typically meant as a form of visible recognition for an achievement, earning achievements or badges can motivate behavior and can be seen as credentials in their own right.

Motivation, recognition, and credentialing skills–sounds like what we want to do in the classroom, right?