Participatory research, calls for participants, and ‘the usual suspects’

In my post on ALPS last week, I wrote about how participatory research can be a beneficial active learning experience for students. In a recent project that I ran at the University of York, student partners came away from the research process reporting that they had a much stronger knowledge of research methods and delivery than they had before. However, as I also noted in that post, this is, for both methodological and practical reasons, an exclusive form of learning.

Nicole Brown (an Associate Professor at UCL’s Institute of Education) has made the case that student partnership research is vital in developing and delivering innovations in teaching, which in turn can boost student engagement and satisfaction. Brown is not alone in this assertion – a quick Google Scholar search will return a wealth of literature that details the benefits of student partnership research. Importantly, this literature also shows that, largely, students simply enjoy getting to be involved in the research process.

From this, an important question emerges – which students get to be involved in participatory research? Answering this question is important both to the students that get to be involved and to the conclusions that will result from the research. 

Continue reading “Participatory research, calls for participants, and ‘the usual suspects’”

Participatory research as active learning

Over the last semester, I have been running a research project, funded by the University of York’s Learning and Teaching Fund, exploring assessment norms and innovations utilised on Politics and International Relations degrees.

The project has been my first foray into participatory research with students. Throughout the project, six students took on the role of ‘Student Partners’ to help design, deliver, and conclude the research activities – activities that included running focus groups with other students. It made sense from a methodological standpoint to have this level of student involvement – after all, I was hoping to gain a student-centred understanding of the challenges and opportunities with innovating assessment practices.

Last week, we had our final research activity – a half-day workshop with the Student Partners to discuss research findings and to conclude the project. We also had some time to discuss the experiences of the students in collaborating with staff and each other on the research project. That discussion was an eye-opening one and made me keenly aware of how beneficial this kind of hands-on research experience can be for students as a form of active learning.

Continue reading “Participatory research as active learning”

Dissertation Excellence Made Simple: What to say to your students as they get started

Neil Sutherland, having a chuckle

Today we have a guest post by Neil Sutherland, from UCL’s School of Management, in which he offers some excellent tips on shepherding students through the opening stages of a dissertation.

I have supervised Undergraduate and Postgraduate dissertation students for well over a decade, and have summarised my advice into a series of bitesize YouTube videos: The Dissertation Toolkit. These are mainly aimed at students, but in this post I thought it’d be useful to bring together some useful bits of guidance I have  given to students and offer it to you, as fellow supervisors.

The starting point here is that embarking on the dissertation journey is daunting, and it’s crucial to acknowledge the overwhelming feelings that students experience at various points along the way. As academics we’ve routinised the business of research so that these apprehensions dwindle (or perhaps we find a way to box them up and ignore them altogether!), but students are, in the communities of practice parlance, novices. 

To help them overcome this apprehension I try to help my students see this as not only a manageable and achievable process, but also one that is exciting and rewarding. By the end of this journey, I tell them, you will have expertise in a chosen field that very few others in the world will have! But that grand plan is only achieved through a series of more mundane moments, or, as our friend Vincent Van Gogh tells us: “Great things are done by a series of small things brought together”.

So what follows is a set of exchanges between an imaginary student and me…

“There’s too much to do, and the word-count is imposing”

Being overwhelmed is completely normal at the start (or even during) your dissertation. It is likely that you haven’t written to this kind of wordcount before, so are figuring out the rules of the game. The number one trick here is to (a) get a structure mapped out, and (b) tackle the research and writing ‘little and often’. 

“But I want to start writing right now!”
When we are faced with uncertainty, it can seem easiest to just start writing – to begin to eat into that word-count. But pause. Your best approach is to use 1-2 sides of A4 to simply map out what will ‘happen’ in your work. What does the marking criteria ask of you? What chapters must you include? What do you know about the topic already that might be important to include? What advice have you been given by your tutors in taught sessions and handbooks? Collate this all in one place, and use it to create a numbered guide – almost a table of contents – that can help you to stay on track. 

“I just can’t stay focussed on the topic. I’m spending hours in the library but don’t seem to get anything done”
When you have your drafted structure, you can start to set specific goals and deadlines, when you want to hit certain markers, and work out when you will dedicate your time to the dissertation each week. Don’t be overambitious, but do treat it in the same way that you do any of your other timetabled classes, and even just a few non-negotiable hours a week will keep you focussed (yes – even more so than hours-upon-hours in the library without a clear goal). Don’t say “I want to complete XX hours in a week”, say “I will complete XX hours on Tuesday from 2-5pm”. That way, you can adopt the mindset that on Tuesday between 2-5pm you will be in dissertation mode, and at other times, you can put it to the side. 

“I’ve blocked out time, but when I sit down I can’t get ‘into’ my work and am easily distracted”

Of course, it is inevitable that on some days, you will not be feeling motivated. Don’t chase ‘inspiration’ (spoiler alert: it rarely comes when you want it to) and know that uninterrupted work time will inevitably get you in the flow. With any task that I am completing, I know that I will feel uninterested, uninspired and overwhelmed in the first 20minutes while I am recalibrating to the task. I keep reminding myself of all the other jobs I have to do, and how this one could just happen tomorrow instead. Ignore that voice. After that, something magic happens, my brain clicks into gear and time ceases all meaning. If I get interrupted during that initial stage, I go back to square one. Minimise your distractions and the flow will come. 

And… on the days that you are really not in the mood, you can focus on the ‘invisible work’: finding new sources, returning to the criteria, re-watching key lectures, or organising data. This is your back up to ensure that you are always moving forward, even when the conceptual work feels challenging (and more often than not, that invisible work is precisely what you need to kick-start your flow state)

“So… now I have blocked out my time, I can start my first draft now, yes?”

Not yet. Do not worry about starting to write the dissertation straight away, because at the start of the process, you simply have not got the knowledge to do so, and that’s ok! In the initial stages, to compliment the plan you have already created, you can just focus on reading, understanding and finding out new information about your topic (if you need guidance on finding literature, I have tips here and here), and making good quality notes to come back to later. 

The time to begin your drafting is when you feel like every piece of literature you read is telling you something you sort-of already know, or, ‘theoretical saturation’. This means that you’ve got a good sense of the field, the various subject-positions of scholars, previous work completed, and potential gaps. At this stage you might want to revise your initial structure (or even research questions) to reflect what you’ve found, and then you can confidently begin drafting. 

“But will this make for a better dissertation?”

Whilst it feels like this will make the whole process more laborious, and will be dealing with your friends telling you how many words they have written: Do not be fooled. Too many times I have seen students arrive to a meeting with thousands of words ‘finished’, only to discover that they are on the wrong track. Little and often, and dedicating non-negotiable specific time to the task, is the way forward. Don’t worry, the dissertation will steadily grow, and this will be driven by your background knowledge of the topic, not just the words you write on the page.

“What next?”

For further information and advice, subscribe to ‘Neil’s Common Room’, and do get in touch! Videos in The Dissertation Toolkit series include: 

  1. How to start your dissertation the easy way (11mins)
  2. How to make the most of your dissertation supervisor meetings (9mins)
  3. Storytelling: The secret behind all great dissertations (8mins)
  4. Overcoming obstacles in writing your dissertation (18mins)

Discussion-Based Quizzes 2

First, I hope to see folks at some of the pedagogy-oriented sessions at this week’s ISA meeting in Montreal. Feel free to chat up me or the illustrious Dr. Simon Usherwood.

Second, a follow-up to my post last month about no-stakes quizzes and class discussion.

I gave students another quiz, on survivorship bias, a topic of the reading assigned earlier in the week. Here is the prompt for the questions (note that the scenario is a fictionalized version of a famous World War II example):

The United States has gone to war against Zambia. A high percentage of U.S. warplanes are being shot down by the Zambian military. You work for the U.S. Department of Defense as an operations research analyst. You have been given the task of recommending where additional armor plating should be installed on U.S. warplanes to better protect them from enemy fire. The image below shows a representative pattern of damage from Zambian anti-aircraft fire to U.S. warplanes that have made it back to home base. 

Responses to question 1:

Responses to question 2:

Despite the tiny sample, I think the quiz scores indicate the ease of memorizing a concept’s definition while being unable to meaningfully apply it. Students frequently equate memorization with knowing and hence learning, when mostly it’s not.

Discussion-Based Quizzes

Alternative title: Stuff Just Happens, a persistent student mental paradigm that I’ve written about before (here and here).

I’m teaching an undergraduate introduction to research methods course this semester, for which I created a series of practice quizzes that contribute nothing to the final course grade. I expected the quizzes, which range from one to three questions, to function as retrieval practice and launch pads for class discussion. This is the same idea behind polling students to check their comprehension while lecturing, but I used the Canvas LMS quiz tool instead of a polling platform because 1) I was too lazy to learn the latter, and 2) I wanted all course content housed in one place.

The activity is not working as well as I thought it would, for a few reasons. First, Canvas identifies the correct answer to each question when displaying quiz results, as shown below, which shuts down discussion about which option is correct and why. A pie chart that doesn’t label the correct answer would work better — i.e., Google Forms.

Second, this is a small class that meets at 8:00 a.m. The quantity and quality of discussion declines markedly when only a few students are absent, which is usually the case.

But most importantly, given their answers to the quiz questions, students are still having a lot of difficulty forming plausible and potentially valid causal inferences from the data that I present. I’ve given six practice quizzes so far, and on average students answer questions correctly only about 50% of the time. Here is a typical example from a prior quiz:

Based on the visual representation of data below, a logical hypothesis is:

Serendipity in Research Methods

Sometimes it is easier to demonstrate real-world relevance than others.

Last week students in my research methods course read Charles Wheelan, Naked Statistics, Ch. 12, and Ashley A. Smith, “Students Taking More Credit Courses and Introductory Math Faring Well,” Inside Higher Ed, 7 December 2018.

They then had to answer this question: What mistakes are Nevada officials making with data about community college students?

As written, the Inside Higher Ed story describes people who should know better falling victim to omitted variable bias and confusing correlation with causation. Although I might be making similar mistakes in evaluating in-class discussion about the assignment, I think that students found it more interesting than most because the assignment was about other students.

Soon afterward, two similar items came across my radar:

Students prefer mixing and matching online with on-campus courses.

Common premises about college students are wrong.

I shared these with my students, as additional examples of analyzing (or not) data about their peers.

Critical thinking and the Ukraine invasion

I’m not an IR person, and I know it.

Unfortunately, a lot of the people I follow on social media do think they are now specialists in warfare, diplomacy or the operations of civil nuclear facilities. These people were also once ‘experts’ in epidemiology, Brexit, macroeconomics, US presidential politics, populism, immigration and many other things besides.

I have my doubts.

This is probably also a problem you face as you try to make sense of the world around us: yes, you know some people who do actually really know stuff, but they get buried in a big pile of hot takes, motivated reasoning and even propaganda.

So what to do?

I’m guessing that Ukraine is an easier case for the readership of ALPS blog to handle, since it’s closer to many of our research interests: even if we don’t work on relevant topics ourselves, then we know the people who do and tap into their expertise.

Of course, as the whole Mearsheimer thing has shown in the past week, even very competent people come up with dubious positions, although you at least get lots of material for your next IR theory class.

(For my part, I’ve limited myself to working up the one element I do feel competent to speak on).

However, for your students this might still be at the edge of their knowledge, abilities and confidence, so how can we help them parse the situation?

For me, task number one has be a strong refresher on how to evaluate information (and it’ll be a refresher, because of course you teach this as a matter of course, right?).

That means making sure they understand the importance of verification, of triangulation, of expertise and of all the other things that we have probably internalised over the years. If we running a class that needed to engage with this I’d be asking students to locate good guides to how to do this, then pulling them together into a master document that they can all use for their subsequent research.

For as fluid as case as an active conflict, information is incomplete and often contradictory, so giving students the tools to determine what they know and what it means is essential. The growing OSINT community is a really good starting point for looking at the operational end of things, while the more strategic reasoning requires engagement with those working in a number of different domains, including Russian politics, military doctrine and sanctions.

As we’ve seen in recent years with whatever crisis you care to imagine, there is a huge potential to access properly informed and well-evidenced specialists on any given topic. But that means cutting through the guff and being able to contextualise what we read.

And that’s a great life-skill to be developing in our students, regardless.

What You Think Depends On Where You Stand

Our superb librarians survey students and faculty annually. Results from this year’s survey are in. Student responses to one of the questions:

Faculty responses:

Notice that the frequencies of responses from these two groups are essentially mirror images of each other. Students are extrinsically motivated by grades, so they think in instrumental terms: I need correctly formatted citations and the specified minimum number of sources. Otherwise my grade will be negatively affected. Knowing whether a source is reputable is far less important. Faculty think the reverse: the ability to locate scholarly source material and analyze information for bias matters most.

I have tried to solve this problem in the past, and could not find a satisfactory solution. Consequently, I have focused more on curating quality content for student to consume than on marking down students because of their reliance on websites that are top-listed in Google searches. In fact, it’s one of the reasons I decided to stop assigning traditional research papers.

Given the survey results though, the problem extends far beyond my small corner of the curriculum. I’m not going to solve it independently.

Readers might find these other posts on information literacy skills to be of interest:

The Methods Silo Effect and Fixing Poor Research Skills


Write Your Own Headlines Activity

Write Your Own Headlines Activity

This post comes from Chelsea Kaufman, assistant professor of political science at Wingate University. She can be contacted at c[dot]kaufman[at]wingate[dot]edu.

In teaching undergraduate research methods, I often find that the students are intimidated by the subject matter and don’t see its relevance to their lives. I have increasingly emphasized to students that it prepares them to be savvy consumers of political information wherever they might encounter it. This approach introduces an additional challenge, however: students often lack the information literacy skills to evaluate the sources that they access. If I want students to have the skills to evaluate the political information they encounter, I obviously need to teach them these skills. How exactly can this be accomplished? 

It is not enough to tell students which sources are acceptable, because people tend to trust information that aligns with their political predispositions. Simply lecturing to students about the dangers of misinformation can reinforce false beliefs and increase their distrust of reliable sources. 

To avoid this conundrum, I have students write their own headlines based on public opinion poll data. I first find a poll with results covered in several media outlets. I then send students a link to (or print out) the results of the poll, without providing them any context as to how it was covered in the media. After writing the headlines, students share them and compare theirs with those of their classmates and with published headlines about the data. Students learn to interpret data and evaluate whether it was given accurate coverage in the media. As the final part of the lesson, I then ask them to evaluate the polling methods used to obtain the data, by, for example, considering how a question’s wording might have impacted the responses. 

You can view detailed instructions for the activity on APSA Educate. You can also read more about this topic and find examples of additional activities in my article Civic Education in a Fake News Era: Lessons for the Methods Classroom or my chapter in The Palgrave Handbook of Political Research Pedagogy