Audio Feedback and Transparency as Teaching Interventions

This is a review of “Enhancing formative assessment as the way of boosting students’ performance and achieving learning outcomes.” Chapter 8 of Early Career Academics’ Reflections on Learning to Teach in Central Europe, by Nikita Minin, Masaryk University.

Nikita Minin of Masaryk University is motivated by a goal we can all appreciate: ensuring that his students achieve the learning outcomes of his course.  In his case, the course is a graduate seminar on theories of IR and energy security and the learning outcomes include improving student skills in critical thinking and writing.  He noticed that students in his class did not seem to really improve on these skills during the class, and introduced three teaching interventions in an attempt to fix this. 

First, Minin provided more intense instruction on the writing assignments at the start of the course, providing a grading rubric and examples of successful student work. Second, he gave students audio rather than written feedback on their papers.  Finally, using a sequential assessment system, the instructor gave formative feedback first and grades much later in the course. Minin assessed the impact of these three interventions, comparing course sections with and without them, and concluded that the first two interventions achieved the objective of improving student achievement of the learning outcomes.

The interventions described in the chapter are in line with current thinking regarding in-course assessment. While Minin does not use the language of transparent teaching, his first intervention falls exactly in line with the Transparency in Teaching and Learning Project’s (TILT)approach. Transparency calls on instructors to openly communicate about the purpose of an assignment, the tasks they are to complete, and the criteria for success, and Minin does exactly that in this first intervention.  Given the data so far on the TILT project, it is not surprising that Minin saw some success by taking this approach. Likewise, now-ubiquitous learning management systems allow for giving feedback in multiple platforms, including audio and video. For years now, advocates for audio-based feedback claim that this can be a more effective tool than written feedback. Minin’s observations therefore, also fit nicely in line with existing work.

Where the chapter falls short, then, is not in the design of its interventions, but in the claims made based on the available data. The sample sizes are tiny, with just five students receiving the interventions. With final grades used as the primary dependent variable, it is difficult to tease out the independent impact of each of the three changes. Using final grades is also an issue when the experimenter is also the person who assigns grades, as it is more difficult to avoid bias than when more objective or blind items are used. Lang’s (2016) bookSmall Teaching: Everyday Lessons from the Science of Learningtells us that engaging in self-reflection is itself an intervention, and Minin’s use of minute-paper style self-reflections to assess the impact of feedback, while itself an interesting and potentially useful idea, mean that a fourth intervention was used in the course.  While I do not doubt Minin’s observations that his interventions had a positive impact, as they are backed by existing research, the evidence in the chapter does not strongly advance our confidence in those findings.

However, I have never been one to dismiss good teaching ideas simply because of a lack of strong evidence from a particular instructor.  Minin highlights a crucial concern—that we should never assume that our courses are teaching what we intend them to teach, and that ‘time and effort’ do not necessarily achieve the desired results, even for graduate students. Reflecting on this, seeking out innovative solutions, and then assessing the impact is a process we should all be following, and Minin sets a great example.

Reminder: International Teaching and Learning Conference proposals due Monday

Just a reminder that the joint International Teaching and Learning Conference proposal deadline is this Monday, 19 November. The conference  will be held in Brighton, United Kingdom, 17-19 June and will focus on teaching politics in an era of populism. It is jointly sponsored by APSA, PSA, BISA, and ECPR. All kinds of proposals are welcome, including panels, papers, individuals for roundtables, lightning talks, workshops, and any other innovative pedagogical approach you want to propose.  For more information, head to the PSA conference website. 

(Trans)formative Assessment in Teaching

Today I’m attending ISA’s inaugural Innovative Pedagogy Conference in St. Louis.  Victor and I are doing a workshop on using games and simulations to teach political violence, showcasing activities like Survive or Die!, Prisoner’s DilemmaModel Diplomacy, identity exercise, and others.  But I’m most interested in reflecting on the session offered by Matthew Krain and Kent Kille of the College of Wooster on Assessment and Effectiveness in Active Teaching in International Studies. Their framework for formative assessment (that can, in fact, be transformative) is very helpful as an overall approach to teaching.

Continue reading

Teaching Election Results

The 2018 midterm elections are over in the US, and it was a night of mixed results.  The Democratic Party took control of the House of Representatives, winning at least 27 seats previously held by the Republican Party, while the GOP increased their majority in the Senate, toppling North Dakota’s Heidi Heitkamp and my own state of Missouri’s Claire McCaskill.  

While clearly not as momentous an election as 2016, or the Brexit referendum, or many other elections, the midterms were still an important point to take stock of the impact of Trumpism on American politics, and whether Democrats who were somewhat over-confident in the fall of 2016 could manage to overcome pro-Trump sentiment, a strong economy, congressional district gerrymandering, voter suppression efforts, and the other structural reasons why succeeding at the polls can be difficult. 

Teaching the results of American elections is a frustrating enterprise.

Continue reading

Unexpected Teachable Moments

I had two unexpected moments in the last week completely outside of the classroom that led to (or will, in the future lead to) great teaching moments.

Last week I was recording a script I’d written in a studio, for a project I’ve been working on for two years.  In the lecture I was reviewing cross tabulation tables, and the example I had looked at 2016 presidential vote choice and gender.  Unbeknownst to me, someone in the control room changed the text on the teleprompter so that every time I used the words ‘man’ or ‘women’ they used ‘male’ and ‘female’.  Rather than stop the recording, I just started changing the words back as I spoke, which led to some awkwardness (man-Clinton voter does not roll off the tongue the same as ‘male Clinton voter’).  Finally the folks in the control group stopped me, and I explained that the original text used man and woman deliberately, and that changing it without speaking to me was an issue. I was talking about gender, not sex, and I wanted it to stay consistent.  This paused the recording for a few minutes, and during the lull I explained to the camera operators why this was important enough to stop for. We had a really nice conversation on the difference between sex and gender, and this turned into what we sometimes call ‘teachable moments’.  Script corrected, I continued recording.

The second moment was completely different.  Today I had jury duty. For those of you outside the US, this is when we get called to the courthouse to sit and wait until you are called into jury selection for a trial.  Sometimes you just sit in the waiting area for a day or two and are dismissed; other times you get put on a case right away, and spend the next day or two answering questions from the attorneys while they pick and choose who to put on the jury itself.  Sometimes they case will settle or be dismissed while jury selection is going on

I don’t mind jury duty.  Its one of the only things asked of me as a citizen, and I do see it as a civic duty. Since I teach American politics, I’d like to have the experience of actually serving on a jury, but that has yet to happen, and many of my legal friends indicate it’s unlikely to ever happen. Since I already had guest lecturers lined up this week for my daytime class, I didn’t request a postponement, and headed downtown this morning to serve.

Walking into the building, right after I passed through security, a police officer noted that I was carrying a heavy bag, and I stopped and chatted with him for awhile.  It turned out this was the sheriff himself, and he used to be a public school teacher in his early career.  Soon we were exchanging business cards, and he agreed to come talk to my students during our lesson on the judiciary next semester.  Did I teach him anything in our short conversation this morning? No.  But I’ll be able to create a great teachable moment for my students in a few months, all because I stopped to chat with someone rather than doing what I normally do, which is plug in my headphones and hurry on my way.  I’m not going to suddenly change my ways or anything, but sometimes an unexpected but nice conversation can serve as a reminder of the good things in this world.

Badges and recognizing success

Badges are not exactly a new concept, but like many ideas that come from games, they are still not widely known in the world of higher education and learning.  As many of us know from playing sports or participating in Scouts-style programs as children, badges are physical marks of achievement given to those that show competency at a particular skill or for doing well in a competition. In video games like World of Warcraft, meeting a particular goal is called an ‘achievement’.  Each achievement you earn flashes across the public chat, allowing for recognition, and some of them come with special items or titles in the game.  Defeat the Lich King, and you not only get to bask in the glory of victory, but you earn the title ‘Kingslayer’. While typically meant as a form of visible recognition for an achievement, earning achievements or badges can motivate behavior and can be seen as credentials in their own right.

Motivation, recognition, and credentialing skills–sounds like what we want to do in the classroom, right?

Success in online teaching: working with your LMS

I’m teaching my online graduate research methods course this fall, and as it is a 9 week course it starts next week.  Since each new section of the course is cloned from the ‘master’ version of the course, every time I teach it, I have to go in and manually update the due dates for assignments.  Most of the syllabus simply says that things are due in Week 3 or Week 6, and the weekly assignments are listed on an ‘activities’ page for each week, but many of the assignments have due dates too, and those need to be changed. It’s tedious but doesn’t take too long.

I’ve noticed in the past that students sometimes miss assignments.  There are 3-4 each week, a mix of discussions, quizzes, and other assignments, plus scaffolded project components, and I will occasionally have students that miss an assignment or two.  I’ve been teaching this course for years, and rather ironically never noticed until today that there was something systematic about the assignments that students tend to miss.

I gave specific due dates to some, but not all, of my assignments.

Continue reading

Electoral College Exercise

While I realize many of our readers are not based in the US nor teaching American government, the Electoral College is such an interesting oddity in electoral decision making that its a subject that may come up in Comparative Politics courses as well.  Certainly when I teach US politics I use quite a few comparative examples, as one of my themes of the course is how government arises from a series of decisions made by individuals and groups, none of which are or were set in stone.  Showing alternative models is a very useful way of doing this.

So here is a data analysis exercise that I use to teach the American Electoral College. It can be done either as homework or as an-in class as an activity after a basic introduction to the Electoral College and how it works (the basic premise of state-by-state popular vote, proportional votes based on number of seats in Congress, winner take all systems, and if no one wins a majority, the decision is made by the House with state-by-state voting).  

This exercise can be easily reformed for a final exam. Simply change the data and situations.  In the version below I use 9 states in a fictional world; in the exam version, I use about 20 states in a different world.  I never use the entire US or actual vote totals–this is largely to keep the math simple enough that it is not a test of arithmetic but of analysis.  Feel free to change the names of candidates and states to suit your own interests. 


Continue reading

Joint PSA/APSA/ECPR/BISA Pedagogy Conference on Teaching Politics in an Era of Populism

While I am very much looking forward to the ISA Innovative Pedagogy Conference, I’m also excited to share the call for proposals for this new pedagogy conference on Teaching Politics in an Era of Populism, a joint effort by the Political Studies Association, American Political Science Association, European Consortium for Political Research, and British International Studies Association. I am on the planning committee and very excited about bringing together a wide cross-section of scholars to debate these issues.

The conference will be held in Brighton, UK on 17-19 June, 2019.  We are accepting a wide range of proposals, including: individual papers, panels, workshops, 10 minute pedagogical TED-style talks, roundtables (submit as an individual, not a group), and ‘open source’, which is an invitation to be as innovative as you like in what you propose.  Submissions are due November 5th.  You can find more information on the conference web site.

From the call:

“This conference aims to provide a forum in which political science educators from different countries and contexts can come together to explore these challenges and share their experiences and teaching practices. We welcome contributions which explore the challenges faced in national, international, or comparative contexts. We also welcome different approaches to understanding populism and the challenges that it may present to political science educators in different contexts.”

  • Can or should political science education be ‘politically neutral’? Should we nurture values of democracy, equality, and citizenship and, if so, how?
  • How can we support students in developing knowledge, understanding and skills relating to the complex nature of politics, society and government? What role might different approaches to teaching such as simulations, civic engagement and other pedagogies play?
  • What are the challenges of constructing a curriculum and developing learning resources in a period of rapid and sometime dramatic political change?
  • How can we collaborate across different national and educational contexts to support critical learning in political science and international relations? What best practices are there for collaboration in both pedagogical research and cross-cultural classroom experiences?
  • Are there practices or pedagogies from other disciplines that can be adopted or adapted to address these issues?

Free-riders or silent participants? Appreciating silence in active learning environments

This guest post comes from Patrick Bijsmans & Afke Groen of Maastricht University’s Department of Political Science

We have been following the ALPS-blog discussion on students’ participation between Amanda and Simon with great interest. The situations they discuss are very familiar.

In Maastricht, learning takes place according to the principles of problem-based learning (PBL); through active participation and discussions in tutorials.

In the programmes that we teach in, we can grade students’ participation with a +0.5 on top of the exam grade for exceptionally good participation or a -0.5 for insufficient participation – a system introduced following discussions about the problem of ‘free-riders’.

We too see students who remain silent. We train students, encourage participation and discuss group dynamics, but students may not feel comfortable or skilled to live up to our expectations – certainly not in their first weeks at university.

Indeed, in the discussion between Simon and Amanda, the “problem” seems to be students who do not talk. Teaching is about “getting students to talk” and about “[getting] them to a point where they do the readings and are willing and able to talk about them”.

But to what extent is not talking a problem? Why doesn’t a student talk? And if it’s a problem, who’s problem is it?

Continue reading