Study: Students feel they learn more from lecture…but don’t.

Everyone should check out this important study by Deslauriers et al, published recently in the Proceedings of the National Academy of Sciences and currently open access. It outlines an experiment at Harvard that tested direct learning in an introductory physics class compared to indirect reports of learning. The takeaway is that students reported they learned more during the lecture—but performed better on quizzes taken following active learning sessions. This has tremendous implications for how we do active learning research–and shows the dangers of relying on student reports of how they learn.

In the experiment, students attended 11 weeks of the introductory course together, and then in the 12th week were randomly assigned to two groups–one with an instructor giving a compelling lecture, and the other with a instructor running a session using active learning techniques. The instructors were both well versed in active learning approaches and had experience in giving great lectures. Students took a survey afterward reporting on their learning along with a 12 question quiz on the material (created by a different instructor to prevent teaching to the test). In the following session the instructors changed their method, so each set of students experienced both a lecture session and an active learning session. The material in the lecture and active learning sessions was identical, as was the handout. In the lecture, the instructor worked through slides based on the handout and solved problems with students passively observing and filling in the answers, while in the active learning session students worked in small groups to solve the same set of problems with the instructor offering assistance as needed. As the authors say “students in both groups received the exact same information from the handouts and the instructor, and only active engagement with the material was toggled on and off” (2).

Students reported greater frustration with the more disjointed nature of the active learning exercise, and thought they learned better from the flow of the lectures, but the researchers found that students performed better on the quiz instrument on the material in their active learning sessions.

We’ve often noted when reviewing research on active learning techniques that indirect measures of learning–that is, student reports on their learning–are not ideal, but this study shows us one of the dangers of relying on such instruments. Less than stellar support by students can derail efforts to increase active learning in a particular institution. We need to be more cautious, then, in how we examine and evaluate evidence that supports–or opposes–the use of active learning in the classroom.

Audio Feedback and Transparency as Teaching Interventions

This is a review of “Enhancing formative assessment as the way of boosting students’ performance and achieving learning outcomes.” Chapter 8 of Early Career Academics’ Reflections on Learning to Teach in Central Europe, by Nikita Minin, Masaryk University.

Nikita Minin of Masaryk University is motivated by a goal we can all appreciate: ensuring that his students achieve the learning outcomes of his course.  In his case, the course is a graduate seminar on theories of IR and energy security and the learning outcomes include improving student skills in critical thinking and writing.  He noticed that students in his class did not seem to really improve on these skills during the class, and introduced three teaching interventions in an attempt to fix this. 

First, Minin provided more intense instruction on the writing assignments at the start of the course, providing a grading rubric and examples of successful student work. Second, he gave students audio rather than written feedback on their papers.  Finally, using a sequential assessment system, the instructor gave formative feedback first and grades much later in the course. Minin assessed the impact of these three interventions, comparing course sections with and without them, and concluded that the first two interventions achieved the objective of improving student achievement of the learning outcomes.

The interventions described in the chapter are in line with current thinking regarding in-course assessment. While Minin does not use the language of transparent teaching, his first intervention falls exactly in line with the Transparency in Teaching and Learning Project’s (TILT)approach. Transparency calls on instructors to openly communicate about the purpose of an assignment, the tasks they are to complete, and the criteria for success, and Minin does exactly that in this first intervention.  Given the data so far on the TILT project, it is not surprising that Minin saw some success by taking this approach. Likewise, now-ubiquitous learning management systems allow for giving feedback in multiple platforms, including audio and video. For years now, advocates for audio-based feedback claim that this can be a more effective tool than written feedback. Minin’s observations therefore, also fit nicely in line with existing work.

Where the chapter falls short, then, is not in the design of its interventions, but in the claims made based on the available data. The sample sizes are tiny, with just five students receiving the interventions. With final grades used as the primary dependent variable, it is difficult to tease out the independent impact of each of the three changes. Using final grades is also an issue when the experimenter is also the person who assigns grades, as it is more difficult to avoid bias than when more objective or blind items are used. Lang’s (2016) bookSmall Teaching: Everyday Lessons from the Science of Learningtells us that engaging in self-reflection is itself an intervention, and Minin’s use of minute-paper style self-reflections to assess the impact of feedback, while itself an interesting and potentially useful idea, mean that a fourth intervention was used in the course.  While I do not doubt Minin’s observations that his interventions had a positive impact, as they are backed by existing research, the evidence in the chapter does not strongly advance our confidence in those findings.

However, I have never been one to dismiss good teaching ideas simply because of a lack of strong evidence from a particular instructor.  Minin highlights a crucial concern—that we should never assume that our courses are teaching what we intend them to teach, and that ‘time and effort’ do not necessarily achieve the desired results, even for graduate students. Reflecting on this, seeking out innovative solutions, and then assessing the impact is a process we should all be following, and Minin sets a great example.

(Trans)formative Assessment in Teaching

Today I’m attending ISA’s inaugural Innovative Pedagogy Conference in St. Louis.  Victor and I are doing a workshop on using games and simulations to teach political violence, showcasing activities like Survive or Die!, Prisoner’s DilemmaModel Diplomacy, identity exercise, and others.  But I’m most interested in reflecting on the session offered by Matthew Krain and Kent Kille of the College of Wooster on Assessment and Effectiveness in Active Teaching in International Studies. Their framework for formative assessment (that can, in fact, be transformative) is very helpful as an overall approach to teaching.

Continue reading

Two Online Games From The New York Times

Last month The New York Times published an updated version of its confirmation bias game that might be useful for teaching research methods or political psychology. The newer version includes an explanation of how confirmation bias affects government policy.
Also of note is another game on President Trump’s plan for changing U.S. immigration criteria. I failed to qualify under these new proposed rules. Probably all of my students will fail also.

Ethnography of Consumption – A Twist

Buying is believing

The spring semester is ending here, and as usual everyone is dragging themselves across the finish line. There is not much left for students to do in my globalization course, but — given the structure of the U.S. higher education system — we still have to meet in the classroom. So, in search of something to fill time, I created an activity based on the last paragraph in my response to Leanne’s scavenger hunt exercise.

I collected some random products from home, all branded: running shoes (Saucony), cell phone charger cord (Samsung), bag of lentils (Jack Rabbit), hand towel (Cannon), and a tote bag to transport everything (London Review of Books). In class I laid all these items on a table and lied about getting them from a friend. I then told students to examine everything and write an answer to “What story is the owner of these things trying to tell others about the kind of person he or she is?” Students had already completed ethnographies of consumption, so they had some understanding of the connection between identity construction and consumption choices.

After five minutes, I grouped students into teams to create presentations, which they delivered during the second half of class. I got to listen to them unknowingly analyze my self-image as practical, value-oriented, and health conscious. (I would have been just as happy with other adjectives.) Once the last team had presented, I revealed that the items were my own by plugging my cell phone into the wall socket with the charger cord.

As a last minute invention, this activity worked fairly well. Students got to practice collaborating, communicating to an audience, and applying concepts. And all I had to do was bring a tote bag to class.

Teaching Trump #2

A follow-up to Amanda’s post about teaching in the time of Trump. Trump’s campaign and administration represent an opportunity to teach about racism in the construction of national identity. I stumbled into this subject accidentally in my globalization course last week, with an assignment that asked students to write a response to “Is France or the USA a more global society?” Students were supposed to reference the following items:

Yesterday I heard this radio report about residents of Newton, Massachusetts, one of the wealthiest and therefore whitest communities in the USA: Continue reading

The Change All Around, Part 1

The spring semester is about to begin, and as usual I have changed what I will be doing and how I will be doing it. I thought it might be interesting for the dear readers of this blog to learn about some of the factors that will affect my teaching over the next few months. But just to be contrarian, I’ll start in reverse chronological order:

I’ve decided to use Michelle’s technique of placing simulations at the end of the semester when both students and I will be approaching burnout mode. I have done a complete redesign of my comparative politics course, dropping the themes in favor of more traditionally-presented content like the diverse institutional arrangements of democracies. In an attempt to get at this topic, I have resurrected my Gerkhania simulation, but have tweaked it to give it more structure. First, I will assign students roles that have different ethnically-oriented, two-sentence backstories. The class will then convene as the Gerkhanian Commission for National Reconciliation, which has the following tasks: Continue reading

Farewell to Surrey

A few thoughts inspired by the recently-concluded Political Studies Association-sponsored Workshop on New Pedagogies at the University of Surrey in Guildford, as I sit in Terminal 3 at Heathrow.

Not a hobbit
Not a hobbit

Active learning strategies present several advantages and disadvantages for teachers. First, they often recognize that people don’t necessarily all learn the same things at the same speed. Any classroom in which these techniques are employed can be thought of as an effort in differentiated instruction — which can be beneficial when students possess varying amounts of prior knowledge.

Second, these methods create spaces where students can and often must behave in different ways. Not only can this force students to figure out how to ask the important questions in the right ways, it can also increase their motivation, an important intermediary variable when it comes to learning.

Third, many active learning exercises include a meta-cognitive stage in which a student’s articulation of his or her understanding is what produces understanding.

Also not a hobbit.
Also not a hobbit.

Fourth, active learning can, if implemented properly, offer opportunities for students’ conceptual, skill, and emotional development. Integrating all of these dimensions into the classroom requires careful consideration on the part of the instructor, but the payoffs can be quite high.

There are no bad ideas

Activating prior knowledge is a critical step in engaging students and facilitating learning.  There’s a whole bunch of literature in cognitive science that tells us that learning happens when students connect new knowledge to existing knowledge.  Brainstorming is one simple technique that can activate prior knowledge and encourage students to make these connections. Continue reading

What We Know About Learning

Eddie WatsonAnother follow-up to the 2016 TLC — this year’s keynote address was delivered by Eddie Watson, director of the Center for Teaching and Learning at the University of Georgia. He devoted part of his address to what the cognitive scientists and psychologists know about learning, and how we can put that knowledge to use in the classroom:

  • Identify what your students already know and teach accordingly. Use knowledge probes, pre-semester surveys, and pre-test/post-test instruments.
  • Performance followed by immediate feedback increases learning. The feedback doesn’t have to come from you — it can be just as effective if it comes from peers. [Perhaps this points to a reason why some simulations and games are pedagogically valuable.]
  • Learning is often facilitated by social interaction. [See above.]
  • Frequent testing promotes learning; it is more powerful than passive “studying.”
  • Being able to actively process information — for example, by applying it in a particular context — tends to result in greater learning.

For additional details, read Daniel Willingham’s work, which is linked to this post and to the blogroll on the ALPS homepage.