More on Designing for Cognitive Load

A follow-up to my last post about cognitive load:

I remember a few conversations over the years — once during a job interview — on whether it’s better to give students a concept first and specific examples second, or to provide examples first and then the concept. Bokan and Goodboy (2020)* studied this question with an interesting experiment.

Focus on the triangles’ colors for now; I’ll explain how it relates to something called democratic peace theory later.

They randomly assigned 275 students to one of two conditions in which the order of information in a narrative instructional text moved either from (a) concrete examples to abstract definitions or from (b) abstract definitions to concrete examples. Students reported their perceived cognitive burden during the experiment. Bokan’s and Goodboy’s underlying hypothesis was that poorly designed instructional materials increase students’ extraneous cognitive burden, leading to working memory overload and decreased learning.

They found that placing concrete examples after abstract definitions in an assigned text resulted in higher scores on tests of information recall, retention, and application, even when controlling for students’ prior familiarity with the subject and grade point average. Students “scored almost a whole letter grade lower for every point they reported facing a higher working memory overload.” The authors concluded that the order in which information is presented matters for students reading instructional materials, perhaps because people have a “natural tendency to look for organizing principles before they move on to study more detailed information.” When specific examples are presented before the larger concepts to which they pertain, people are forced to keep detailed information in their minds while simultaneously attempting to categorize it.

*San Bolkan & Alan K. Goodboy (2020) Instruction, example order, and student
learning: reducing extraneous cognitive load by providing structure for elaborated examples,
Communication Education, 69:3, 300-316, DOI: 10.1080/03634523.2019.1701196.

Designing for Cognitive Load

Cognitive load theory is one perspective on learning that can be applied to teaching in this unusual time. The theory sees working memory — the part of the mind that temporarily stores and manipulates information — as a constraint on learning, because it can only manage a few pieces of information at once. Placing a load on working memory is like trying to push water through a pipe with a constant diameter; you can shove only so much water through the pipe at any given time at a given pressure. If the water’s pressure exceeds what the pipe is able to withstand, you get a flooded basement.

What is the square root of 1,137?

There are three types of cognitive load: intrinsic, germane, and extraneous. Intrinsic load represents the essential actions that occur when learning information that is specific to a topic or task. The load varies according to the information’s inherent level of difficulty for the learner, which makes it partially a function of prior learning: the more one has practiced using the information, the less the effort that needs to be expended the next time it’s encountered. Tying shoelaces requires our full concentration when we are in kindergarten; as adults we perform the task almost automatically and can attend equally well to other cognitive demands at the same time.

Germane cognitive load consists of the work of converting the information in working memory to permanent knowledge, or what the psychologists refer to as a schema. You can think of germane cognitive load as the physical actions of a bank employee storing your shoebox full of cash in the bank’s vault for you to retrieve when needed at some future time. Germane cognitive load is the mental effort that occurs when something is actually “learned.”

Continue reading “Designing for Cognitive Load”

Stress and active learning

One for the EU specialists…

This weekend I caught up with an old friend. He works for a software company, overseeing the sales team.

Recently, he’s been doing some work with occupational psychologists, to get a better handle on the team’s stress levels. He told me about all this over a cuppa, including the SCARF model, which I’d not heard of.

SCARF is a diagnostic framework for identifying sources of stress, where individuals encounter challenges to their Status, Certainty, Autonomy, Relatedness (being part of the group) and Fairness.

There’s a useful summary (and diagram) here.

Listening to my friend, telling me how this works for his team (status is the big thing, apparently), I was struck by how this works in the educational context.

For example, one of the reasons why assessment is so stressful is that it hits most of these areas: students might feel success brings status with teaching staff, it’s relatively uncertain, it’s out of their control, and it’s not necessarily a fair way to judge achievement. The gain of a shared experience with other students pales next to all this.

Clearly, there are general lessons about student welfare to be picked up from this model, but it’s also useful to consider how it relates to active learning.

In traditional, transmission-centred approaches, life might appear to be relatively stress-free: most of the time you sit then, soaking up material, with the occasional bouts of panic at assessment time.

By contrast, active learning might be more challenging.

The biggest issue is likely to be the increased requirement for autonomy: active learning requires participation and the production of contributions on a rolling basis. This front-loads requirements on students, at a point where they might feel they know relatively little (raising issues of status (you want to look good in front of friends) and relatedness (you don’t want to get marginalised in the group if you fail)).

Similarly, the relative absence of the instructor means students have to self-regulate more than usual, so fairness might become more of a factor than in a situation where fairness gets imposed from above.

And it’s also worth highlighting that the model points to active learning being more stressful for teaching staff too, with lower status, higher uncertainty and a big hit to autonomy: no longer is everyone doing just what you want of them.

Despite this, I think that active learning’s benefits outweigh these costs.

Firstly, precisely because students are brought actively into the process from the start, they have much more time to prepare themselves for any summative assessment, both in terms of having to consider materials and of practising producing ideas. The stress is spread out, rather than concentrated at the back end.

But equally, if stress is managed properly, it also comes with raised engagement. If we are making our active learning spaces safe (as we always should be), then we are offering students both the opportunity and the tools to manage stress better, which not only points them to thinking more about the matter in hand, but also how to deal with other sources of stress in their life.

We’re helping our students to learn about the world and how to engage with it. That means skills matter at least as much as substantive knowledge. And handling stress is one of those skills. Yes, active learning is more stressful for all involved, but the benefits that flow from that are ones that might serve us all well.

Study: Students feel they learn more from lecture…but don’t.

Everyone should check out this important study by Deslauriers et al, published recently in the Proceedings of the National Academy of Sciences and currently open access. It outlines an experiment at Harvard that tested direct learning in an introductory physics class compared to indirect reports of learning. The takeaway is that students reported they learned more during the lecture—but performed better on quizzes taken following active learning sessions. This has tremendous implications for how we do active learning research–and shows the dangers of relying on student reports of how they learn.

In the experiment, students attended 11 weeks of the introductory course together, and then in the 12th week were randomly assigned to two groups–one with an instructor giving a compelling lecture, and the other with a instructor running a session using active learning techniques. The instructors were both well versed in active learning approaches and had experience in giving great lectures. Students took a survey afterward reporting on their learning along with a 12 question quiz on the material (created by a different instructor to prevent teaching to the test). In the following session the instructors changed their method, so each set of students experienced both a lecture session and an active learning session. The material in the lecture and active learning sessions was identical, as was the handout. In the lecture, the instructor worked through slides based on the handout and solved problems with students passively observing and filling in the answers, while in the active learning session students worked in small groups to solve the same set of problems with the instructor offering assistance as needed. As the authors say “students in both groups received the exact same information from the handouts and the instructor, and only active engagement with the material was toggled on and off” (2).

Students reported greater frustration with the more disjointed nature of the active learning exercise, and thought they learned better from the flow of the lectures, but the researchers found that students performed better on the quiz instrument on the material in their active learning sessions.

We’ve often noted when reviewing research on active learning techniques that indirect measures of learning–that is, student reports on their learning–are not ideal, but this study shows us one of the dangers of relying on such instruments. Less than stellar support by students can derail efforts to increase active learning in a particular institution. We need to be more cautious, then, in how we examine and evaluate evidence that supports–or opposes–the use of active learning in the classroom.

Audio Feedback and Transparency as Teaching Interventions

This is a review of “Enhancing formative assessment as the way of boosting students’ performance and achieving learning outcomes.” Chapter 8 of Early Career Academics’ Reflections on Learning to Teach in Central Europe, by Nikita Minin, Masaryk University.

Nikita Minin of Masaryk University is motivated by a goal we can all appreciate: ensuring that his students achieve the learning outcomes of his course.  In his case, the course is a graduate seminar on theories of IR and energy security and the learning outcomes include improving student skills in critical thinking and writing.  He noticed that students in his class did not seem to really improve on these skills during the class, and introduced three teaching interventions in an attempt to fix this. 

First, Minin provided more intense instruction on the writing assignments at the start of the course, providing a grading rubric and examples of successful student work. Second, he gave students audio rather than written feedback on their papers.  Finally, using a sequential assessment system, the instructor gave formative feedback first and grades much later in the course. Minin assessed the impact of these three interventions, comparing course sections with and without them, and concluded that the first two interventions achieved the objective of improving student achievement of the learning outcomes.

The interventions described in the chapter are in line with current thinking regarding in-course assessment. While Minin does not use the language of transparent teaching, his first intervention falls exactly in line with the Transparency in Teaching and Learning Project’s (TILT)approach. Transparency calls on instructors to openly communicate about the purpose of an assignment, the tasks they are to complete, and the criteria for success, and Minin does exactly that in this first intervention.  Given the data so far on the TILT project, it is not surprising that Minin saw some success by taking this approach. Likewise, now-ubiquitous learning management systems allow for giving feedback in multiple platforms, including audio and video. For years now, advocates for audio-based feedback claim that this can be a more effective tool than written feedback. Minin’s observations therefore, also fit nicely in line with existing work.

Where the chapter falls short, then, is not in the design of its interventions, but in the claims made based on the available data. The sample sizes are tiny, with just five students receiving the interventions. With final grades used as the primary dependent variable, it is difficult to tease out the independent impact of each of the three changes. Using final grades is also an issue when the experimenter is also the person who assigns grades, as it is more difficult to avoid bias than when more objective or blind items are used. Lang’s (2016) bookSmall Teaching: Everyday Lessons from the Science of Learningtells us that engaging in self-reflection is itself an intervention, and Minin’s use of minute-paper style self-reflections to assess the impact of feedback, while itself an interesting and potentially useful idea, mean that a fourth intervention was used in the course.  While I do not doubt Minin’s observations that his interventions had a positive impact, as they are backed by existing research, the evidence in the chapter does not strongly advance our confidence in those findings.

However, I have never been one to dismiss good teaching ideas simply because of a lack of strong evidence from a particular instructor.  Minin highlights a crucial concern—that we should never assume that our courses are teaching what we intend them to teach, and that ‘time and effort’ do not necessarily achieve the desired results, even for graduate students. Reflecting on this, seeking out innovative solutions, and then assessing the impact is a process we should all be following, and Minin sets a great example.

(Trans)formative Assessment in Teaching

Today I’m attending ISA’s inaugural Innovative Pedagogy Conference in St. Louis.  Victor and I are doing a workshop on using games and simulations to teach political violence, showcasing activities like Survive or Die!, Prisoner’s DilemmaModel Diplomacy, identity exercise, and others.  But I’m most interested in reflecting on the session offered by Matthew Krain and Kent Kille of the College of Wooster on Assessment and Effectiveness in Active Teaching in International Studies. Their framework for formative assessment (that can, in fact, be transformative) is very helpful as an overall approach to teaching.

Continue reading “(Trans)formative Assessment in Teaching”

Two Online Games From The New York Times

Last month The New York Times published an updated version of its confirmation bias game that might be useful for teaching research methods or political psychology. The newer version includes an explanation of how confirmation bias affects government policy.
Also of note is another game on President Trump’s plan for changing U.S. immigration criteria. I failed to qualify under these new proposed rules. Probably all of my students will fail also.

Ethnography of Consumption – A Twist

Buying is believing

The spring semester is ending here, and as usual everyone is dragging themselves across the finish line. There is not much left for students to do in my globalization course, but — given the structure of the U.S. higher education system — we still have to meet in the classroom. So, in search of something to fill time, I created an activity based on the last paragraph in my response to Leanne’s scavenger hunt exercise.

I collected some random products from home, all branded: running shoes (Saucony), cell phone charger cord (Samsung), bag of lentils (Jack Rabbit), hand towel (Cannon), and a tote bag to transport everything (London Review of Books). In class I laid all these items on a table and lied about getting them from a friend. I then told students to examine everything and write an answer to “What story is the owner of these things trying to tell others about the kind of person he or she is?” Students had already completed ethnographies of consumption, so they had some understanding of the connection between identity construction and consumption choices.

After five minutes, I grouped students into teams to create presentations, which they delivered during the second half of class. I got to listen to them unknowingly analyze my self-image as practical, value-oriented, and health conscious. (I would have been just as happy with other adjectives.) Once the last team had presented, I revealed that the items were my own by plugging my cell phone into the wall socket with the charger cord.

As a last minute invention, this activity worked fairly well. Students got to practice collaborating, communicating to an audience, and applying concepts. And all I had to do was bring a tote bag to class.

Teaching Trump #2

A follow-up to Amanda’s post about teaching in the time of Trump. Trump’s campaign and administration represent an opportunity to teach about racism in the construction of national identity. I stumbled into this subject accidentally in my globalization course last week, with an assignment that asked students to write a response to “Is France or the USA a more global society?” Students were supposed to reference the following items:

Yesterday I heard this radio report about residents of Newton, Massachusetts, one of the wealthiest and therefore whitest communities in the USA: Continue reading “Teaching Trump #2”

The Change All Around, Part 1

The spring semester is about to begin, and as usual I have changed what I will be doing and how I will be doing it. I thought it might be interesting for the dear readers of this blog to learn about some of the factors that will affect my teaching over the next few months. But just to be contrarian, I’ll start in reverse chronological order:

I’ve decided to use Michelle’s technique of placing simulations at the end of the semester when both students and I will be approaching burnout mode. I have done a complete redesign of my comparative politics course, dropping the themes in favor of more traditionally-presented content like the diverse institutional arrangements of democracies. In an attempt to get at this topic, I have resurrected my Gerkhania simulation, but have tweaked it to give it more structure. First, I will assign students roles that have different ethnically-oriented, two-sentence backstories. The class will then convene as the Gerkhanian Commission for National Reconciliation, which has the following tasks: Continue reading “The Change All Around, Part 1”