Notes from a conference III, or: why you hate grading

It’s probably bad to write three posts, one straight after the other, but hey: tough.

We’ve got more grading/assessment-related posts coming from people more on it than me, but a couple of times in the first days of ISA, we’ve had the classic teaching-related panel moment of everyone nodding along to ‘but everyone hates grading, right?’

[Parenthetically, one of those moments led to talk of how good it would be if our jobs didn’t including grading at all, whereupon I noted my job doesn’t include it, to many jealous looks. I’m mostly happy that I have so little grading to do (like a dozen scripts a year little), but part of me does miss it, mainly for the reasons set out here.]

The obvious question that no one really seems to keen to investigate is why we all seem to hate grading.

After all, it’s the main mechanism we have for evaluating student learning and, by extension, whether our teaching is getting the results we intend. It’s more rigorous than a chat at the end of class about learning gain or the impressions we glean from observing debates in the classroom, so why are we so down on it?

Let’s consider the usual suspects.

First up, ‘it’s repetitive’. Sure, if you have a pile of a hundred essays on the same topic, that is super-dull, especially if there’s a lot of overlap in the intellectual sources your students used (your lectures, the core textbook, the readings you suggested): it’s a rehash/mangling of what you’ve taught them and it’s probably not pretty.

So don’t make that your assessment. Give them choice, make them responsible for finding their own sources, get them to bring something more personal and individual.

My old negotiation course asked students for their only assessment point to tell me what they had learnt from the course: I was emphatic that it could be absolutely anything, as long as it was grounded in what had happened in class and was connected out to whatever literature they had found useful. Result was such a rich variety, every time I ran that course.

Second up, ‘it takes such a long time.’ Partly that’s about the time-elongating/mind-numbing of reading the same stuff a bunch of times that we just discussed, but partly it’s our demands of the process. Often, we normalise the essay of X pages or Y thousand words as proof of learning, even as we rool our eyes at ‘all that text’.

So go for concision, not length. You know it’s harder to produce a good short text than a good long one: the demand to focus much more tightly on what’s important, to lose the blah, is something we find tricky (to judge by the number of ropey abstracts you and I have read). Set a much shorter length piece, or ask for a 2-page policy brief, or a poster, or an infographic, or a 3 minute video.

Of course, shorter assessment outputs doesn’t automatically mean quicker grading, but it helps. As does trying things like videoing your feedback, or asking students to make their own evaluations of their or their peers’ work.

Finally, ‘students find it boring/unpleasant (and so do I)‘. Find me the student who loves to do assessments and I’ll just start wondering what happened to them to make it come to this. I get that everyone thinks it’s an imposition and chore.

So involve your students in finding assessments that engage them more/turn them off less. Amanda’s breakfast exercise for research methods is my go-to here: super-fun, super-engaging allowing them to really capture their experiences in an assessment.

If assessment is about cementing the bond of teacher and learner, then why wouldn’t you want to bring them in to find a mutually-satisfactory assessment regime?

Unless you’re working under a highly prescriptive institutional system, you hold the power to make your assessment and grading less boring. So use it.

Notes from a conference II

Probably a learning moment here to, but more just a reminder that we are an award-winning blog

Apart from my worries, ISA is also making me think about the purpose of teaching.

I’d been thinking about this anyway, partly because of this post from John Warner that popped up on my timeline recently.

The instrumentalisation of education as a stepping stone to the next thing, working, has been around for a long while now and it is pernicious in its compromising of student engagement: what they are told they need is a good grade, not a good education and given the incentive structure we shouldn’t be surprised if they act accordingly.

I’m guessing you’ve had these students in your class, more or less often. Either the ‘just tell me what I have to do to get a good grade’ version or the ‘my corporeal self is present, but not my mind’ version.

Regardless of whether we cleave to a knowledge transmission model of teaching, or (more likely among our readership) an active learning approach, you still rely on the student being present and engaged.

So what to do?

One of the conversations I sat in on here spoke about teaching as problematisation; making things uncertain to open up space for reflection, debate and development.

I like that idea. We aren’t here to solve the world for our students, but to invite them to create their own understandings of situations and problems, to recognise and act on their agency in that world.

But again, if our students aren’t in the metaphorical room, what good is challenging conventionalities?

My reflection is that much of what I have done over the years is try to distract students from their learning.

Using games and simulations, taking the class out onto the campus lawns, literally removing the physical and conceptual constraints of ‘the class’: it’s all grounded in a notion of ‘I’m not expecting this: what is it?’

The disruption of the moment hopefully leads to the delight of doing something different, and that hopefully leads further to ‘dammit, I’ve been learning stuff’.

Colleagues here have spoken about how alums tell them how they remember the random moments – the simulation, the talk of Russian hookers, the exercise with the blindfold – long after they forget the ins and outs of whatever IR theory was on the syllabus that week.

It’s why I will always argue that those who say fun can’t be serious are wrong.

Precisely because fun isn’t (linguistically) serious, it opens up the possibility of serious learning, of helping students to gain insight from all of their interactions with the world.

That’s why I can learn from the movie I watch on the plane, why I can learn from watching my child play with Lego, why I can learn from a big erection (you’re not clicking that link, are you?): because all these things are surprising moments to learn. The only difference is that I’m consciously looking for that.

Our present-not-present students are not, but that doesn’t mean we can’t keep on challenging their assumptions about what ‘learning’ is and keep on offering them cause to think.

It’s not foolproof, it’s not a comprehensive solution, but it’s a way in.

Notes from a conference I

Also worried about whether the duck can cope with that harsh New England climate

We’re a bit more than half-way through ISA here in snowy/rainy Montreal and I’m concerned about a number of things.

Most mundanely, I’m worried that even as we’re getting people interested in writing some guest posts for us, the top post is about the movie I watched on the way over.

Almost as mundanely, I’m worried that I’m not built for such an extended period of rich dining.

But mostly, I’m worried about how I teach.

There have been lots of roundtables and discussions about teaching at the conference, touching on all aspects of our practice. It’s been a while since I’ve had so much time to reflect and debate these points, in the broad way that we have been.

Obviously, much of this has been couched in the language and frame of the US experience, with its particular concerns and touchstones. But abstracting from that isn’t so much of a problem as the nature of my work.

As regular readers (who are more numerous and engaged than I realise: props to you all) will know, I work in a distance-learning University, where I never interact directly or synchronously with my students.

Much of our discussions have been about building the relationship with the students in our class, to create a space in which to exchange and learn, however we might specifically go about that.

But I don’t have that. Instead, I have to imagine my students.

Imagine who they are, how and why they study, what they know and what they want to know. My colleagues and I take an educated guess and pitch our materials in that general direction.

And then some (long) time later we find out how those students did on assessments, get some feedback and tack for the next time around. Not that the next time will necessarily be the same types of students.

I’d not really thought about this until now, mainly because I know that the system works pretty well in practice: student performances are strong, feedback is positive, our institution’s reputation is sound – none of that would be the case if we weren’t getting it right most of the time.

But the general is not the particular: just because we got it right before, doesn’t mean I will get it right now.

My worry is perhaps the worry we all hold, whatever our situation – that there is a risk in teaching, for our students, for us, for the process.

That worry is also perhaps what keeps us motivated to be thoughtful about what we do. By turning our worries into action, we might forestall the failure, we might improve our chances of making a difference, we might help our students to be, well, just more than they were before.

So I’m owning my worry, because my worry is pushing me forward, not holding me back.

And it shouldn’t hold you back either.

Don’t think, just do: Why Maverick is a bad teacher

Yes, I’ve been sent across the ocean once again, this time to ISA in Montreal.

We’ll be talking about teaching here (although not as much as planned after the cancellation of the pre-conference day) so I thought I should use the flight over to bone up on my movie representations of teaching.

As I’ve previously discussed, good teaching is apparently beyond any film producer, so I didn’t approach Top Gun: Maverick with much confidence. The film has already stimulated a pile of thought pieces about what it says about the US’s confidence in global leadership (spolier: not much positive), but no one has recognised the insights that Maverick has to convey about the practice of pedagogy.

[We’ll pause here to say that I probably have lots of spoilers hereafter, but frankly, if you didn’t already see the film, then you’re not going to now, so tough]

At first blush, this is a movie all about active, experiential learning: the team have to discover for themselves how to perform the seemingly impossible task, with Mav as some kind of sage mentor to help them believe in themselves. There are threshold learning moments, as pilots find out what more is possible when they do things like, um, go faster or, err, disobey orders. Hence the recurring motif of ‘don’t think, just do’.

There’s also a heterodox spin here: rule-breaking here is the goal, to help get inside the enemy’s OODA loop. ‘It’s not the plane, it’s the pilot’ comes up quite a lot and the unpredictability of the latter is the key to defeating SAMs and 5th-Gen fighters.

But all this falls down on the conceit that Maverick himself is the best there is: his excellence is the key to unlocking his students’ potential.

He is the one who has to show that the timed bombing run can be done within the time, that the g-forces can be withstood, that the fractious individuals can be knitted into a team through a session of beach ‘football’.

Yes, this is a very particular kind of teaching situation, with a very specific learning objective, but leaves us with the trope of teaching-as-performance, where students are to be stunned into learning by the sheer majesty of what they see before them.

If nothing else, most of us aren’t those kinds of people.

And nor should we be.

Active learning is about helping students to see the doors to learning and getting them to open those doors and walk through: not smashing them down then showing students the limits of what can be done beyond them.

The objective is not to be the hero of the classroom, but its facilitator, because the classroom is about the learners, not the teacher.

Which is why I always like coming to conferences like these, where I can learn from others who’ve done stuff that I haven’t and we can reflect on how to move things forward.

[I also watched Tom Cruise in Edge of Tomorrow on the flight, which I now realise also involves a lot of teaching, but you really don’t want my opinions on that one]

Performing as academics

What’s your evidence base for this?

I had the too-rare pleasure to chat with an anthropologist last night.

We are both part of a big European-funded project that is having its final conference in Brussels this week, with a focus on connecting to policy-makers and other practitioners in the field of refugee protection.

It’s a big deal, for me as much as anyone: I’ve been leading on the conference planning for the past 18 months as we bring together speakers from at least three continents and juggle a very large number of moving parts.

The conclusion of the first day has shown that planning does pay off, from bringing some marker pens that got used within ten minutes of arriving to thinking about the structure of the sessions.

Let’s focus on the second of these elements here, important though ready access to marker pens might be.

From the start, I was clear with myself that I didn’t want this to be an academic conference.

By that I mean the format you know from pretty much every single big conference you go to: 90 minute sessions with 4 or 5 papers, hopefully not just read out in a monotone but quite possibly, with 5 minutes for someone to make more of a comment than a question.

The world has plenty of those kinds of things already and this project does not need to add to the pile.

Our work has a very clear and evident policy application: studying refugee protection in Europe and beyond was already a big thing three years ago when we started, even before the effects of covid or the war in Ukraine shook things up even more.

So I wanted to break out of the mold as much as possible.

My initial suggestion that we should just have an afternoon where we sat in an informal space and attendees came up to chat with us was not popular, so we rowed back to the format we have now: some keynotes, some roundtables with significant non-academic presence, and lots of time for chat/networking.

In broad terms, this seems to be working in breaking down barriers between academics and practitioners: our lead investigator did a great job of opening the event with a clear single message about what our work shows to practitioners, for example.

But it’s also hard.

At points, we drift back into the comfort of roles – and this is where the anthropologist and I found common cause – rather than challenging ourselves to push back and to make things work for us.

This isn’t even so much about trying not to talk in jargon or to argue on precise points of definition (although also that), as it is about falling into classroom modes, with lecturers and students, each doing what they are supposed to.

If we want to build more effective and significant links from academia into other communities then we have to be mindful of this; we have to try really hard to change the incentives for our colleagues to adapt appropriately.

That means more work on formats, more challenging of assumptions, more reflection about the specific needs of others.

I’m taking this conference as a win even before it ends, but I know I could do better and next time I will try harder.

Getting your bearings

Not in full-time education, bu still interested in learning

My household is full of people involved in education, either as students or as teachers. However it’s only this academic year that my (much) better half has moved into a regular teaching role, after years of research-only activity.

It’s been great to see how she has moved from anxiety about what to do to reflection on how best to make her classes work and how it all marries up to her research activity, not least because it’s also given me an opportunity to reflect on what I do too.

Indeed, it was this process itself that was perhaps the most stimulating, since it highlighted one difficulty for a new teacher, namely what – practically – to do.

Training programmes are typically on-the-go, using your classroom experience to stimulate reflection on pedagogy and learning and theory-level stuff. But how to get to a point of being confident about stepping into that classroom in the first place?

What seems to be missing is a simple ‘how to’ guide that offers practical advice on really mundane stuff, like dealing with questions, or designing your first lecture, or what to keep in mind for your first session.

All the stuff you have internalised from your teaching, basically.

Looking around, I don’t see a resource like this, which surprises me (and also makes me wonder if I’m just looking in the wrong places): we all have to start somewhere.

In my case, a long time ago, it came from a lot of watching people teach/try to teach me and reflecting on what did and didn’t work. But that’s not a normal path, I’m aware.

Maybe you came to this some other way, in which case I’d love to hear about it, or indeed anything like the resource I describe.

This comes back to some of the themes we touched on in Baltimore, namely lowering costs to entry to better pedagogy: it’s as true for ‘basic’ stuff as it is for fancier approaches like active learning. One reason why teaching doesn’t have the same rep as research is precisely that we don’t institute the same level of training in the latter.

But that might another post for another day.

Advice for Job-Fillers

Although probably too late in the calendar year to put into practice, here is a flip-side follow-up to my prior post about campus interviews:

It’s time to stop the costly performative rituals that are contributing to the disintegration of the academy.

Move to a bi-annual or even tri-annual hiring cycle. Academia is the only industry that I know of that limits hiring to an annual schedule. If you operate your searches on a staggered timeline, your applicant pool will probably improve and your top choices will be less likely to be snagged by competitors.

Stop requiring recommendation letters at the beginning of the search process. Demanding otherwise wastes the time of letter writers and applicants. We all know that an overwhelming percentage of these letters are never read because the applications they are part of quickly get tossed. Get references only for those on your short list and then check them.

While the science supports eliminating job interviews entirely, this probably isn’t going to happen, so at least make them less onerous. The pandemic demonstrated that there is no need to bring finalists to campus. And there is no demonstrated benefit in subjecting them to one to two days of back-to-back meetings with people who have no direct effect on the specified duties of the position. Is it essential for every candidate to have a 30-minute conversation with the Associate Director of Strategic Student Development Initiatives? No one who interviews for an IT or facilities management staff position has to suffer through this, and those units function perfectly well.

Finally, per the article linked to above, structure the applicant evaluation process to minimize bias and noise. Use rubrics to score candidates on clearly defined criteria. Collect the results, average them, and distribute this information to the search committee before discussion of the applicants’ relative merits. This will help prevent any single person in the room from unreasonably affecting the outcome.

#TLC2023 prep time

I open by freely admitting that this post is totally self-serving, but since it’s the first time in ages that I actually wrote the paper ahead of the conference, I’m going to extol the virtues of writing your paper ahead of the conference.

APSA’s TLC is nearly upon us and the heritage it brings of a track model means that pre-written and -circulated papers is still very strong. You can check out the submissions so far here.

As an opportunity to get focused and specific feedback on your work, especially if you’re looking to publish, this is pretty hard to beat: getting a bunch of people working in the same field, in a room for most of two days, sharing ideas, isn’t something you get to do that often in conferences.

And if, cough, one of those people is an editor on a leading L&T journal, then all the better.

Having already started to work my way through the papers on my track, I can see we’ll be having a lot of very productive debate in Baltimore, which we’ll be doing our best to share with you all.

How soon is too soon?

We’re in the process of building some new undergraduate modules for our degrees here and it’s been reminding me of a rather basic dilemma in programme design: when should we start students on the ‘hard stuff’?

The air quotes point to a first issue, namely that there are different kinds of ‘hard stuff’ out there.

Part of it is topics that require more advanced skills or more detailed knowledge, often applied, that seems to rest on a strong foundation of core competences: think advanced quants modelling or very particular policy issues.

But it is also the openly structured, student-led work in general. Obvious case in point: we put the dissertation at the end so the student can explore a subject in their own way, having got a bunch of stuff from us.

You’ll be shocked* to learn that I disagree with many people about this model.

Firstly, we typically get students joining us who’ve been socialised into a particular learning model by schools – much of it learning to hit the ‘right’ elements from the curriculum – that we try to shift towards more critical and self-reflective approaches, so why not work on that from Day One?

Secondly, as the examples above show, you could argue that dropping students straight into more ‘advanced’ work might be a strong incentive for them to buy into different ways of working, plus if you know your students are new to it then you can make some allowances about how far they can go.

Case in point: I’ve been looking at courses with my son and there’s one that involves an independent project every year of study. It’s a good way to highlight the applied value of the other content, practice for more ambitious work down the line and a training ground for thinking and reflection.

Finally, given how most degree classifications put weight on later stages of study, does it make utilitarian sense only to be introducing ‘hard stuff’ in those later stages, instead of early on, when they can iron out the wrinkles?

In practical terms, my experience has been that first years are more willing to do something new (because they don’t know ‘how things are’) and they have more ability than we typically recognise, even if their achievement isn’t the same as a final year student.

If we’re using active learning systems, then that’s all fine, because these aren’t so predicated on prior learning, but instead give space to students to take things as far as they can. Moreover, the applied nature of many such environments also makes it easier for new students to understand the wider value of what they’re doing.

Yes, it also means having a robust system of student support as they make the transition, but that should be something that we provide in any case. Indeed, the values of more advanced study – reflection, criticality, resilience – might ultimately help them to feel better able to support themselves.

None of this is particularly novel (as Amanda could tell you), but it’s still good to be reminded of it as we build new content.

Our confidence in building ambitious learning environments for our students is likely to translate into students who can become more confident about their learning.

* Not shocked

Advice For Job-Seekers

We’re in peak season for campus interviews. At this stage of my career, I’ve had and seen many of them. So a bit of advice to those whose applications ended up at the top of the list:

We’ve all probably heard the statement, “No one in the room knows as much about your topic as you do,” intended to alleviate the anxiety of speaking before an audience of strangers. In an attempt to strengthen the performance of job candidates, I now propose the Chad Raymond Corollary: “No one in the room is as interested in your topic as you are.”

If your interview includes a research presentation or a teaching demo, practice multiple times — by which I mean full dress rehearsals, not just reviewing what you think you’re going to say in your mind. Boil the talking down to one or two main points. Eliminate words that you stumble over. Whittle down the content until you can deliver the entire presentation at a steady, deliberate pace within the specified time limit.

Then cut at least another 25 percent. Preferably more.

Why? Expect a delay because of the technology in the room isn’t working properly. Then introductory announcements by hosts. And you will need to entertain questions at the end. But mainly because people don’t really want to hear someone else speak non-stop for 45 minutes about a topic that has less import to them than what they will eat for dinner that night.

The above also applies to conference presentations.

Last, and I can’t stress this enough, figure out how you are going to engage your audience. If you prompt people in the room to do something, they are more likely to have a stronger, more positive view of your abilities. At minimum, they won’t be looking at their phones.