Seen Exams

Everyone’s working with seen papers…

This past semester I got to try out using a seen exam for the first time.

For those of you unfamiliar with it, you publish the exam paper some time ahead of the sitting date (a week, in this case), so students can prepare their responses, which they then write under controlled exam controls (without notes or materials to hand).

The logic of this is that it provides a more meaningful test of students’ abilities, since they since have to revise, plan and produce, but without the added peril of “I can’t find a question I can do” or “I answered the question wrong”.

Having inherited the format from a colleague, I was keen to try it out, especially since last year’s use of an open-book, online exam had worked very well. Indeed, this year’s module was with the same students.

The practicalities are very simple indeed: an email to the class and a posting on the VLE at the appropriate time, plus being available through the week to answer any queries or clarifications.

The day before the exam I emailed everyone again, just to run through any points that had come up and to remind them again that the format meant some things were different from a ‘normal’ exam.

Firstly, my expectations on factual accuracy would be higher, since they’d have had time to prepare.

Secondly, I’d like to see more references to the literature: not direct quotes, but certainly mention of relevant authors.

And most importantly, I’d expect clear organisation and argument in each of their answers.

So?

Having now finished my marking, I’m able to say a bit about how this all played out.

As with the other format, this approach seems to be good for pulling up the tail of students who might otherwise have found things difficult: even the worse-performing student still produced relevant answers with some detail.

Likewise, the almost total absence of factual errors and of very short answers was a pleasant development, suggesting everyone had actually done work for the exam.

So the knowledge front seems to be positive.

Having seen a few students straight after the exam, I’m not sure that they found it any less stressful though: yes, they knew what the questions would be, but they also noted that they were also conscious I would be marked in line with that, so maybe their extra work wouldn’t count for anything.

While we’ve yet to complete all the feedback cycle, I think that anxiety is understandable, but hasn’t played out. Instead, the performance of the class has been strengthened and their capacity in the subject will be that bit more for future modules they take.

In sum, this exam has further convinced me that closed-book, unseen exams aren’t that useful, either in measuring knowledge or managing student stress: unless I have to use them in future, I’m not going to be.

Stress and active learning

One for the EU specialists…

This weekend I caught up with an old friend. He works for a software company, overseeing the sales team.

Recently, he’s been doing some work with occupational psychologists, to get a better handle on the team’s stress levels. He told me about all this over a cuppa, including the SCARF model, which I’d not heard of.

SCARF is a diagnostic framework for identifying sources of stress, where individuals encounter challenges to their Status, Certainty, Autonomy, Relatedness (being part of the group) and Fairness.

There’s a useful summary (and diagram) here.

Listening to my friend, telling me how this works for his team (status is the big thing, apparently), I was struck by how this works in the educational context.

For example, one of the reasons why assessment is so stressful is that it hits most of these areas: students might feel success brings status with teaching staff, it’s relatively uncertain, it’s out of their control, and it’s not necessarily a fair way to judge achievement. The gain of a shared experience with other students pales next to all this.

Clearly, there are general lessons about student welfare to be picked up from this model, but it’s also useful to consider how it relates to active learning.

In traditional, transmission-centred approaches, life might appear to be relatively stress-free: most of the time you sit then, soaking up material, with the occasional bouts of panic at assessment time.

By contrast, active learning might be more challenging.

The biggest issue is likely to be the increased requirement for autonomy: active learning requires participation and the production of contributions on a rolling basis. This front-loads requirements on students, at a point where they might feel they know relatively little (raising issues of status (you want to look good in front of friends) and relatedness (you don’t want to get marginalised in the group if you fail)).

Similarly, the relative absence of the instructor means students have to self-regulate more than usual, so fairness might become more of a factor than in a situation where fairness gets imposed from above.

And it’s also worth highlighting that the model points to active learning being more stressful for teaching staff too, with lower status, higher uncertainty and a big hit to autonomy: no longer is everyone doing just what you want of them.

Despite this, I think that active learning’s benefits outweigh these costs.

Firstly, precisely because students are brought actively into the process from the start, they have much more time to prepare themselves for any summative assessment, both in terms of having to consider materials and of practising producing ideas. The stress is spread out, rather than concentrated at the back end.

But equally, if stress is managed properly, it also comes with raised engagement. If we are making our active learning spaces safe (as we always should be), then we are offering students both the opportunity and the tools to manage stress better, which not only points them to thinking more about the matter in hand, but also how to deal with other sources of stress in their life.

We’re helping our students to learn about the world and how to engage with it. That means skills matter at least as much as substantive knowledge. And handling stress is one of those skills. Yes, active learning is more stressful for all involved, but the benefits that flow from that are ones that might serve us all well.

…like riding a bike

It is the most basic of tropes about learning. It’s tricky to start with, but once you’ve learnt to ride a bike, you never forget.

Writing as someone who’s just started their fifth calendar decade of bicycle riding for pleasure, as someone who’s ridden up (and down) mountains, commuted by bike for years in various urban areas, and who’s taught his kids to ride, I’d just like to challenge this.

The reason?

This.

That’s right: I’ve bought a tandem for a very reasonable sum on EBay.

It’s a bike – it’s got only the two wheels – and it works in just the same way as all of the other bikes I’ve ever owned or ridden.

But it’s also very different. I now have to think about the other person on the machine and what they’re doing and how I will need to communicate what I’m doing.

And even getting past that, how I ride will have to change very markedly too: the brakes – like the bike – are relics of a past age, so assumptions of stopping distances will have to alter radically. As will my memory of indexed gear-shifting at my fingertips.

None of this is going to be helped by the fact that I’m going to keep on riding my other (‘normal’) bikes, so the potential for immersing myself into this is constrained, even if I can rustle up a family member to share this experience with me.

In short, I appear to have acquired a large metaphor for the learning process.

Right now, I’ve had this metaphor for a grand total of two days and I’ve got as far as the end of the road with it, so I’m still at the stage of not even being particular sure what it is that I will need to learn.

I’m especially worried about doing that learning with a loved one right there, learning too: the various scars on my body are testament to my periodic efforts to understand the limits of what I can do on a bike.

Much as could go back to that other great saying about bikes – when you fall off, you’ve just got to jump back on – the sense of responsibility is somewhat constraining. Indeed, much more so than I feel in a classroom when trying something new: at least there the failures don’t result in road rash or broken bones (unless something’s gone extremely wrong).

But I can also tell you that our short trip down the road also reduced us both to tears of laughter, so I already know that this can be an enjoyable process: indeed, that’s why I got the thing in the first place.

So if you find I’ve stopped posting here, then maybe it’s because I’m off having an adventure on a tandem with a loved one. Rather than because we’ve had a crash.

Back in the saddle

Apart from a dull ache from the thought that the 1990s are now, on average, a quarter-century ago, it’s been a good break.

It’s been especially good to get away from all those social media posts about how much one has changed over the past decade, complete with youthful/haggard profile pics to chart one’s maturation/decline.

The usual thing to think about at this time of year is what you’ll change.

I’ve already had one colleague inform me they’ll be focusing on writing every Friday, even as they undermined it by querying whether it will actually happen.

So I’m going to suggest you try not changing things right now.

Instead, focus on what you do that works. It’s a bit harder than picking out what’s not working, but it’s a more positive starting point.

In practice, it’s what most of us do in any case: building out from successes and chipping away at the problems around the edges. Sure, it’s probably not as likely to treat root causes of those problems, but if the latter are really ramping your style, then we should be having a different conversation.

Part of this is about perspective.

Writing personally, I’ve been thinking a lot recently about how I manage my work and my career, probably as a result of a podcast interview I’ve discussed already.

I’m aware that I’ve got various pressing deadlines right now, and that this post is another element of my avoidance strategy, and I’m aware that I could be doing a whole bunch of other things that would be Good Things To Do too.

But instead of succumbing to the winter blues, I’m trying to pull myself out of it, reminding myself that I am reasonably competent and have handled much worse situations than this before.

It doesn’t mean it’s going to be easy or particularly enjoyable, but the prospect of what is to come cheers me up.

As one trivial example, I made a simple wall-planner for the half-year, listing my various events and talks. Not only does it remind me of More Stuff To Be Done, but it also makes me think about how much I enjoy sharing my work with others.

So, stick your head out the door and notice the days getting longer*, and think on reasons to be cheerful.

* – Sorry to any Southern Hemisphere readers on this one

Left-field lessons

The other week, I sat in on a session run by another part of the university for people to show their teaching practice.

One of the people presenting kick-started their slot by asking everyone to write down their definition of ‘education’ on a post-it, which they then used to elaborate on some key themes.

Of course, one person wrote down something that bore little relation to what everyone else had done.

Oops.

The presenter choose not to get into whether education really is a system of indoctrination, and the person who wrote it down didn’t press them on it, but it did raise an more general question.

How do you cope with stuff coming in from left field?

I’m guessing you’ve had this too: running a session, then someone either saying something so far from the mark that you worry they’ve totally misconstrued things, or offering up a very radical take.

In both cases, you really need to explore what’s happening, either to offer a corrective or to embrace the new breadth that opens up.

This has really been brought home to me this semester by the new course I’ve been teaching, which involves the use of a lot of critical theory to understand European integration.

It’s a course I inherited from a colleague, who was kind enough to let me use her materials, which I largely retained, because I wanted to challenge myself.

That’s been a really positive experience, both because I’ve had to reconsider the ways of talking about the material and because I’ve had to learn about using some new methods, so I can teach about using those new methods.

Fortunately, the use of critical approaches does necessarily invite challenging of ideas and approaches, so the space was very conducive to working with the broad range of ideas present, but it’s something we have to work on, whatever we’re doing.

Personally, I see such moments as opportunities to get students to articulate their thinking and to connect that to what else is happening in the room, which ultimately serves everyone’s learning: if nothing else, if one person is struggling to make sense of a point, then others will most likely be too.

But back to that teaching session.

You’d all recognise that panicked expression as the presenter read out the helpful/unhelpful contribution, and the rapid adjustment of language to note that almost everyone in the room has identified some key themes.

In that case, going off into the outlier wouldn’t have worked in hitting the learning objectives, but often it will, because it is a moment either to bring the confused into the fold, or to bring new ideas to the group.

Both of those are Good Things to do, so do make the most of them.

And think about who you invite to your teaching sessions.

CALL for APPLICATIONS: 5th ECPR Teaching and Learning Summer School

The ECPR Teaching and Learning Politics (TLP) standing group together with Comenius University in Bratislava invite applications for the 5th ECPR Teaching and Learning Summer School to be held in Bratislava, Slovakia, from 6 July to 13 July 2020.

The summer school is organized around the effective use of three methods— simulation, games and academic debate. The summer school will draw upon best practice in the scholarship of teaching and learning and will offer a combination of practical and theoretical sessions.

The fee for the 2020 summer school is €270, which includes accommodation, lunches/dinners, coffee breaks, tuition, e-reader and a trip to a UNESCO World heritage site.

The applications should include a CV, a motivation letter and a statement of teaching philosophy. Application deadline is 16 February 2020. Full call for applications is available from the TLP website (here).

Petty Officer Usherwood?

It’s nearly Christmas, if anyone’s short of ideas

Yesterday I had about a third of my students turn up for class. Possibly that was related to the deadline for an essay due for me later that day – certainly the number who turned up was about the same as the number who’d already submitted the work.

Since I’ve known that this was going to be an awkward timing since the start of semester, back in early October, I’d left some of the session open, so I could be flexible about what to do, including not asking for any specific prep beyond the general reading.

In the event, I spent a block of the class talking about assessment. Unsurprisingly, since they’d already submitted, none of the students who turned up wanted to talk about the essay, but they did want to talk about the exam, which’ll be after the Christmas break.

So we discussed how that would work (we’re doing a seen-paper format, so they get it a week beforehand) and what I was looking for.

So what’s the problem?

Well, the people who turned up yesterday are the ones who most likely didn’t need the discussion, either because they’d have worked out the salient points already, or because they’d have asked. Indeed, the student who asked about the exam some weeks ago was there.

The issue is for those who didn’t turn up, the ones still working on their essay a couple of hours ahead of the deadline, the ones will the poor attendance throughout the semester.

This is a classic of the Matthew Principle: those that have, get more. And it’s not really helped by me being a bit petty-minded.

I could have waited until next week’s final class to discuss the exam – and probably someone who wasn’t there yesterday will ask about it – but I have also spent two months trying to reinforce the message that the rational choice for a student who’s finding it hard going is to come to class, because that’s the best place to get the essentials together, and to get questions answered.

Partly, this is about incentives. For my other class, on negotiation, I have great attendance, mainly because the classwork is very active and because the assessment is about what you’ve done in class. In this case, the work is more mixed and it’s not directly linked.

Maybe I need to be thinking about whether I can change that, in a way that works for the subject matter.

But maybe I also need to think more about how much this is a case of taking horses to water: where do me responsibilities lie and where do they end?

Maybe one for class discussion next week.

Why does no one come and talk to me anymore?

This guest post comes from Patrick Bijsmans of Maastricht University.

I’ve always considered myself an approachable teacher; someone students can come to with questions or worries or just for a talk. And from what I hear, I am considered to be approachable.

Still, I am noticing something that worries me. I have been having open office for about 9 years now, but fewer students have been showing up. Weeks go by when no one comes, even in periods when I am teaching and coordinating courses.

I know that I am not the first one raising this issue. It is even the topic of students’ research! But I still believe that students can learn from meeting with us for input and feedback, whether this concerns a relatively simple question or my assessment of their paper.

So, why does no one come and talk to me anymore?

Turnout during open office hours again was low during the first weeks of this year, when I coordinated and taught a first-year course on academic research and writing. At the end, students write a short paper. These are randomly distributed among teaching staff, myself plus 10 other colleagues – together we teach 25 problem-based learning groups of about 12 students. As soon as results are out, all students, whether they have failed or passed, are invited to meet with the person who marked their paper to discuss the assessment during scheduled open office hours.

This year I asked colleagues to inform me about the number of students that had shown up. The table below shows the data for those who failed the course. Interestingly one colleague had to do her open office hours via Skype; no less than 7 out of 9 students showed up. Yet, there is some research that suggests that using technology does not make a huge difference.

Number of failed students Number of failed students attending open office hours
Patrick95
Colleague 1 72
Colleague 296
Colleague 394
Colleague 4124
Colleague 594
Colleague 61711
Colleague 7159
Colleague 8174
Colleague 9156
Colleague 1097
Total12862

Why did so few students show up?

I decided to ask some simple questions to the students themselves during a session in our mentor programme. The approximately 100 students who attended (out of nearly 300) might not be representative of the group of students that does not turn up in my office. But I still learned something interesting.

Of the 86 students completing questions via an online survey tool, 36 had failed the course and 29 had attended the open office hours. Those who attended, generally did so to get clarification regarding their paper’s assessment.

Of those who did not attend, some simply stated that they passed the course and saw no need to discuss the feedback. Others referred to having been sick, stressed and/or busy with the new courses – when asked, quite a few of these students did not write to staff to ask for another appointment.

Asked why they thought others had not come, some answered that these must be lazy students or that they missed motivation because they knew what they had done wrong.

But quite a few answers touched upon something that we might all too easily overlook, namely students’ expectations regarding feedback opportunities. These answers did not just concern not knowing what to do with feedback. For instance, one student wrote that students who did not show up might be “insecure and/or uncomfortable with getting feedback”. Another student wrote that “you have limited time with the tutors and tutors often have a lot of work and not much time for you”.

Could it be that low attendance during open office hours is due to barriers to students’ engagement with feedback or, more generally, a lack of feedback literacy?

This is something that I want to explore in more detail. I have already briefly discussed this with our academic writing advisor, and we may want to see whether we can specifically address this issue in a forthcoming curriculum review.

But what about solutions for the here and now? There are many ways in which open office are organised, but what works best?

One colleague suggested changing times. Admittedly, my open office hours are Wednesdays from 08:30-09:30, but this never was a problem – and the feedback open office hours during the aforementioned course were scheduled in the afternoon. Elsewhere in cyberspace people have been suggesting other solutions, including a rethink of faculty office space. I’d love to squeeze in a couch, but my office is rather tiny.

On Twitter someone suggested that the wording ‘open office hours’ is unclear to students and that ‘student drop-in hours’ may make more sense. So, the name plate next to my door now mentions my student drop-in hours and so does the syllabus of an upcoming course.

Let’s see what happens. I hope students will come and talk to me again. The door’s open, simply turn up at the stated time!

Too much, too close

For non-UK readers, you may or may not be aware that we’re having a general election here pretty soon, in what looks like a season 4 finale for “The Brexit Saga” (many, many more seasons still to come).

I mention it here mainly because it’s made my home life come a lot closer to my work life, as I try to marry Political Science to The State of This with loved ones.

Usually when we talk about learning and teaching, we value proximity. Active learning assumes that more visceral experiences are more likely to generate deep understanding of substance and process than are passive, transmission models.

And it’s certainly the case that the immediacy of the election and the salience of the issues has driven a markedly higher level of inquiry from both voting and non-voting members of the household.

For context, our constituency has become a much more contested one of late, including the current MP having lost the party whip (but still standing) and an electoral pact between a couple of the other parties to try and improve their chances. Let’s just say I’ve got a lot of campaign materials through our door of late.

However, for some of the voters in the house, the situation is profoundly unhappy: levels of trust in anyone are low, the head vote isn’t the same as the heart vote, especially as the heart vote option might not even be possible in any case.

Some voters in the house are distinctly hacked off by this, and my efforts to apply the salve of rational choice action under FPTP aren’t going that well (especially when some voters in the house are behavioural scientists).

But enough of my exciting mealtime conversations: what might we learn from this for our teaching?

Well, it reminds me that we have to be careful not to let our students overly-invest in their learning environment. This does happen with big. multi-day simulation exercises, which is why it’s essential to have a comprehensive debrief straight after to allow they to step back out.

But it’s also an issue with emotional/sensitive topics. We need to be very careful about how we set up those discussions and how all participants frame their contributions. We also need to give space for students to give voice to their concerns and preoccupations, be that in the group or individually.

What we can’t – and shouldn’t – do is try to pretend that emotion doesn’t come into it. Politics is emotional at a pretty basic level, so if we cut that out then we lose both only the recognition of individual engagement with it, but also a part of our understanding.

Stepping back and asking people to reflect on their emotions is part of that, but even this isn’t easy. Which is why we also have to show that we acknowledge and empathise.

Sometimes we don’t have good – in the sense of being satisfactory – answers to issues, and that’s fine to admit too, because it’s in the gaps that our own reflection and understanding grows.

Maybe something to discuss over a family meal sometime?

Learning in the army

This guest post comes from Lt Chad Barrigan, Learning Development Officer in the Army’s Educational and Training Services.

I’m an Education Officer in the Army. Like every teacher, we are faced with the eternal question about what our role is. Are we there to be a font of and dispenser of knowledge or are we there to exploit the knowledge of the students in the room? I find myself firmly in the latter camp and despair at the idea of teaching classes based off PowerPoint and research tasks. The problem we face is how do we get the same base level of knowledge across the class to actually be able to do a good exploitation activity?

I think I may have found a way to do this and it is a combination of using learning technology, scenario-based learning, motion graphic design, Black Mirror’s Bandersnatch and Star Trek in an exercise called Clarke’s Crisis (solely named so as to wind up a colleague.)

Continue reading