Opening the book on exams

We’re just about getting to the end of semester’s block of teaching weeks, so my attention is turning to final assessment once again.

Let’s take it back, let’s take it back, let’s take it back to the Law School…

With my first-years I’ve inherited a module on the EU that used to be mine some time ago and for which I’ve stuck to the assessment regime through curiosity as much as anything else.

As I’ve discussed elsewhere here, we’re piloting our new computer-based assessment system on the module, so I was keen to see how that changed things. Much of my attention in that regard has been to do with the coursework, but we’re also doing the final exam on it too.

It turns out that this is an excellent opportunity for me to get into open-book exams.

My student memory of these are watching law students carting in a dozen or more lever-arch files (ask your parents) into an exam hall, usually with at least one person have the entire thing spill out across the corridor outside or (on one tremendous occasion) across a busy street and towards a near-by canal.

Happy days. But not very enticing.

But because so much of the work has moved online, not least the exam itself, this seems like a good moment to visit the format.

For those who’ve not encountered it before, an open-book exam is simply one where you can bring and use any materials you like during the exam period. The idea is that it’s much more like a situation you might encounter in real-life than sitting in a bare room, answering questions you’ve hopefully prepared for, but using only what you can haul from the back of your mind.

The reason it’s not been so popular has been a mix of the aforementioned mess, the fear that students will just copy out other peoples’ work and the vague air that it’s not ‘right’.

Of course, I’m a big believer in changing what you when situations change, so why not try an open-book format?

It’s helped by the system being able still to detect plagiarism (final submissions are run through the usual software), plus it can note when a student suddenly dumps several hundred words at once.

Moreover, giving an open-book exam removes any feeling of accommodation to students about factual errors: my lovely mnemonics will be left at one side should I meet anyone who tries to tell me about the Council of Europe in leading the EU.

Of course, an open-book exam – while superficially attractive to students – is a big bear-trap. The temptation to ‘go check something’ will be very high, taking time away from actually writing an answer to the question asked. As those law students used to discover (when we talked to them on our way to the bar), it’s one thing to have access to lots of information, but quite another if you don’t know how to find the right information.

So, we’ll see. My impression so far has been that a lot of my students haven’t really clocked the different issues involved. If nothing else, if they’re relying on my flipped lectures as much as I think they are, then they’ll discover rather quickly that those are in possibly the least-helpful format for an exam.

Let’s hope those lecture notes are in good order.

Innovating in Large Groups: Pair work as a strategy for student engagement

This guest post is part of a series linked to the publication of G. Pleschova & A. Simon (eds.) Learning to teach in central Europe: Reflections from early career researchers. This post comes from Theresa Reidy.

Shpend Voca provides an excellent resource for academics working with large student groups in this chapter on pair work. Building interaction into sessions with students can become a challenge when group numbers exceed the 40-50 mark but this chapter reminds us that innovation and engagement should not be traded when faced with larger student groups.

The chapter is important in four respects.

First, Voca strikes a chord with the opening discussion of the traditional lecture as a passive format and in referring to Bloom’s Taxonomy, highlights the low level learning outcomes which students experience and achieve through this approach. In this sense, the challenge that lecturers face with big groups is clearly set out in the opening section of the chapter.

Second, several examples of discrete engagement strategies which can be used with large groups are elaborated. This is perhaps the most important part of the chapter in that it provides specific options for teaching staff to use in their own practice. The options given include pausing during lectures to allow students to speak with the person sitting next to them to discuss the material provided up to that point, and clarify notes. This is a simple idea and one that could be especially valuable for early year undergraduate students. The author includes further options, which build on this starting suggestion.

The third noteworthy aspect of the chapter relates directly to the robust research design used to evaluate the use of pair work. A clear approach is taken with data collected through several routes. Student evaluations, final grades and a peer observation rubric are used in what is essentially a natural experiment, as there is also a control group taking the same module but without the pair work strategy. This multi-pronged strategy allows for a rigorous interrogation of the pair work and a persuasive presentation of the outcomes.

Which brings me to the final point, the chapter provides conclusive evidence that pair work in large groups enhances the overall learning experience of students. The student evaluations were positive, the learning dividend notable in the comparison with the control group who did not have pair work incorporated into their classes. Indeed, the author downplays the outcomes in the final discussion, they could have been more positive about the pair work given the solid evidence supporting its impact.

Introducing pair work was a small change deployed in a challenging classroom environment but one that delivered clear and demonstrable outcomes. That is the take away message from the chapter and hopefully will one that will be absorbed by colleagues seeking to make their own practice more engaging and effective for students.

This is an important and useful contribution to the scholarship of teaching and learning in the sciences. It is well written, in an accessible style and the structure lends itself to those who are interested in pedagogy and also to those seeking practical advice and guidance. It should be read widely.

What’s good for me and what’s good for my students?

Evidently, my existential rut continues.

Last week I had three separate occasions teaching on the same subject, plus also getting filmed talking about that subject too.

I’d been a bit anxious about this concatenation of coverage, even if it was on my research specialisation: talk enough about something and eventually you end up saying all you’ve got to say. Or worse, I might forget who’d I’d said what to whom.

In the end, that was not a problem: I’d been good about defining quite clearly what each session was focused on, so the internal consistency was fine, as was relationship between the different bits.

However, each time I was doing something quite different.

There was one conventional lecture, one set of seminars and one Q&A-type session to support a flipped lecture. Plus that filming.

Now I’m not one to blow my own trumpet, but I gave a good lecture: it flowed, it had coherence, the students were engaged (and occasionally entertained) and I even got a little round of applause at the end.

Lovely.

But it was also the session that I worried gave the least to the audience: they were passive recipients, rather than active learners. In the Q&A and in the seminars, the content was driven by their needs and learning processes: even the filmed lecture is going to become part of a more interactive package.

However, my ability to give lectures seemed to be better than my ability to give the other formats (not that they were bad, to be clear: just not as good as the lecture).

So what to do?

Stick with what I’m best at, or focus on what I understand to be best for my students’ learning?

Of course, I’m setting up a bit of a false choice here: my evaluation of my relative capacities is completely anecdotal, plus I know that the evidence about pedagogic formats isn’t completely nailed down.

However, the point still stands, because our subjective view of such situations shapes how we engage with pedagogy: I think we all know plenty of people who stick with what they know because they’re good at it, or at least better at it than some other approach they’ve not tried before.

The difficulty is separating our anxiety/indifference about new pedagogies from any more objective limitation on our ability to use them. My personal view is that just because I not smashing it in my seminars, it doesn’t mean I shouldn’t be doing them: instead, I should be trying to identify and address my weaknesses.

What am I good at?

I’ve just been reading I Am Dynamite!, Sue Prideaux’s very engaging life of Nietzsche, mainly because I knew nothing about the man or his work.

As well as being a good read, and to appreciate much better the intentional ambiguities of Nietzsche’s thoughts, it was also very striking to see a story of someone finding their place in the world.

Also an excellent improviser on the piano…

From those of you unfamiliar with the details, Nietzsche was a brilliant philologist, in an age when philology was the very highest discipline in German academia.

After only three semesters as an undergraduate at university, Basle wrote to him in Leipzig to offer him a chair in the subject, on the recommendation of his tutor: impressive for a 24 year old.

But for me it was even more impressive that relatively quickly Nietzsche decided that he didn’t want to be a philologist, but a philosopher. Having tried to make that switch within the university system, and failed, he struck out on his own.

Now you may quibble that he only made his big breakthroughs just at the point he lost his mind and that these things might be connected, but the point still stands: life, for Nietzsche, was about overcoming yourself, loving your fate while simultaneously making the most of your situation.

I see this as a somewhat more positive message than my usual go-tos in philosophy, the Existentialists, who make the very good point that life is meaningless, but who seem to be a bit too “I’m having such fun” to ring quite true.

But what does this have to do with you and me?

I think it’s partly a function of the time of year: teaching’s coming to an end, we’re all exhausted, not least because of the pile of research activity waving to us from the other end of the desk. Talking with various colleagues, it’s the time of “why do I do this? should I be doing this?”

That’s a difficult conversation to have with yourself and with others: indeed, I’m regularly struck by people who I consider to be very able, competent and enthusiastic about their work who turn out to be not so sure it’s what they want to be doing. Success does not necessarily mean satisfaction, as young Nietzsche found out.

Even if we don’t have the idea to tear down the foundations of Western civilisation and thought, it is still good to recognise and engage with these feelings inside us,

I know that I derive great enjoyment from my work, in part because it is so varied and irregular, but I also know that this is a source of anxiety: what will come next? How will I cope? Where does it lead?

Thinking back to my previous post, it’s really essential to have a good support network, people you can talk with, frankly and openly. That can be friends, family, colleagues, it’s not really important who, as long as you recognise that you don’t have to tread your path alone.

As Nietzsche noted, it’s all too easy to fall back into comforting patterns of behaviour and not to ask yourself difficult questions, but ultimately it’s only by doing that you progress, a sentiment that shouldn’t be so problematic for academics. You don’t have to buy into the rest of it, but acknowledging your situation and what you want from it all isn’t the worst thing to do.

And if all that’s a bit heavy, wait until you find out what I’m reading next.

On motivation: you had my interest, but now you have my attention

This guest post is part of a series linked to the publication of G. Pleschova & A. Simon (eds.) Learning to teach in central Europe: Reflections from early career researchers. This post comes from Silviu Piros

Barbora’s chapter provokes from the onset: ‘It Takes Two to Tango: How to get IR students engaged in their learning’. First, it provokes the reader to shift attention from a teacher-centred to a student-centred approach to delivering education. Second, it challenges the reader to consider innovation not for the sake of just ‘doing things differently’, but to solve an ever-growing real-life problem – in this case – increasingly lower attention spans and rapidly waning engagement levels. Finally, it encourages the reader to take own charge by equipping him both with a practical DIY-kit of how the design of such a course could look like, as well as critically observing and empirically evaluating its results.

The academic community is now giving a more attention to student motivation and engagement in the classroom. The instant and constant access to a ubiquitous mass of information has in turn diminished attention spans and the ‘digital natives’ or ‘Gen Z’ – who are now starting their undergraduate studies – are being the most affected group.

Continuously focusing for 90 minutes during a classic lecture, as well as thorough note-taking are becoming serious challenges undergraduate students are often struggling with.

On this backdrop, Barbora reveals her tri-fold challenge that required an innovative approach: to teach an optional course on a perceivably ‘exotic’ topic (Arctic Geopolitics), with an only pass/fail grade, to a mixed cohort of bachelor students with variable affinities and interest in the subject. Her approach aimed at systematically tackling all these perceived weaknesses through a strategy that would gradually shift students’ extrinsic motivation in the course (i.e. the engagement delivered through topic and classroom activities) with genuine intrinsic motivation (that is driven from within).

Thus, she designed and developed her course based on three pillars: expert skype lectures, in-class group work, and interactive mini-lectures. With this mix of methods, the danger of losing students’ interest and engagement in a 90-minute lecture was skirted, as each of the three components brought a different angle while keeping everybody constantly involved: be it through practical Q&As in the expert sessions, interaction with the course convenor in the mini-lectures, or peer interaction during group work.

The effort associated with building such a curriculum is praiseworthy. The risks associated with it, make is so even more.

The technology-enabled expert sessions are a great tool for creating momentum on a theme, and exposing students to real-life scenarios, however the failure of technology on the spot, or the sudden unavailability of the expert will have a serious  impact on the overall outcome of the course and will impact negatively motivation levels, and therefore it needs to be factored in.

Similarly – and this is something the chapter considers – the importance of design in the group activity and the pitfalls of failing to keep students engaged during the group sessions should not be overlooked.

What makes this chapter a great and commendable reading is indeed its openness. Openness to challenge, to change, to reconsider. The key message Barbora conveys throughout this chapter is straightforward: the ever-changing needs of our students drive teaching and learning innovation, and with the right amount of commitment and determination, each new provocation will make for a great opportunity to move things forward.

Advice to my younger self

I’ve finally to attending a networking event for Early Career Researchers at the conference I’m at, having decided that I’m far enough down the line to have something useful to say.

(Why yes, I have recently had a birthday. Why do you ask?)

The idea is a pretty standard one: speed-dating round the room to make some contacts and share some ideas. It’s a really good format for semi-structured getting-to-know-yous.

Of course, me being me, I’ve spent a couple of days talking through ideas with my partner, who’s also an academic: what is the most useful advice you could give someone if you only had a couple of minutes?

Since I don’t see why I should limit my unburdening to the ECRs at the conference, here’s my shot.

Continue reading

What makes good teaching?

But… but… he’s… he’s teaching himself. That’s not even on the NSS

Think about the teaching you have received over your life; especially the teaching that made a difference.

You might remember a teacher or lecturer who was incredibly enthused by their subject, who shared that passion to you through your time together, dazzling and enticing you to learn more.

Or maybe you remember someone who presented their material very clearly and systematic, who flagged the key points and was an excellent communicator.

Or was there someone who was a really good laugh, who made you want to be in class for the LOLZ?

I’ve had all of those.

But I’m not sure that those people where necessarily the ones that I learnt the most from.

Continue reading

Ways to make a university course more attractive and to improve student learning

This guest post is part of a series linked to the publication of G. Pleschova & A. Simon (eds.) Learning to teach in central Europe: Reflections from early career researchers. This post comes from Ulrich Hamenstädt

In her chapter, Ludmila Kašpárková discusses several methods to make a relatively unpopular university course more attractive for the students. The chapter presents first the challenge of teaching a course for which the students’ motivation is low. Kašpárková then outlines three changes she made to improve students’ motivation and learning outcomes. Finally, she presents the results of her evaluation on how the new teaching style has helped to improve the course and student learning.

For me, this chapter is a good example on how to deal with a challenge that a university lecturer faces from time to time: taking over a course that is part of the compulsory curriculum, but is unpopular among the students for different reasons. While the content of this course was relatively fixed, the approach to teaching offered many opportunities for improvement. In this sense, this chapter presents three ways to improve learning that can be easily applied to various university courses.

The first approach Kašpárková used was to implement John Bigg’s concept of constructive alignment in the didactic design of the course. To ensure that the students understand the syllabus, the learning objectives and develop ownership of learning she discussed with the students their possible takeaways of the course. This may seem obvious, but since it takes time, it is often neglected or shortened out in practice. The second change of the course was the introduction of new learning activities such as role plays and peer discussions to replace passive learning from lecturing. This was a key element of this innovation, since learning in social sciences is not about imparting knowledge but it implies – above all – enabling the students to develop their skills. Thirdly, the examination at the end of the semester was replaced by continuous assessment to enhance the quality of student learning.

Overall, it is interesting to read how all these changes together influenced students’ learning outcomes. Ludmila Kašpárková’s work presents a valuable approach to transforming and improving a course for students. The series of changes reported in the chapter can also be applied individually to course sessions or teaching situations. 

Getting flipped

So last week was our final teaching week before the Easter break. For reasons we don’t need to go into here, it’s not the end of our teaching block for the semester, so it’s a bit of a breather.

As such, my usual expectation is that class attendance takes a bit of a hit.

However, my lecture of 120 students only saw 15 turn up, which – even by my standards – isn’t good. At all.

Obviously those who came didn’t have the answers for their fellow students’ absence, so I decided to gather some data.

Using a quick SurveyMonkey poll, I offered students a range of options to choose from. As you can see, I got about 40% of the class (including a couple who had come) to respond, so reasonable enough to make some observations.

The first is that timing does matter. My lecture is 1000 on a Thursday, and the night before had been a big sports night, so there was certainly a bunch of students incapacitated by that. Also, a couple of students noted that Thursdays are solid with classes for them, abetted by some deadlines for other modules due at the same time.

However, while those factors explain part of it, I was also rather curious about whether our experiment in flipping plays a role.

My module is a first-year introduction to European integration, and I’ve been trying out a flipped format. This involves a pre-recorded lecture online, with the conventional lecture time being given over to Q&A on the recording and elaboration of key themes: there’s also a seminar session, which runs on more conventional lines.

So far, that Q&A part has not been running as well as I would like: only a small number of students ask questions and there’s evidently a block who haven’t watched the lecture beforehand, so can’t ask.

Part of my concern in trying out flipping was that students might see it as a way to disengage with the face-to-face element of the module. My efforts to tackle this including highlighting that the recording doesn’t have enough in itself to give all the material and framing students need to do well in the module: the Q&A always includes stuff that relates much more directly to the final exam (and I say as much).

Clearly, the survey highlights that this isn’t resonating with the students.

Even if we allow for a degree of “what might look like the answer least likely to cause offence”, there’s a big block covering the lack of utility of the (Q&A) lecture element.

Certainly, I can see that if you’ve not watched the video, then the lecture isn’t that useful, but I’m more concerned about those who feel that the recording suffices.

All of which leaves me in a quandry.

Part of me wants to rework the remaining sessions after Easter to be much more explicit in leaving material out of the online stuff, with the lecture picking it up instead.

But another part of me wants to stick with my approach to date and then we see how it goes with the exam.

Right now, I don’t have the answers to this one. I need to explore some more to see if attendance was similarly down in other modules, to better triangulate what’s happening here.

Your thoughts are welcome.

It’s not me, it’s you: framing grade-inflation

I’m still enough of a kid to be excited to see the place I work at mentioned in the news, especially if it’s in an outlet my mum might see.

Of course, it’d be better if the context of this particular mention were different, but I guess you can’t have it all.

This all comes off the back of the on-going debate in government about grade inflation.

I wrote about all this last summer, and I’m not sure I’ve gotten much further in my thinking about this, except to note the shift in framing to combating ‘artificial’ grade inflation.

While this might seem to start to take account of the other factors at play, what it singularly doesn’t do is set out a means of calculating this in practice.

Obviously, there are changes in student characteristics that have a direct bearing and these are relatively simple to capture: socio-economic status; entry grades; progressive performance in each year of study.

However, there are also obviously changes in the teaching environment: staffing changes; changes in pedagogic approach; changing curricula (we’ve made our final year dissertation optional thus year, for example); changing provision of learning resources outside the degree programme, at the library or in welfare; changes in programme regulations.

Continue reading