We’re just about getting to the end of semester’s block of teaching weeks, so my attention is turning to final assessment once again.
With my first-years I’ve inherited a module on the EU that used to be mine some time ago and for which I’ve stuck to the assessment regime through curiosity as much as anything else.
As I’ve discussed elsewhere here, we’re piloting our new computer-based assessment system on the module, so I was keen to see how that changed things. Much of my attention in that regard has been to do with the coursework, but we’re also doing the final exam on it too.
It turns out that this is an excellent opportunity for me to get into open-book exams.
My student memory of these are watching law students carting in a dozen or more lever-arch files (ask your parents) into an exam hall, usually with at least one person have the entire thing spill out across the corridor outside or (on one tremendous occasion) across a busy street and towards a near-by canal.
Happy days. But not very enticing.
But because so much of the work has moved online, not least the exam itself, this seems like a good moment to visit the format.
For those who’ve not encountered it before, an open-book exam is simply one where you can bring and use any materials you like during the exam period. The idea is that it’s much more like a situation you might encounter in real-life than sitting in a bare room, answering questions you’ve hopefully prepared for, but using only what you can haul from the back of your mind.
The reason it’s not been so popular has been a mix of the aforementioned mess, the fear that students will just copy out other peoples’ work and the vague air that it’s not ‘right’.
Of course, I’m a big believer in changing what you when situations change, so why not try an open-book format?
It’s helped by the system being able still to detect plagiarism (final submissions are run through the usual software), plus it can note when a student suddenly dumps several hundred words at once.
Moreover, giving an open-book exam removes any feeling of accommodation to students about factual errors: my lovely mnemonics will be left at one side should I meet anyone who tries to tell me about the Council of Europe in leading the EU.
Of course, an open-book exam – while superficially attractive to students – is a big bear-trap. The temptation to ‘go check something’ will be very high, taking time away from actually writing an answer to the question asked. As those law students used to discover (when we talked to them on our way to the bar), it’s one thing to have access to lots of information, but quite another if you don’t know how to find the right information.
So, we’ll see. My impression so far has been that a lot of my students haven’t really clocked the different issues involved. If nothing else, if they’re relying on my flipped lectures as much as I think they are, then they’ll discover rather quickly that those are in possibly the least-helpful format for an exam.
Last week I had three separate occasions teaching on the same subject, plus also getting filmed talking about that subject too.
I’d been a bit anxious about this concatenation of coverage, even if it was on my research specialisation: talk enough about something and eventually you end up saying all you’ve got to say. Or worse, I might forget who’d I’d said what to whom.
In the end, that was not a problem: I’d been good about defining quite clearly what each session was focused on, so the internal consistency was fine, as was relationship between the different bits.
However, each time I was doing something quite different.
There was one conventional lecture, one set of seminars and one Q&A-type session to support a flipped lecture. Plus that filming.
Now I’m not one to blow my own trumpet, but I gave a good lecture: it flowed, it had coherence, the students were engaged (and occasionally entertained) and I even got a little round of applause at the end.
But it was also the session that I worried gave the least to the audience: they were passive recipients, rather than active learners. In the Q&A and in the seminars, the content was driven by their needs and learning processes: even the filmed lecture is going to become part of a more interactive package.
However, my ability to give lectures seemed to be better than my ability to give the other formats (not that they were bad, to be clear: just not as good as the lecture).
So what to do?
Stick with what I’m best at, or focus on what I understand to be best for my students’ learning?
Of course, I’m setting up a bit of a false choice here: my evaluation of my relative capacities is completely anecdotal, plus I know that the evidence about pedagogic formats isn’t completely nailed down.
However, the point still stands, because our subjective view of such situations shapes how we engage with pedagogy: I think we all know plenty of people who stick with what they know because they’re good at it, or at least better at it than some other approach they’ve not tried before.
The difficulty is separating our anxiety/indifference about new pedagogies from any more objective limitation on our ability to use them. My personal view is that just because I not smashing it in my seminars, it doesn’t mean I shouldn’t be doing them: instead, I should be trying to identify and address my weaknesses.
So last week was our final teaching week before the Easter break. For reasons we don’t need to go into here, it’s not the end of our teaching block for the semester, so it’s a bit of a breather.
As such, my usual expectation is that class attendance takes a bit of a hit.
However, my lecture of 120 students only saw 15 turn up, which – even by my standards – isn’t good. At all.
Obviously those who came didn’t have the answers for their fellow students’ absence, so I decided to gather some data.
Using a quick SurveyMonkey poll, I offered students a range of options to choose from. As you can see, I got about 40% of the class (including a couple who had come) to respond, so reasonable enough to make some observations.
The first is that timing does matter. My lecture is 1000 on a Thursday, and the night before had been a big sports night, so there was certainly a bunch of students incapacitated by that. Also, a couple of students noted that Thursdays are solid with classes for them, abetted by some deadlines for other modules due at the same time.
However, while those factors explain part of it, I was also rather curious about whether our experiment in flipping plays a role.
My module is a first-year introduction to European integration, and I’ve been trying out a flipped format. This involves a pre-recorded lecture online, with the conventional lecture time being given over to Q&A on the recording and elaboration of key themes: there’s also a seminar session, which runs on more conventional lines.
So far, that Q&A part has not been running as well as I would like: only a small number of students ask questions and there’s evidently a block who haven’t watched the lecture beforehand, so can’t ask.
Part of my concern in trying out flipping was that students might see it as a way to disengage with the face-to-face element of the module. My efforts to tackle this including highlighting that the recording doesn’t have enough in itself to give all the material and framing students need to do well in the module: the Q&A always includes stuff that relates much more directly to the final exam (and I say as much).
Clearly, the survey highlights that this isn’t resonating with the students.
Even if we allow for a degree of “what might look like the answer least likely to cause offence”, there’s a big block covering the lack of utility of the (Q&A) lecture element.
Certainly, I can see that if you’ve not watched the video, then the lecture isn’t that useful, but I’m more concerned about those who feel that the recording suffices.
All of which leaves me in a quandry.
Part of me wants to rework the remaining sessions after Easter to be much more explicit in leaving material out of the online stuff, with the lecture picking it up instead.
But another part of me wants to stick with my approach to date and then we see how it goes with the exam.
Right now, I don’t have the answers to this one. I need to explore some more to see if attendance was similarly down in other modules, to better triangulate what’s happening here.
Below is an example of a recent in-class exercise that I used in my comparative politics course, created in collaboration with our crack staff of librarians. The exercise is designed to teach students that the scholarly journal articles they are reading represent an ongoing conversation between experts. I have included the information students were supposed to locate in italics. Feel free to modify the activity to meet your own needs.
Which former president of the American Political Science Association was one of Dr. Raymond’s professors? (Lucian Pye) First person to submit the correct answer before time expires earns 10 points toward their final course grade.
Locate this article: Larry Diamond, “Why Are There No Arab Democracies?” Journal of Democracy 21, 1 (January 2010): 93-112.
In this article, find citations for academic journal articles about democracy in the Arab world. Locate these articles. Write their citations:
Alfred Stepan and Graeme B. Robertson, “An ‘Arab’ More Than a ‘Muslim’ Democracy Gap,” Journal of Democracy 14 (July 2003): 30–44.
Mark Tessler and Eleanor Gao, “Gauging Arab Support for Democracy,” Journal of Democracy 16 (July 2005): 82–97.
Amaney Jamal and Mark Tessler, “The Democracy Barometers: Attitudes in the Arab World,” Journal of Democracy 19 (January 2008): 97–110.
One of the above articles (Jamal and Tessler) contains a citation for an academic journal article written by another former president of the American Political Science Association. Locate this article. Write the citation:
Samuel Huntington, “Will More Countries Become Democratic?” Political Science Quarterly 99, 2 (Summer 1984): 193-218
In the above article, find citations for these two works: the original version of an academic journal article that you have read for this course, and a book that you have read about for this course. What are the citations for the two works?
Seymour Martin Lipset, “Some Social Requisites of Democracy: Economic Development and Political Legitimacy,” American Political Science Review 53 (1959): 75.
Barrington Moore, Jr., Social Origins of Dictatorship and Democracy (Boston: Beacon Press, 1966), 418.
How do the ideas contained in the scholarly works listed above relate to the ideas presented by Diamond in the initial article?
Your Own Research
Choose a research topic of interest (does not have to be related to this course). On the back of this paper, track citations of academic journal articles about the topic through time. What do the authors of these articles say about the topic? How do their ideas correspond or differ?
This guest post is part of a series linked to the publication of G. Pleschova & A. Simon (eds.) Learning to teach in central Europe: Reflections from early career researchers. This post comes from Maxine David.
In her chapter Life after academia: preparing students for successful collaboration, Kovačević talks us through her 2017 experiences teaching a course on EU Enlargement at the University of Economics in Bratislava (EUBA). We are first given a little insight into teaching practices at EUBA and into Slovak Higher Education legislation, before moving on to the detail regarding the problems she felt needed remedying, the method she employed, the rationale for it and expected results. Data collection and results are then discussed, albeit the latter more comprehensively than the former. The chapter ends with Kovačević’s reflections on the application of group presentations and the challenges involved in devising a reliable research design to generate data on applied teaching methods.
academics who have been teaching for a good number of years will recognise
their own early teaching days in the experiences Kovačević describes. At one
level, this is rather depressing; I, for instance, have assessed students
through group presentations for a decade and more. Even at the beginning, I did
not consider it as terribly innovative given it was what I had encountered on
my own Bachelor’s degree back in the 1990s. The chapter therefore raised
questions for me about what might really be called innovative. In turn, that
suggests the real value in Kovačević’s chapter: first, that it adds to our
knowledge of other contexts; second that it highlights the wider failure of
many academics to engage sufficiently in an exchange of pedagogical knowledge
is therefore to be commended for the degree to which she has problematised the
learning process, thinking about impediments to learning and how to overcome
them at this early stage in her career. She grounds her thinking in the
literature explaining the benefits of group work and presentations, especially
in respect of developing transferable skills and enhancing employability. Based
on that literature and her prior experience in teaching the course, she comes
up with three hypotheses. The first of these is somewhat unclear: “The
innovation—i.e. group work—takes place in a classroom environment that is
supportive of learning via collaboration”. Does she mean that she is
hypothesising groupwork is innovative, that groupwork is learning by collaboration,
or something else? The other two hypotheses are clearer: students’ interest in
the subject matter will be increased as a result of the process of creating a
group presentation; and there will be a noticeable increase in subject-specific
knowledge, as well as related skills.
for many teaching in environments that regularly apply such methods, these will
be self-evident: as Kovačević herself acknowledges, the benefits of
student-centred learning are already well-recognised. Clearly, however, whether
as a whole or just in pockets, methods that put the student at the centre of
the learning process are not the norm for EUBA (and many another institutions).
her course, working with seminar groups of around 13 students, Kovačević began by
having students collaborate to produce a poster on Turkey in the EU’s
enlargement process before moving on to the creation of a Powerpoint
presentation. She is keen to point out the support that was offered throughout
the process, including instructor and peer feedback. It is a shame,
incidentally, that we did not hear more about this peer feedback, a notably
tricky area (see, for instance: Liu and Carless).
chapter is weakest on talking us through definitions and in the section on data
collection and methods, though the latter aspects are partly addressed in the
conclusions. On definitions, it is not entirely clear what is meant by
“presentation”. Presumably, it is confined to a Powerpoint presentation but it
could be more (role plays) or less (students acting as rapporteurs) extensively
construed. The question is an important one for those thinking about adaptions
to the method Kovačević applies.
about measurement and comparison are also insufficiently considered. For
example, the third hypothesis (“Student learning, including knowledge and
skills after collaborative group work, is noticeable”) begs questions about how
levels of learning can be measured, and compared to what. If we accept group
presentations as innovative, we must accept also that others need to be
persuaded of the relative benefits of
such innovative teaching, otherwise, why change? As such, the persuasive
potential of the chapter is reduced. Methodologically, it would have been
useful to know how Kovačević recorded and evaluated the “student activity and
behaviour” she observed.
the number of unanswered questions, the chapter is an important one. It
functions as a reminder that there is still much to be done to convince others
of the benefits of changing ways of thinking and doing because innovation is
not contagious. It is a reminder too that rigorous and reliable evidence is
sometimes difficult to generate and that without that, it becomes all the more
difficult to overcome resistance to change. Finally, the chapter is important
because it raises implicitly the question of whose responsibility it is to
bring about change. Should it be contingent on young scholars, under pressure
in so many other ways, to undertake all this work? I think we all know the
answer to that question, what we are doing to address it is another matter.
My first-year module this semester has been a real training ground for me. Not only am I going all-in on flipping, but I’m also trialing the new assessment software that the University is thinking of using.
By extension, that also means it’s a training ground for my students, something that I’ve been very open about with them.
The flipping seems to be working and I’ll be writing up my thoughts on that later in the semester, but having coming through the first use of the software I need to make some decisions now.
In part, my situation arises from wanting to push how we used the software past a conventional approach. Not only did students submit a literature review to it, but they then had to review someone else’s using the system, all in aid of a final piece of self-reflection (which we’re marking now).
Using the marking function is a bit more involved than just submitting work and a couple of people did get a bit lost on that. But the bigger problem was that not everyone submitted work.
In the good old days (i.e. last year and before) we did all this in-class, so it was much simpler to cover (the exceptionally few) missing pieces. However, because we’d pre-selected peer reviewers, we ended up with some students having nothing to review and others not getting their work reviewed.
That’s a failing on my part: next time, I’d leave allocation until after the first submission was in, so everyone who submitted got allocated and reviewed.
But that’s next time. What about now?
Already, I’ve indicated to everyone that not getting peer feedback won’t count against them in marking, but a couple of students have felt that absent such comments they’re not in a position to complete the self-reflection.
To that, I’ve had to underline that it’s self-reflection, so peer feedback was only ever one component of that: indeed, the whole purpose of the somewhat-convoluted exercise is to get students becoming more independent and critical about their learning.
All that said, peer review was added in here to help prompt everyone to think more about what they’ve done and what they could do.
As we sit down to mark, the question will be much we can, and should, take the circumstances into account. Until we’ve seen the full range of work, that’s going to be a tricky call to make.
However, it all highlights an important point in such situations: do we have fall-backs?
Trying new things is inherently risky – that’s why many colleagues stick with what they know – but with some risk management, that need not be a barrier to moving practice forward.
Annoying through our situation here is, it’s not fatally-compromising to the endeavour: we know who’s affected and how; they’re still able to submit work; and the assessment is relatively small in the overall scheme of things.
Yes, we’ll be using the system again for the final exam, but without the aspects that have proved problematic. Indeed, the exam has already been trialled elsewhere in the University, so that’s well-understood.
So, on balance, I feel comfortable that we can manage the situation and implement the necessary changes next time around to remove the problems identified.
Which is, of course, a big part of the reason for trying it out in the first place.
A few years back I wrote about the The Lego Movie and how it captured the operation of fascism in a form that was both accessible and about as enjoyable as fascism ever can be.
Some readers of this blog were unhappy that I’d ruined that film for them, so it’s only right I try to do that for another generation of scholars and young parents.
The Lego Movie 2 isn’t as good as the first one (confine discussion of whether this is ever possible to yourself, away from me), but obviously I watched it, because, well, Lego.
This aside, there’s still plenty of politics going on in the film., which I’ll write about now with some mild spoilers (because you’re not 9 years old and because you’re a serious scholar).
For the IR types among you, there’s a whole bunch of realism going on, with security dilemmas, anarchy and the brutishness that this engenders in actors. The collapse of the (ultimately benign) dictatorship in the first movie produces a literal and metaphorical wasteland in which actions are guarded and security is everything. The final reconciliation of the actors this time occurs when they identify a common external threat and work together to overcome it.
As a musing on power in its various forms, the movie offers a useful way to conceptualise how actors operate under uncertainty and the tension between collaborative and conflictual action, as captured in the notions of being a ‘master builder’ or a ‘master smasher’.
For the gender scholars, there’s a bunch of gendered roles, paternalism (and maternalism), as well as how children ‘become’ adults going on in all this too. It’s also a classic of the ‘absent father’ trope and all that implies.
But for my purposes, the film is all about constructivism.
The logic of appropriateness runs through the entire piece, as individuals strive and struggle to either fit into their environment or communicate their intentions.
If Emmet’s arc is one of finding a persona and an attitude that works from him (puberty alert!), then Bianca’s is one of learning to signal intentions less ambiguously.
As someone who teaches negotiation, I recognised a lot of these tensions from my classroom, where students are apt to adopt personae as ‘negotiators’ that don’t always sit comfortably with their more general sense of being.
In both classroom and the film, individuals try out different gambits, with varying degrees of sincerity and of success. But ultimately, as the film suggests, it is when there is a more open exchange of views that progress is made, clearing up the confusions and misunderstandings and realigning how we view other’s actions.
Indeed, the whole film turns on how individuals perceive one another and themselves: Finn misunderstands Bianca; Emmet is misled by Rex; Lucy struggles (as in the first film) with her sense of identity; Batman has to learn about living with light as well as dark; and Superman has to find accommodation with the Green Lantern.
Identity here is thus not purely about being true to yourself, but also about being true to others. Whether you accept that being doing both you end up with a happy society – as the movie argues (as movies are wont to do) – is another matter.
But however you take it, this case highlights how we can use cultural products to illustrate and illuminate our teaching: the beauty of politics is that it is pervasive, so we can find it pretty much anywhere we look, if we choose to see.