Going digital

“This should improve our module evaluations by 0.4…”

Source

The big question of how we forward (not back) in our teaching practice is one that continues to bother me, partly because it’s going to be a major personal challenge for me in the coming years, but also because the variety of discourses about this vary rather more than I’d expect.

As a case in point, I noticed that my VC/President wrote a long blog about this question just the other week. In it, he writes about the possibilities that digital technologies open up and how we need to be receptive and pro-active in making the most of these.

And that’s all fine.

However, what strikes me about the piece is that there’s no mention at any point about pedagogy. Instead it posits a system driven by what the tech can do.

Having gotten to spend some time with him, I know that he does have a genuine and deep interest in teaching in itself, rather than simply as a side-show from research or a money-generating activity (unlike some VCs I’ve encountered), but it’s a bit disappointing to see a senior leader get caught up in the tech.

Tech matters. This past 12 months have demonstrated that all too clearly, but tech is (and can only ever be) a function of pedagogy. More precisely, the fundamentals of good pedagogic practice – clear learning objectives; alignment of content and assessment; responsive design – are just that, fundamentals.

Consider last spring, when you were scrambling around for a means to continue your classes. You probably had an institutional VLE or platform intended specifically for that purpose, plus access to some other tools, either supported by your institution or not.

In the first instance, I’m guessing you took the path of least resistance in setting up ad hoc ways to get content to students and/or having interactions with them.

But then you started to look around at the world of possibilities, just like my VC is suggesting. But in making your new choices, the key driver was likely to have been “what works best for my and my students’ needs” than it was “what amazing thing is possible here”.

In twenty-something years of teaching, I’ve gone from acetates to Zoom, blackboards to Google Docs. But I can think of very few technologies that have fundamentally changed how I teach and only one that changes what I’m trying to achieve with my teaching.

The one change in objectives was the arrival of the digital world and the cornucopia of data that made available. The result was a need to shift from prioritising the acquisition of techniques to find data to stressing ways of managing all-too-much data. And even then, I still find myself telling students how to track down hard-to-find sources.

But otherwise, the bulk of my learning objectives are the same: building substantive knowledge of a topic; acquiring and using skills that make the student into a critical learner; situating all of this within a wider body of understanding.

In short, tech is a means, not an end.

Again, I’ve tried lots of different technological options: some have been great, others alright, a few rubbish. But I could only judge that against the yardstick of my pedagogy and the learning of my students. Great that they can make a whizzy Prezi, but does it actually help them to learn? And I say that as someone who’d love experience more engaging presentations.

So, in the time-honoured cliché of science-fiction, we have to stop wondering what what we can do and start thinking about whether we should do it.

If not, then we risk falling into another cycle of expensive tech acquisition that doesn’t work for our needs, just like we did most of the other times our institutions bought some tech.

Forward, Not Back

A brief response* to Simon’s last post about not slipping back into old habits:

The Chronicle of Higher Education recently ran this article (paywalled) about how the pandemic might have permanently altered campuses. People interviewed for the article believed that the use of a hybrid delivery format that includes online, asynchronous components will persist, because even full-time, traditionally-aged students like the convenience and flexibility.

Another reason to continue the practice: hybrid design enables students to get out of a passive environment and into more active experiences. For example, last fall my previously 100% face-to-face course on economic development went on Zoom. I do not teach by lecturing in fifty-minute increments three times per week; students in my classes experience a lot of interaction with peers. However, many of these activities can’t be easily replicated in an online environment given the usual tools I have at my disposal. So as a substitute I created a series of assignments in which students documented evidence of economic inequality in the local community with photos and presented their findings in online asynchronous discussions. The assignments met my intended objectives and the students were really engaged, so I’m going to use them again in Fall 2021, when (most likely) the course will once again be delivered face-to-face on campus. But the assignments can’t be completed effectively in fifty-minute time blocks. My proposed solution? Just cut the students loose — not hold class on certain days. My hope is that the institutional hype about maximizing student learning matches reality, and my plan isn’t quashed by higher ups.

If you’re interested in modifying any of your course in a similar fashion, The Aga Khan University has produced an excellent step-by-step guide on the development, design, and implementation of online courses. The guide also applies to hybrid courses.

*The title of this post is also a very obscure reference to the phrase uttered by Muammar Qaddafi during the now decade-old Arab Spring — إلى الأمام — immortalized in Zenga Zenga.

How to not slip back

A new cycle…

I’ve touched before on the opportunity that Covid has presented to embed new, improved practice in our teaching, but I didn’t really get into practical ways to do that.

So let’s have a crack now, yes?

For this, I think we have to side-step the question of whether your institution will give you a free hand in this. The range of options from ‘you’ll do as we say’ to ‘go wild’ is huge, even if no one is being told to ‘go wild’ by anyone. Fortunately, that shouldn’t matter too much.

The first step is to review and evaluate what you’ve been doing during lockdown. That means taking as impartial-as-possible a look at each of the elements of your practice and the overall package.

The whole matters as much as the parts because you often get synergistic effects at work. In a good scenario, your forum-based chat might have been a raging success (in stark contrast to pretty much everyone else), but that might have been as much because you made a lot of it in your direct work with students as because you structured it well. If you’re going to be doing some rearrangement of elements, then it’s really important to have a sense of what relates to what in all this.

This step needs input from multiple directions: your own reflection; that of colleagues; and that of students. If you’ve been relying on particular services, it’s also good to find out if the same level of support will exist in future (given everything, hopefully that’s less of an issue, but new-generation EdTech might be arriving that changes things for you). If it helps, treat it like a piece of research and triangulate as much as possible, and apply some thought to how much you weight each piece of evidence.

At the end of this step you should have a clearer idea of what works, how it works and in what context it works. Remember that those are three different aspects.

Step two is building a repository of elements and ideas for whatever new environment you will be facing.

This doesn’t have to be a literal collection of materials, although such things are useful. Instead, it’s a more conceptual process, of thinking, discussing and optimising new collections of pedagogic elements.

Broadly, you’ve got three choices here. The first is that you’re driven by institutional constraints: if you’ve got a major requirement coming out of management then there’s not much point in resisting (unless it’s very silly and not fully decided upon, in which case you should push back hard (maybe even using some of that evidence you’ve been gathering)).

Alternatively, you can be driven by the nature of the subject matter: some topics lend themselves better to some pedagogies than others, so check out what others are doing.

Finally, you can be driven by a desire to try out (or optimise) some specific pedagogic element. I’ve done this with flipping, because I really wanted to see how I could make it work. Lucky break there, given that we all had to flip last year.

However, in all three choices, you’re still trying to think about building a learning environment that works as well as possible for your students. So ultimately, while your starting point might vary, you should still be coming to a rounded and balanced provisional solution. And again, if you’re resisting institutional pressure, then having a credible and thought-through plan is only going to help.

The third step is implementation. As you close in on running your new class, you need to keep up the evaluating and reviewing as the realities start to hit home. In practice, this means being ready to adapt and regroup as you go.

Hopefully, that’s easier because you’ve got your evidence, your repository and your reflections to help guide you. Importantly, it’ll help in making a call on whether something is just a passing problem or something more structural (which is, admittedly, not always particularly clear).

Of course, all of this is stuff that I’d argue you should be doing in any year, because it’s a key part of maintaining and developing our pedagogic practice. The difference now is that you have probably been exposed to a lot more pedagogic practices than one year ago: so try to internalise these and see them as part of what you can do.

Some things didn’t work, but really very few (because you picked up on problems as you went). Some things worked as stop-gaps, because things were very hectic and pressured (but you’ll have picked up on that from your students, so you’ll not try to repeat the exercise at ‘leisure’). And some things worked in ways that you didn’t really anticipate.

All of that is valuable, whether or not there’s a global pandemic. So make something better out of it all.

Hello again

“Are you ready for our next panellist, people!?”

At the risk of being too Eeyore-ish about it all, the announcements this week about aiming to end all Covid restrictions in England by the middle of June feels ambitious. Experience so far in this pandemic should at least make us cautious about any kind of planning for a ‘return to normality’.

So we have instead to plan for a lack of normality and even – and perhaps more pertinently – the desirability of a new normality, where we can take some constructive points out of our rushed shift to lockdown living/working for the future.

I’ll come back to the classroom aspects of this another time, but today I’ve been thinking about conferences.

Much like last year, we’re seeing all the key events announce that they are going fully online, at least through to the September round. But it’ll invite us to consider what might happen thereafter.

Narrowly, we have a situation where it might be a couple of years before global vaccine uptake (not to mention travel restrictions) is enough that colleagues from across the world can meet up without significant limitations, so we have a considerable period to cover in any case.

But the benefits of online events are also becoming clearer. The ability to bring together people who might not be able to meet in person; the removal of significant costs in time and money for travel and accommodation; the capacity to create more lasting records of our discussions. The coffee being almost always better.

All these things matter and shouldn’t be lost in any rush to get back to the good old days.

And what was good about them in any case?

Yes, I miss getting to see colleagues in person, and to discovering a bit of a new city, but it’s also easy to end up idealising what was often a less-than-perfect experience. You have your bad conference story and I have mine, so we can spare the blushes of those involved, other than to say that an escape to a heavy metal bar shouldn’t be the highpoint of an international conference.

Especially if you don’t really like heavy metal.

Any way, back to the main thought, namely the need to use this moment to consider how we can bring together the best bits of all these things into something new and improved.

Various colleagues do now tell me about different ways of running online events that work much better than the original stick-it-on-Zoom approaches, with more thought about scheduling and technology to make the most of these things. But it’s still the lack of in-person interaction that chafes.

And that’s a major problem, especially for those newer to the profession. I’m lucky/old; two and a half decades of conferences and workshops has left me with a big network of colleagues who can chat away with online, drawing on that prior interaction. For someone who hasn’t had that, their way into creating and sustaining a community is much tighter.

Put it this way, even though our work has been almost entirely conducted online for a decade, the ALPS Blog only happened because we got to spend four nights in Albuquerque. (Good conference BTW).

Maybe we have to find other ways to allow colleagues to get out of their institutions and make connections that don’t require them to travel hundreds of miles and to spend piles of money.

The half-thought is that we could something more intermediate: local gatherings.

Most of work/live near another HE institution, so why not have periodic meetings for people in our area? Maybe to talk work, but mainly just to make connections and put faces to names. You could connect it to big conferences, so there’s a reason and a focus to talk, maybe even chuck in a speaker/roundtable, but mainly it’d be the coffee break/sampling of local delicacies bit of a conference, the stuff that you remember.

It’s not a problem-free idea: some won’t be close to anyone; others will find it hard to justify to bosses/partners that a social is ‘work’; you’re not going to meet that person from another continent who’d be just right for your new project. But it’s a start and something might come from it.

Changing a Comparative Politics Course, Part 4

My previous post explained how students will complete the template that identifies possible causes of either increased democracy or increased authoritarianism in two nation-states from 2000 to 2020. The next step in this project is for students to work in teams to produce qualitative comparative analyses. Here are my instructions for this collaborative assignment:

Continue reading “Changing a Comparative Politics Course, Part 4”

Comparing the performance of on-campus and online students in an accidental experiment

This guest post comes from Patrick Bijsmans (Maastricht University) & Arjan H. Schakel (University of Bergen)

Students and staff are experiencing challenging times, but, as Winston Churchill famously said, “never let a good crisis go to waste”. Patrick recently led a new undergraduate course on academic research at Maastricht University (read more about the course here). Due to COVID-19 students could choose whether they preferred online or on-campus teaching, which resulted in 10 online groups and 11 on-campus groups. We were presented with an opportunity to compare the performance of students who took the very same course, but did so either on-campus or online. Our key lesson: particularly focus on online students and their learning.

In exploring  this topic, we build on our previous research on the importance of attendance in problem-based learning, which suggests that students’ attendance may have an effect on students’ achievements independent from students’ characteristics (i.e. teaching and teachers matter, something that has also been suggested by other scholars). We created an anonymised dataset consisting of students’ attendance, the number of intermediate small research and writing tasks that they had handed in, students’ membership of an on-campus or online group, and, of course, their final course grade. The latter consisted of a short research proposal graded Fail, Pass or Excellent.

316 international students took the course, of which 169 (53%) took the course online and 147 (47%) on-campus. 255 submitted a research proposal, of which 75% passed. One of the reasons why students did so well – normal passing rates are about 65% – might be that, given that this was a new course, the example final exam that they were given was one written by the course coordinator. Bolkan and Goodboy suggest that students tend to copy examples, so providing them may therefore not necessarily be a good thing. Yet students had also done well in previous courses, with the cohort seemingly being very motivated to do well despite the circumstances.

But on closer look it’s very telling that 31% of the online students (52 out of 169) did not receive a grade, i.e. they did not submit a research proposal. This was 9.5% for the on-campus students (14 out of 147)[1]. Perhaps this is the result of self-selection, with motivated students having opted for on-campus teaching. Anyhow, it is clear that online teaching impacts on study progress and enhancing participation in examination among online students needs to be prioritised by programme directors and course leaders.

We focus on students that at least attended one meeting (maximum 6) and handed-in at least one assignment (maximum of 7). Out of these 239 students, 109 were online students (46%) and 130 on-campus (54%). Interestingly, on average these 239 students behaved quite similarly across the online and on-campus groups, they attended on average 5 meetings (online: 4.9; on-campus: 5.3) and they handed-in an average of 5 to 6 tasks (online: 5.0; on-campus: 5.9).

We ran a logit model with a simply dummy variable as the dependent variable which taps whether a student passed for the course. As independent variables we included the total number of attended meetings and the total number of tasks that were handed-in. Both variables were interacted with a dummy variable that tracked whether students follow online or offline teaching and we clustered standard errors by 21 tutor groups.

Unfortunately, we could not include control variables such age, gender, nationality and country of pre-education. This would have helped to rule out alternative explanations and to get more insight into what factors drive differences in performance between online and offline students. For example, international students may have been more likely to opt for online teaching and may have been confronted with time-zone differences, language issues, or other problems.

Figure 1: Impact of attending class on the probability to pass

Figure 1 displays the impact of attending class on the probability to pass for the final research proposal. The predicted probabilities are calculated for an average student that handed-in 5 tasks. Our first main finding is that attendance did not matter for online students, but it did for on-campus students. The differences in predicted probabilities for attending 3, 4, 5, or 6 meetings are not statistically significant (at the 95% confidence level) for online students but they are for on-campus students. Students who attended the maximum of six on-campus meetings had a 68% higher probability to pass compared to a student who attended 3 meetings (89% versus 21%) and a 52% higher probability to pass compared to a student who attended 4 meetings (89% versus 37%).

Figure 2. Impact of handing-in tasks on the probability to pass

Figure 2 displays the impact of handing-in tasks on the probability to pass for the final research proposal. The predicted probabilities are calculated for an average student that attended 5 online or on-campus meetings. Our second main finding is that handing-in tasks did not matter for on-campus students, but it did for online students. The differences in predicted probabilities for handing-in 4, 5, 6, or 7 tasks are not statistically significant (at the 95% confidence level) for on-campus students but they are for online students. Students who handed-in the maximum of seven tasks had a 51% higher probability to pass compared to a student who handed in four tasks (69% versus 18%) and a 16% higher probability to pass compared to a student who handed-in five tasks (69% versus 53%).

Note that we do not think that attendance does not matter for online students or that handing-in tasks does not matter for offline students. Our dataset does not include a sufficient number of students to expose these impacts. From our previous research we know that in general we can isolate the impact of various aspects of course design with data from three cohorts (around 900 students). The very fact that we find remarkably clear-cut impacts of attendance among on-campus students and of handing-in tasks for online students for a relatively small number of students (less than 240) reveals that these impacts are so strong that they surface and become statistically significant in such a small dataset as ours.

This is why we feel confident to advise programme directors and course leaders to focus on online students. As Alexandra Mihai also recently wrote, it is worth investing time and energy in enhancing online students participation in final examinations and to offer them many different small assignments to be handed-in during the whole time span of the course. This is not to say that no attention should be given to on-campus students and their participation in meetings but, given limited resources and the amount of gain to be achieved among online students, we think it would be wise to first focus on online students.


[1] The difference of 21% in no grades between online and offline students is statistically significant at the 99%-level (t = 4.78, p < 0.000, N = 314 students).

Working the assets around you

As well as being the home of queuing, the UK is also a big fan of politics. Indeed, recent production rates of politics have been at all time highs, as you might have noticed.

One of the more niche bits of politics of the last week has been the recording of an online meeting of a parish council (i.e. the very lowest tier of local government). Actually, of a sub-committee of the council.

In what has been a shock to anyone who has failed to anything to do with local politics, this particular meeting was filled with more, um, drama than most.

I’ll leave you to watch at your leisure, and to check out the endless memes and gifs it has spawned, but it’s a good example of how we can connect Fun Things Happening On The Internet to our work as educators.

For some, the meeting was an opportunity to discuss the application of rules on council meetings, in a legal context.

For me, it was a great moment to share with my negotiating students, to consider multiple aspects of the theory and practice that we had been covering through last semester.

To take a non-exhaustive list, we might watch and consider: the issue of gender and engendering in negotiations; the practice of online negotiating spaces; the role of technologies; and, that old chestnut of dealing with difficult people.

Even in the more lurid scenarios I have created for my classes, I’ve never been close to something like this (and I’m not sure I’d like to have been, either), so having this kind of (legitimate) access is a godsend for enriching our materials.

And this isn’t an isolated case.

The more we can help make connections for students to Stuff Outside The Classroom, the more chance we have of putting them in a position to become critically engaged with their environments, applying their learning and making actual use of it themselves.

This doesn’t have to be about providing the answers, but about asking the questions that stimulate reflection and debate.

And if you don’t think this example is good for your class’ needs, then there’ll be something along before you know it.

Changing a Comparative Politics Course, Part 3

As promised in my last post, here is Comparison 3, an assignment in which each student chooses independent variables and operationalizes them for the two cases chosen in Comparison 1. The instructions for the assignment:

Using the template that you submitted for Comparison 2, replace “Variable 1” and “Variable 2” with two independent variables that you think have a causal relationship with the dependent variable. Replace “Indicator 1” and “Indicator 2” with indicators of each of the independent variables you have selected — indicators are the means by which you will measure changes in your variables. Enter the data for the indicators for 2000 and ~ 2020 as shown in the template. Enter + or – to show where these measures increased or decreased.

Data on many indicators for independent variables can be found at:

Continue reading “Changing a Comparative Politics Course, Part 3”

Online persistence

This isn’t going to be about trolls who just won’t give up, so apologies from the start about that.

toot toot!

Yesterday I spent my coffee-break reading about trains across Russia. Partly that was from a random thought, partly it’s because I’m a man-child who really struggles to concentrate, partly because the site I was reading about this was really engaging.

For those not in the long-distance-train-journey community, you only need to know that the guy behind this started off many moons ago, putting together some pretty basic info and, well, it rather snowballed from there.

I mention it because the site is a good example of someone not only carving out a niche, but turning it into the kind of resource that is both connected-to and thought-about as a place to go by a wide audience.

Of course, this makes me think about a number of things.

Most immediately, I makes me reflect that while ALPS has built up a considerable profile and reputation within the immediate L&T community in Poli Sci/IR/European Studies, we still have a way to go with other disciplines (or with those less motivated by the prospect of teaching): so do continue your good work of recommending us, and writing for us.

More generally, it touches on a thought that’s bothered me for a while about creating online resources: how do people find stuff?

Our decision here to go for a blog format, made nearly a decade ago in Albuquerque, made sense at the time for our needs and capacities, but we now have over 1400 posts and we didn’t do a particularly assiduous job of tagging stuff. Even I struggle to track down things, especially if different posts have covered different aspects.

It’s also becoming an issue with the Brexit-related graphics I’ve been making for a couple of years now. I tweet these out, and have hosted PDFs on a Google Drive folder, but it’s not searchable and if you don’t know me, or even couldn’t remember my name, then you’d struggle to even track down the things in the first place.

This isn’t an issue of “look at all my work”, but rather of the annoying feature of political life (and L&T) to keep on circling around to the same issues. Getting quick feedback in a class is just as much a problem now as it was in 2011.

It’s also a matter of trying to save colleagues having to reinvent the wheel all the time: if you knew what was out there and could access it easily and quickly, then you might not start from first principles. Sure, in some cases you still would, but at least you’d have more sense of what those first principles might be.

So the open question from all this is how we can build resources that are persistent, visible and useful to audiences?

In the case of ALPS, we have talked at times about making non-blog format material, but keep coming back to the challenge of creating – and, crucially, maintaining – a static site. The Man in Seat 61 has been able to do that, but we’ve not been able to make it work with our other commitments.

As so often, there’s not an easy answer to this, but it is something we’ll all have to keep on working on. Digital materials are only going to become more important with time and as academics we seem to be asked ever more about demonstrating our digital footprint. That means traffic data as much as content production, so that’s another can of worms.

But we can only continue to try. if you have some ideas that have worked for you, then let us know.

Maybe even write us a post and then we can share it with everyone who can find it.

Changing a Comparative Politics Course, Part 2

In Part 1 of this series, I discussed changing my approach to teaching students how to analyze the arguments contained in journal articles. I also think it is important for students to actually do some discipline-related research rather than just read about it. Previously in this course, my students compared two nation-states using either a most similar systems or most different systems design. That assignment never worked very well because of student confusion about the basic nature of cause and effect. I’ve decided to replace this with a scaffolded process culminating in a team-produced qualitative comparative analysis.

There are three individual assignments that I’m calling Comparison 1, 2, and 3. For Comparison 1, each student chooses two nation-states from a list. That’s it. The list comes from Freedom House’s rankings of citizen freedom in countries around the world; I selected a subset of states for which scores differed between 2000 and 2019 — so that students choose cases where the dependent variable varies over time.

For Comparison 2, students calculate a value for the dependent variable. Here are the instructions for the assignment:

Continue reading “Changing a Comparative Politics Course, Part 2”