Biting off more than you can chew?

kerry_and_lavrov_with_senior_advisers_negotiate_chemical_weapons_agreement_on_september_14_2013
We can do better than this. Probably

A while back, I wrote about running a sim on Brexit, the UK’s departure from the EU. (Obviously, ‘departure’ makes it sound grand and stately, rather than the big old mess that it more actually resembles, but I digress). In it, I asked if anyone was up for working together on doing this.

Well, we have a winner: Matthew LeRiche from Memorial in Newfoundland. Matthew joined us in Surrey earlier this year for our ALPS workshops, and this doesn’t seem to have discouraged him from working with us again. Continue reading

New Brave New World (again)

A university, yesterday

Like the ticking of a rather slow clock, we’ve just had the publication of a White Paper on Higher Education here in England and Wales. This comes after a long consultation process that stretches back into the previous Parliament and which leaves as many questions open as it addresses.

I’m not going to run through all of it because a) it’s very long, b) not all of it relates to L&T and c) I can’t face it: there’s lots of good pieces here and here. But I will update on the famous Teaching Excellence Framework (TEF), which I’ve written about before (here, here and here).

For those of you who have managed to avoid this idea so far, firstly my congratulations: essentially, it’s seen as an equivalent of the Research Excellence Framework (REF), trying to provide some judgement on the quality of teaching provision in a way that is comparable across institutions and disciplines. In so doing, it aims to increase transparency for students and to drive up standards.

As with all good ideas, the operationalisation has been the problem: how do you actually do this? Continue reading

What makes for ‘quality teaching’?

Nope
Nope

British universities are now all back up and running for the academic year, after our traditional summer break, which we like to use laughing at American colleagues who had to start back in August (August!!).

As pay-back for such rude behaviour, we’re also now about to receive the government’s latest thinking about Learning & Teaching, in the form of a Green Paper.

No one is sure what it’ll contain, but it’s likely to set out plans for a Teaching Excellence Framework (TEF), linked to student funding and to assorted other reforms of the sector. I’ve written about TEF before, but I’d like to return to the theme, since it looks set to become the defining feature of L&T in this country for the next decade or more, just like its research counterpart.

Part of the thinking that now seems to be floating around is a progressive building of TEF from a starting point of basic metrics (progression, completion, good degrees, employment, etc.) into something with more of a qualitative element. It’s this second part that I’d like to consider some more.

Personally, I find the concept of ‘high quality teaching’ rather problematic. How does one judge such a thing? Is it about the quality of the outputs, or the inputs, or the improvement between the two? Does it need to be grounded in pedagogic research, or is it sufficient that ‘it works’? Do we judge the individual elements of learning & teaching (whatever that might mean) or the whole package (whatever that means too)? Do we stick to the formal curriculum or to the softer parts of a university experience?

All are valid questions to ask, and all provide very different answers and incentives to act in a particular way.

Consider a challenge that my home department faces. We tend not to get a terribly high percentage of first class and upper second degrees (‘good degrees’ in the jargon), compared to the rest of the university. It’s not a big difference, but enough to be noticed. One of the main reasons that we’ve identified is that we use a wide range of forms of assessment compared to others: not just essays and exams, but also presentations, group projects, negotiating briefs, policy briefs, reflective pieces, and so on. We do that because we believe it develops more useful skills for our students, since they are much more likely to encounter those kinds of things in the outside world.

The problem is that our range of assessment means that students don’t get the depth of practice that they would if they only did essays and exams, which in turn tends to depress their grades. Having seen examples of such assessment monocultures (bicultures, more precisely), I can attest to their value in maximising performance in those assessment forms.

So the question is whether that’s a good thing or a bad thing. And one could see how the argument could be made both ways. The real challenge comes in working out how to balance different claims in any evaluation exercise.

The riposte to this would be that the combination of quantitative and qualitative elements in TEF would exactly allow for universities to contextualise and clarify their situations. But that’s not the same as offering a mechanism for balancing out the different parts, not least for the inevitable league table that will result from all of this.

As I discussed last time, I feel rather conflicted about TEF. On the one hand, it’s an excellent opportunity for L&T to be brought to the foreground and given the attention it requires and deserves. On the other, I worry that it will tend to push us back into small-c conservatism of practice, where innovation and risk are discouraged, because of the potential upsetting of the basic metrics that will weigh heavily in the exercise. We’ve seen something similar with REF, where the gaming of the rules is the dominant mode.

If there were a way to embed an incentive to challenge ‘traditional’ practice and to create spaces for trying out new ideas, then I’d be very happy, concerns over innovation for innovation’s sake notwithstanding. Even just something that required us to justify the choices we make would be a start, maybe in the form of a statement by each teaching unit of its pedagogical underpinnings. Much of it might be rather vapid stuff to start with, but the exercise would at least force people to engage with the research and literature that’s out there and get that ball rolling.

Another possibility might to require universities to host a minimum number of L&T testbeds, to try out new approaches, with their outcomes ring-fenced from the metrics, so as to allow for some risk-taking and to generate internal potential for more wide-spread change. Certainly, there’s an issue of how students might feel being used as guinea-pigs, but that’s manageable.

These are just random thoughts, but I hope they point towards some of the tensions that have to be resolved with TEF. That the Green Paper has been delayed suggests that the politicians have not agreed on what should be included: academics need to be ready to engage and contribute to shaping whatever comes out.

Proxy, proxy, proxy: out! out! out!

IMG_0077
Right place, right time, wrong conversation

At some point we must have run out of conversational topics on holiday, because I found myself on a beautiful Croatian beach, trying to explain the ‘good degrees’ metric to my wife. To my credit, I didn’t try very hard, although it wasn’t helped by the fact that I struggled to come up with a good justification for it.

For non-UK readers, “good degrees” measures the percentages of awards in the top two categories (firsts and upper seconds) for undergraduates. That’s about 70% of them, a figure that’s risen over time.

My Croatian struggle came in trying to unpack what it showed and why that might be useful.

Partly, it’s a reflection of our student body: if we have more academically-able students, then they’ll be more likely to get better grades, and so a ‘good degree.’

But equally, it’s also a reflection of our assessment and grading. Are we the kind of place that rewards good work (assuming that student populations are broadly equivalent)?

So what does it show? Our selectiveness? Our accommodating assessment? Our great students? Something else?

The issue is one that might have remained of minimal interest to anyone, but for the use of the metric in league tables. These aim to rank universities on the quality of their provision and have become an important part of the British HE sector, not least for recruiting new students. However, that also introduces an incentive to improve one’s ‘good degree’ metric.

The obvious responses in this situation would be to have stricter entry requirements (which is also a league table metric) and to have more lenient assessment. The first is very much in the universities’ hands and you’ve seen exactly that happening. The second is a very different kettle of fish.

Assessment in the UK differs from many other countries with its use of second- and external-examining (see this for more), which is in turn bound by a national framework of performance standards: broadly speaking, any one at a UK university should be marked to the same kind of standard, whatever and wherever they study, against benchmarks, rather than against each other. And, broadly speaking, that’s what happens.

The upshot is that any (league table) pressure to go soft on assessment is counter-weighted by the need to meet sector and disciplinary norms: as our university tells us, the norms are the norms and we shouldn’t be messing around with them.

The problem now becomes one of still wanting to improve ‘good degrees’, but losing the most obvious tool to achieve that. So instead, we try other things. These cover pretty much every part of what we do in our teaching. Are there pinch points in assessments or modules that we can redesign, to meet learning objectives in a more accessible way? Are there staff how need help with redesigning assessments or with providing support? Are there early markers of weaker performance that can trigger timely intervention? Are there extra resources we could provide that would help improve student outcomes?

All of those questions are good questions to ask of any educational body, since they all speak to the core question of ‘how do we help our students to achieve their full potential as learners?’

Unfortunately, all of those questions are not ones that convert into nice, simple metrics. Instead we fall back onto things like ‘good degrees’ and back into the question of what we are trying to do.

The issue is an old one: if you make something a metric, then people tend to work towards those metrics, they gain value in of themselves. That’s a problem if we forget that metrics are proxies, which is all too easy to do.

As we move towards a Teaching Excellence Framework here in the UK, that’s something we’re going to have to deal with more and more.

Teaching Excellence Frameworks, and other things to think about over the summer

Soft_ruler
This measures everything. Everything.

One of the mutterings that has flitted about the HE sector here in the UK in recent months has been the idea of a Teaching Excellence Framework, an equivalent of the Research Excellence Framework (REF) that has either come, or is coming, to your local university.

As with REF, the idea would be to use a range of metrics to identify, measure and encourage high quality learning & teaching. That might include completion rates, ‘value added’, dissemination of good practice and case studies of innovation and general excellence.

The idea is one that got a big boost last week, when the new government announced that it would allow British universities to increase their fees if their teaching was of high quality.

Now you might imagine that I’d be all for that, since I could write an impact case study already now and my institution does very well in L&T metrics.

However, I find myself being deeply ambivalent about it all.

In part, that ambivalence is driven by my general concern about any one-size-fits-all approach, especially because – just as in research – there are very many ways to make a useful contribution and (conversely) just as many ways to not capture that. How do you compare a MOOC with lab work with a simulation?

But it also comes from a concern about the abstraction of L&T from its context.

When I run a module, I run it with the particular group of students in mind. Each class or session finds me (like it would most colleagues) trying to adapt the material to meet the needs of the students as we progress. Most mundanely, the confused face will prompt us to rework some section, to unpack it and help that confusion pass. Less mundanely, I might rework an entire module to accommodate the needs of a group (as indeed I will be doing this summer).

In REF, individual researchers submit four pieces of work for peer-evaluation: this forms the bulk of the work, alongside some elements dealing the research environment and its wider impact.

In a TEF, it’s inconceivable that we’d do the same: I submit my four best students? My four best modules? My four best sessions?

No, there’d be a bunch of metrics at programme or departmental or university level, describing some of those things I’ve mentioned above. Then there’d be some kind of case study approach.

This anonymises – or, rather, depersonalises – what we do. As a useful piece by John Canning points out, a TEF might start out with good intentions, but quickly institutions would move towards playing the game and optimising the metrics.

This already happens: institutions in the UK spend a lot of time and effort in trying to improve their league table rankings and that includes targeting poorly-performing indicators. That’s not a bad thing – my own university now has much better systems for supporting students who might fail modules, for instance – but the tendency is always to focus on the metric, rather than the students themselves.

I appreciate that in all of this, I’m just the sort of person who should be getting involved, as someone who cares about, and has a substantial job role in, L&T. So I’m going to try to use my ambivalence to good effect. That’s why I’m going to try and be part of the discussion about TEF wherever I can, so that even if it’s not perfect, it will still be less imperfect than otherwise. And I’d say the same to colleagues: rather than just grumble about what’s being done to you, try and shape it.

All that said, I’m about to have a couple of weeks of not thinking about work at all. Doubtless I’ll return with some moment of inspiration from my break, but I really wouldn’t hold your breath. See you on the other side.

3 ways to teach the 2015 General Election

UK_opinion_polling_2010-2015For a very long time, teaching about British general elections was hampered by one simple fact: you were never quite sure when they would happen. Unless there was a clear play to sit out the maximum term (as in 1992), then students and instructors would be at the whim of the occupant of No.10.

So the introduction of fixed term parliaments by the current administration is a real boon: we have a date (7 May), which also falls nicely towards the end of the teaching period in the spring semester. This lets us think about some creative ways we might teach about it.

I’m assuming that those devoting entire modules/courses to the subject will already be up and running, but for the rest of us, there’s still the opportunity to import aspects of what’s happening into the classroom. Consider these three starting points.

1: real-time information

The most obvious thing to do is to get students aware of, and discussing, what’s happening right now. Major news outlets are setting up micro-sites for the election, aggregating content. Likewise, there are some excellent resources, including those handling polling data (here and here) or more general academic comment.

What’s becoming evident is that there is a huge amount of near-instantaneous material/comment available, so setting up a module account on Twitter to collect and share it is a great way to capture this and funnel it into the class: perhaps you set the last half-hour of a session to discuss what’s happening.

2: active debate

Last semester I got my students to role-play a debate on the television debates, playing different parties. This topic is still on-going and gets the students into thinking about the media, the status of parties and the semantics of public debate, in a form that is readily accessible.

If that’s not your thing, then you could run a hustings, or a putative post-election coalition negotiation. All of these active-learning models force students to take their knowledge and apply it to a situation, where they can see how positions interact with each other and where presentational (and representational) factors can strongly influence the course of events.

3: comparative approaches

Aware as I am that most of our readers aren’t UK-based, it’s also worth thinking about using the British case as a comparator for your own national system. As I’ve noted, the quantity and quality of material available is extremely high, and students’ awareness of the basic features of the system should be pretty good (at least in the relative sense). Couple this to the obvious differences between your country and the UK (electoral system, parliamentary government, party structure, media structure, etc.) and you have a great opportunity to leverage more insight into how national polities work.

This might be of comparing electoral campaigning, or issue development, or even just how your country sees what is going on elsewhere in the world (this last one might well be most suitable for those places that think they are the centre of the world (ahem)).

As I say, this is just a first effort, but if you have more suggestions, either of techniques or of resources, then do please post below in the comment box.