British universities are now all back up and running for the academic year, after our traditional summer break, which we like to use laughing at American colleagues who had to start back in August (August!!).
As pay-back for such rude behaviour, we’re also now about to receive the government’s latest thinking about Learning & Teaching, in the form of a Green Paper.
No one is sure what it’ll contain, but it’s likely to set out plans for a Teaching Excellence Framework (TEF), linked to student funding and to assorted other reforms of the sector. I’ve written about TEF before, but I’d like to return to the theme, since it looks set to become the defining feature of L&T in this country for the next decade or more, just like its research counterpart.
Part of the thinking that now seems to be floating around is a progressive building of TEF from a starting point of basic metrics (progression, completion, good degrees, employment, etc.) into something with more of a qualitative element. It’s this second part that I’d like to consider some more.
Personally, I find the concept of ‘high quality teaching’ rather problematic. How does one judge such a thing? Is it about the quality of the outputs, or the inputs, or the improvement between the two? Does it need to be grounded in pedagogic research, or is it sufficient that ‘it works’? Do we judge the individual elements of learning & teaching (whatever that might mean) or the whole package (whatever that means too)? Do we stick to the formal curriculum or to the softer parts of a university experience?
All are valid questions to ask, and all provide very different answers and incentives to act in a particular way.
Consider a challenge that my home department faces. We tend not to get a terribly high percentage of first class and upper second degrees (‘good degrees’ in the jargon), compared to the rest of the university. It’s not a big difference, but enough to be noticed. One of the main reasons that we’ve identified is that we use a wide range of forms of assessment compared to others: not just essays and exams, but also presentations, group projects, negotiating briefs, policy briefs, reflective pieces, and so on. We do that because we believe it develops more useful skills for our students, since they are much more likely to encounter those kinds of things in the outside world.
The problem is that our range of assessment means that students don’t get the depth of practice that they would if they only did essays and exams, which in turn tends to depress their grades. Having seen examples of such assessment monocultures (bicultures, more precisely), I can attest to their value in maximising performance in those assessment forms.
So the question is whether that’s a good thing or a bad thing. And one could see how the argument could be made both ways. The real challenge comes in working out how to balance different claims in any evaluation exercise.
The riposte to this would be that the combination of quantitative and qualitative elements in TEF would exactly allow for universities to contextualise and clarify their situations. But that’s not the same as offering a mechanism for balancing out the different parts, not least for the inevitable league table that will result from all of this.
As I discussed last time, I feel rather conflicted about TEF. On the one hand, it’s an excellent opportunity for L&T to be brought to the foreground and given the attention it requires and deserves. On the other, I worry that it will tend to push us back into small-c conservatism of practice, where innovation and risk are discouraged, because of the potential upsetting of the basic metrics that will weigh heavily in the exercise. We’ve seen something similar with REF, where the gaming of the rules is the dominant mode.
If there were a way to embed an incentive to challenge ‘traditional’ practice and to create spaces for trying out new ideas, then I’d be very happy, concerns over innovation for innovation’s sake notwithstanding. Even just something that required us to justify the choices we make would be a start, maybe in the form of a statement by each teaching unit of its pedagogical underpinnings. Much of it might be rather vapid stuff to start with, but the exercise would at least force people to engage with the research and literature that’s out there and get that ball rolling.
Another possibility might to require universities to host a minimum number of L&T testbeds, to try out new approaches, with their outcomes ring-fenced from the metrics, so as to allow for some risk-taking and to generate internal potential for more wide-spread change. Certainly, there’s an issue of how students might feel being used as guinea-pigs, but that’s manageable.
These are just random thoughts, but I hope they point towards some of the tensions that have to be resolved with TEF. That the Green Paper has been delayed suggests that the politicians have not agreed on what should be included: academics need to be ready to engage and contribute to shaping whatever comes out.
“The more any quantitative social indicator (or even some qualitative indicator) is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”
That’s also known as Campbell’s Law, after Donald Campbell (all genuflect at this point). And he’s right, as your post shows. Are the students in your department better educated in terms of general competence then those in the “bicultural” programs? Sure; that isn’t even something to argue about. Yet your program finds itself being invidiously compared to others because your students don’t do as well in assessments based on bicultural education methods.
Why? It’s right at the start here. There’s nothing wrong with testing per se, but, to quote Campbell again:
“… achievement tests may well be valuable indicators of general school achievement under conditions of normal teaching aimed at general competence. But when test scores become the goal of the teaching process, they both lose their value as indicators of educational status and distort the educational process in undesirable ways. (Similar biases of course surround the use of objective tests in courses or as entrance examinations.)”
Yep. That’s it alright.
I don’t have a solution for this since, if anything, the UK is even more subject to the Law then the US is. I would suggest, however, that you begin to track your students over the long term and see how they are doing after school. The perfect squelch for an emphasis on corrupted testing is showing that other educational methods work as well or better.