For reasons that I think mainly relate to having missed the relevant meeting, I am my Department’s Impact lead for REF, the multi-annual evaluation exercise of research quality in British universities. The Impact part of it relates to efforts to measure what, um, impact that research has outside of the academic community.
For the purposes of this post, let’s assume I’m thrilled with this honour.
The intention behind REF might well be an honourable one, but as so often in life, the practice is more complicated than the theory. How does one measure any of these things? What does one measure? And so on.
A particular bugbear for me, and other UK-based academics in the same situation, is the ambiguous position of pedagogic research.
REF splits its work into panels, for specific disciplines, including one for education. So I could try to get my pedagogic research outputs considered by that panel, even if I’m returned to the Politics/IR one: it’s a bit cumbersome, and hasn’t happened very often in practice, but it’s possible.
However, the impact side is more complicated. The previous cycle of REF, in 2014, seemed to say that pedagogic impact could be considered, as long as it was in a different institution from your home one, but in practice hardly anyone submitted such a case study outside of the education panel.
The root issue has been a very conservative view of the REF guidance, which seems to suggest that while cross-referrals are possible, it might some how count against you that you hadn’t stayed within the bounds of the discipline. Quite apart from the mixed messages that sends out about inter/multidisciplinary research, the nervousness of universities to submit any pedagogic research – unless from dedicated units – speaks to the rather impoverished situation of such work.
I’m aware of several individuals – not least myself – who were told that their (excellent) pedagogic outputs and impact would be used in the previous REF, because it didn’t fit the rest of the submission, and because ‘people hadn’t done that last time’.
Eagle-eyed readers will notice the circularity involved here: already for REF2021 I hear the same arguments being floated about.
I raise this not because I’m sore about the whole matter, but because when reading the criteria for impact, and when attending events about improving impact, the consistent message I hear is that you should submit work that demonstrates the clearest impact.
For someone like me, that leaves me in an odd position. On the one hand, I’ve got a bunch of impact-y things going on around Brexit and UK-EU relations: it’s very topical, I do lots of stuff, but I’m also not very sure that I can show impact from it.
On the other hand, I’ve got my pedagogic work, which is more modest, less visible, but I know I can show that I’ve reached significant numbers of people and changed how they teach and how they work (you might well be one of them, and you’re very welcome to write me a note saying how my research and activity has changed what you do).
I’m guessing that others of you will be in the same position.
All of this invites me to ask what the ‘best’ strategy might be. Do I focus on the stuff that fits more cleanly with the rest of my Department’s submission, or do I focus on the stuff that might perform better against impact metrics?
Conversations will ensue in the next years, but if you have thoughts then I’d love to hear them.