Evaluation or dialogue: how to listen to students?

Happy new year! I was glad to see Jennifer writing about student evaluations last week because this is a topic I’ve been meaning to write a bit about, since at UCL we have recently been operating a bit of a different approach.

Like most places, we used to have module evaluations which generally used to be administered centrally by departments, with students anonymously filling in an online questionnaire including a top line question along the lines of ‘how satisfied were you with this module?’. For the past 18 months or so, though, we’ve instead been asking tutors to engage in meaningful and ongoing dialogue with students about how they are doing with their learning and how modules can be developed or tweaked to help them learn better. This seems to me to be a sensible institutionalisation of what the most experienced and able teachers do anyway. It’s also an approach that sits better philosophically with me. I think we all have an instinct that it is contradictory to position students both as customers who are in an adversarial relationship with their teachers, always asking for more things that may not or, perhaps, cannot usefully be delivered and as members of our community who have responsibilities as learners as well as a right to exercise voice within that community. Whereas anonymous surveys premised heavily on ‘satisfaction’ play heavily into the consumer model, a deep, thoughtful, two-sided dialogue with students focused on learning sits much more comfortably with a community of practice approach.

If you go on Twitter, you will probably come away believing that most of the academic community in our discipline is opposed to neoliberal framings of students as consumers and disapproves of module evaluations because of the bias and, sadly often, nastiness that always seem to come with them. I was therefore genuinely surprised to find that one main obstacle to implementing a more dialogic approach was…. colleagues who teach, but are not necessarily in educational leadership positions, who wanted to stick with the old ways of anonymous, satisfaction-based surveys! Change is difficult and doing something new is challenging. Being able to hold a deep, thoughtful dialogue with students about learning and how your own practice could be better is very difficult, especially on the spot. Complaints abounded that it was no longer easy to demonstrate how good our teaching is (despite acknowledgement of all that bias) or readily discipline the tutors that weren’t doing it right (despite our often deeply-held conviction that we should trust one another). And messaging on the new approach from different places within the institution was not always clear or aligned with the philosophy that I personally prefer. Colleagues were encouraged to use Mentimer to hold quick in-class check-ins, which could readily be the starting point of a deeper and more dialogic discussion – but out of habit and lack of clear support for doing something different, they became… more student satisfaction surveys. This meant that students also sometimes treated the Menti check-ins not as a jumping off point for deep dialogue but rather as a chance to make consumer demands and, very occasionally, at least so I’ve heard, indulge in unconstructive comments. As I am in a leadership position and expect myself to do what I am asking others to do, I pushed on with Menti along with my own tweaks, but I might have abandoned it and just, you know, talked in a focused and dialogic way with the students if I had been a civilian! We also, as an institution, still ran programme evaluations with free-text boxes for comments on individual modules and, no surprises, some unconstructive or even downright mean comments re-emerged there. And no UK University (unless, apparently, you are Oxford or Cambridge?) can escape the National Student Survey and the frenetic activity that accompanies it, although at least this no longer includes a question on ‘satisfaction’.

I have to hold my own hands up on all this resistance and misunderstanding a bit. I was new in the Vice Dean role when this novel UCL policy came in and, perhaps because I live too much on Twitter, I assumed it would be a welcome new approach and that we could use its ambiguities to shape it in a way that privileged dialogue and joyfully abandoned a survey approach. We did provide lots of written guidance, with pretty clear hints that this was a flexible approach that asked only that tutors are expected regularly to engage with students on how it’s going, listen open-mindedly to what they have to say, make changes where appropriate and devote time to discussing with students how they – as well as we – might make changes to improve the overall learning experience in the module. But if I were doing it again, I would frontload a lot more support for tutors for having difficult conversations with students, and try to have a lot more discussion among tutors about why devoting time to ‘how to learn’ and ‘what is working for you and why’ might be a better way of spending even limited time than focusing on more content. Now I am more confident and have stronger and more trusting relationships with colleagues in various educational roles, I might also be a bit clearer about how the new approach accorded with my own philosophy, regardless of whether that was everyone’s interpretation.

I think for many tutors and students, unfortunately, the new system has bedded in as a system of ‘three Menti satisfaction surveys a term’ rather than the larger philosophical shift I was hoping for. Nevertheless, for those teachers (I think, probably, the large majority) who are keen to reinscribe students as members of a learning community and to reflect jointly with students on how to learn and how to teach, we now have an institutional policy and mechanism that can support that. We also don’t have to learn from students, per Jennifer’s post, that our car is rubbish* or we should hold our office hours in a different office. So, that’s something. Medium steps forward, little steps back. But any thoughts on how to produce the sort of mindset change that would move us towards thinking of ourselves as a learning community who really listen to students as colleagues and not customers, always much appreciated.

*for the avoidance of doubt, I live in London and my ‘car’ = a sturdy pair of trainers