I zoomed into an excellent QAA event this week on implementing racially inclusive practice in assessment, based on a project at University of Leicester, partnering with Birmingham City University and University of Wolverhampton. I’d very much recommend that you have a good look at their report in detail. The take-home for me was that that whilst an inclusive or decolonised curriculum and role models are incredibly important for engagement and for inspiring students, particularly racially minoritised students, if you want to tackle race awarding gaps, the solution is pedagogical.
Their approach is deceptively simple: they focused on making sure that the hidden curriculum is made visible for all students and the tacit is made explicit, that students understand exactly what they have to do to succeed, with no guessing games involved, with clear documentation of what is required, and that all assessment criteria are clearly and transparently explained with examples of what good or less good work against those criteria would look like. One of the staff who had implemented the intervention very disarmingly said that he felt a bit embarrassed that he and his colleagues hadn’t been doing this already! He also said that although there was some initial resistance because of worries about ‘spoonfeeding’, the improvement in the students’ work that he saw and the way they engaged allayed most of those fears. They found that by doing this, they could reduce awarding gaps significantly, improve student enjoyment and confidence, and also improve staff experience of teaching and assessing!
There is a lot to learn from in the report. Personally, I’ve already thought a lot about assessment criteria over the years, in an attempt to be inclusive, yes, but also because I just wanted to communicate with students what I wanted them to do, so they would learn better and I could read better work when assessing. As a less experienced teacher, I realised that I was marking work down for not doing things that I had never taught or told the students to do – which offended my sense of justice. But I knew I did want the students to do those things (such as make a coherent argument, evaluate evidence, use examples, write for an audience, use appropriate referencing), so it got me to thinking about how I might teach those things in the context of a disciplinary and substantive module. I came to the conclusion that having transparent criteria and spending some time making sure that everyone understands them would help me communicate what skills I wanted to see and how they might develop them. It turns out to be a practice that serves all students – not just those who have been previously disadvantaged, but also the ones who keep doing pretty well, but don’t know why.
As we know that tutors are often looking for different things in their students’ work, it usually doesn’t work in a discipline like ours to have generic or departmental criteria. It is an incredibly useful exercise for you, as a tutor, to sit down and write out what it is you are looking for in students’ work. This helps clarify expectations for me and helps me think about what and how I will teach. When team-teaching, working with other tutors to clarify not only what the assessment criteria are but also what they mean in practice is extremely useful for making sure that teaching and marking are fair and consistent. And working with students to help them understand marking criteria doesn’t so much help them ‘tick the right boxes’ in a spoon-feed way, but, much more importantly, understand what skills they are learning and why.
For my current module, the assessment is a portfolio, and the assessment criteria are as follows (although I do allow students to negotiate them, which I won’t dwell on here but will come back to another day):
- Depth of understanding of how politics and power are shaped by, and shape, the natural world
- Ability to weave together ideas from the module into your own coherent text
- Depth and originality of critical evaluation of your own relationship with the natural world
- Ability to argue for your perspective on how nature should be governed or cared for, by whom and in what ways, including use of reasons and evidence
- Appropriate selection of multimedia on the portfolio
- Ability to write appropriately for a particular audience (please specify: eg visitors to an exhibition, policy-makers, everyday readers of narrative non-fiction)
- Creativity of your work on the portfolio
- Evidence of learning and development over time in the module
- Depth of critical engagement with the module materials and readings
- Extent of additional research and further reading
- Craft of writing, including readability, spelling and grammar
- Accuracy of bibliographic materials
I like the approach of starting with a noun plus preposition, like ‘depth of’ or ‘ability to’, because it demonstrates that these are skills one can be better or worse at in a qualitative sense. Thus, this is not a box-ticking exercise for students but rather an invitation to engage in deep and dialogical reflection on what, for example, the ‘ability to argue’ or ‘appropriate selection of multimedia’ really looks like in practice.
It’s very important not to stop with listing the assessment criteria, of course, but rather to make them the centre of an ongoing conversation. Here is my top tip: every time a student asks a question about the assessment, or about what ‘good work’ might look like, I bring it back to the assessment criteria. So, let’s say they ask, ‘does my portfolio need to be consistent week by week?’ I will say, ‘Is that in the assessment criteria? No. So, I won’t be looking for that. If it’s something you want to learn, that is, how to create your own consistent style, that’s great – you can do so and add it to the assessment criteria for your self-assessment. But it’s not necessary from my point-of-view.’
Here is my top tip: every time a student asks a question about the assessment, or about what ‘good work’ might look like, I bring it back to the assessment criteria.
Or let’s say they ask, ‘Can my writing be more personal?’ I will say, ‘Is it in the assessment criteria?’ This is a longer conversation – the answer is, yes, I am asking them to give an account of their relationship with the natural world, so more personal writing in the first person is clearly appropriate. However, if they are using part of their portfolio to write for policy-makers, this can lead to a deeper conversation about what sort of writing, evidence and argument a policy-maker might be interested in. Distinguishing these different crafts of writing and talking about when they are appropriate, or not, is much more useful for learning than just prohibiting one of them without explaining why.
Other ways of getting students to engage deeply with the assessment criteria might include:
- Guided marking exercises where students mark examples of work with reference to the assesment criteria. Your aim here is to get them to focus on the criteria and not make the sorts of vague comments (‘this was not well structured’) that they have probably experienced themselves at times.
- Peer feedback where the focus is on giving each other feedback according to one or more of the assessment criteria.
- Formative feedback from the tutor where they have to tell you which criteria they want feedback on. (I have a form and they can’t have their feedback unless they tell me which criteria they are particularly interested in.)
- Self-assessment where students have to tell you how well they met the criteria, and where they could have done better.
- Any other discussion with examples of the criteria and what they mean, preferably iteratively, so they can improve over time.
Summative feedback should also, of course, refer constantly and closely to the assessment criteria. But by that point, this is just an exercise in demonstrating that you could be trusted to do what you said you were going to do. To return to the QAA discussion on racially inclusive criteria, the return of summative work should not be an opportunity to say: ‘Ta-DAH! This is what you should have done.’ What the students should have done should be clear right from the get-go, or else how can they learn how to do it?