Generative AI changes teaching and learning: how to protect the integrity of assessment

This academic year, the UCL Centre for the Pedagogy of Politics (CPP) is hosting a series of online panel events. Our first event on 30 October was on the theme of ‘Using technology to teach politics’. In this guest post, one of the panellists at that event, Simon Sweeney (University of York), offers further reflections on the challenges involved in higher education’s embracing generative AI, where tools such as ChatGPT call into question issues of authorship and have profound implications for assessment.

A few years ago, we were worrying about students’ using essay mills, a form of contract cheating that plagiarism detection software struggled to identify. The Covid-19 pandemic and online delivery coincided with a reported increase in academic dishonesty (AD). In late-2022 the arrival of generative artificial intelligence (GAI) chatbots like ChatGPT is a further challenge to the integrity of assessment.

Universities realised that banning chatbots was not feasible, as AI has become an established feature in our lives and graduate employment. As educators, we need to respond positively to the opportunities AI presents, recognising its benefits and assimilating AI into teaching and learning practice.

This means developing strategies that accommodate students’ use of GAI while protecting assessment integrity.

Continue reading “Generative AI changes teaching and learning: how to protect the integrity of assessment”

Little by little: Challenging awarding gaps in UK higher education

We have another great guest post by Jeremy F. G. Moulton at the University of York!

The Financial Times recently published an analysis of data from HESA (the UK’s Higher Education Statistics Agency) which revealed a troubling trend in UK universities. As the analysis’s headline read: ‘Non-EU international students at UK universities less likely to get top grades’. The data was clear – undergraduates from outside the EU were twice as likely as UK students to receive a lower-second or third-class degree in 2021-22 (and therefore missing out on the upper-second or first-class degrees that employers most favour).

This trend is especially concerning given the significant growth in the number of non-EU international students studying at UK universities in recent years, with almost three times as many studying in the UK today than in 2007. With these students making up a larger proportion of those within our classrooms, there is a need to ensure that universities are not providing them with a second-class education.

The problem with awarding gaps is not, however, limited to the difference in degree outcomes between non-EU international students and EU and “home” students. There is a wealth of data already covering racialised awarding gaps, socio-economic awarding gaps, and disability awarding gaps, for example. 

The question is – what can be done to help challenge these trends?

I work at an institution which has the ambition of ‘eradicating’ awarding gaps by 2030. This has given those working on teaching and learning a serious opportunity to engage with the issue, to understand the challenges, and to make some early steps to address those gaps.

Of course, there are many factors that shape awarding outcomes where we have limited power to respond. Structural issues will often be the deciding factor on, for instance, whether students will have to take on part-time work to fill the increasing gulf between the costs of student life and the student loans provided. 

This is not a niche issue, over half of undergraduates in the UK are now in part-time employment. Some universities are already experimenting with compacting students’ timetables so classes only cover three days of the week. This gives students clear days for paid work and, hopefully, ensures that they don’t miss any teaching. This compartmentalisation of the week might also be of benefit to academics by providing set days for research, writing, and tackling that ever-growing pile of emails. 

However, there are also opportunities for individual, course-level initiatives to achieve the ambition of combatting awarding gaps.

The first opportunity is to reflect on how we are assessing our students. In particular,  optionality could be one particularly effective path forward. Letting students select from a range of assessment formats, tailored for respective modules, would allow students the choice of assessment to best suit their learning style and the skills that they are seeking to develop through their studies. A recent report on optionality in assessment highlighted its potential to limit awarding gaps, making the recommendation that ‘Educational institutions should prioritise the introduction of diverse assessment formats to explicitly address accessibility and concerns about fairness, ensuring access to necessary resources and skills development to prevent the unintentional widening of awarding gaps’. 

The second opportunity is to take the time to consider the range of cultures and approaches to learning that are present in our increasingly diverse classrooms, and to engage students on these matters. It is easy to take for granted that the approaches to teaching and learning that have been taught or socialised to us are in some way the ‘standard’ but this is a practice that risks alienating some students. 

For example, research on students from China’s learning styles has highlighted that they often do not have experience in or enjoy some of the norms of active learning that are the mark of much of the teaching and learning in Western institutions. Therefore, engaging all students at the start of courses about the hows and whys of the approaches to learning that will be utilised will provide at least some shared understanding of the expectations and benefits of getting involved in seminar discussions. 

Finally, taking the opportunity to support students’ confidence in engaging with learning should never be missed. Learning students’ names; encouraging students to interact and learn with each other outside of the classroom; giving students the chance to speak individually (ending teaching sessions a few minutes early but sticking around can give students the chance to ask questions that they might be self-conscious about asking in front of a group) – these are all relatively easy ways to try and give students more confidence in themselves, as well as the learning process. This, in turn, has the potential for increasing engagement and attainment. 

In sum, these above suggestions boil down to one simple but key idea – that we should find space for empathy in our teaching. Certainly, there are a host of other pathways to begin combatting awarding gaps. Whilst sector- and institution-level data is vital in identifying problematic trends, it will take a more granular approach to understand the specific, course-level issues and responses. This is a big issue but it is also one that academics can begin engaging with and combatting by instituting small changes.

Is it in the assessment criteria?

I zoomed into an excellent QAA event this week on implementing racially inclusive practice in assessment, based on a project at University of Leicester, partnering with Birmingham City University and University of Wolverhampton. I’d very much recommend that you have a good look at their report in detail. The take-home for me was that that whilst an inclusive or decolonised curriculum and role models are incredibly important for engagement and for inspiring students, particularly racially minoritised students, if you want to tackle race awarding gaps, the solution is pedagogical.

Their approach is deceptively simple: they focused on making sure that the hidden curriculum is made visible for all students and the tacit is made explicit, that students understand exactly what they have to do to succeed, with no guessing games involved, with clear documentation of what is required, and that all assessment criteria are clearly and transparently explained with examples of what good or less good work against those criteria would look like. One of the staff who had implemented the intervention very disarmingly said that he felt a bit embarrassed that he and his colleagues hadn’t been doing this already! He also said that although there was some initial resistance because of worries about ‘spoonfeeding’, the improvement in the students’ work that he saw and the way they engaged allayed most of those fears. They found that by doing this, they could reduce awarding gaps significantly, improve student enjoyment and confidence, and also improve staff experience of teaching and assessing!

There is a lot to learn from in the report. Personally, I’ve already thought a lot about assessment criteria over the years, in an attempt to be inclusive, yes, but also because I just wanted to communicate with students what I wanted them to do, so they would learn better and I could read better work when assessing. As a less experienced teacher, I realised that I was marking work down for not doing things that I had never taught or told the students to do – which offended my sense of justice. But I knew I did want the students to do those things (such as make a coherent argument, evaluate evidence, use examples, write for an audience, use appropriate referencing), so it got me to thinking about how I might teach those things in the context of a disciplinary and substantive module. I came to the conclusion that having transparent criteria and spending some time making sure that everyone understands them would help me communicate what skills I wanted to see and how they might develop them. It turns out to be a practice that serves all students – not just those who have been previously disadvantaged, but also the ones who keep doing pretty well, but don’t know why.

As we know that tutors are often looking for different things in their students’ work, it usually doesn’t work in a discipline like ours to have generic or departmental criteria. It is an incredibly useful exercise for you, as a tutor, to sit down and write out what it is you are looking for in students’ work. This helps clarify expectations for me and helps me think about what and how I will teach. When team-teaching, working with other tutors to clarify not only what the assessment criteria are but also what they mean in practice is extremely useful for making sure that teaching and marking are fair and consistent. And working with students to help them understand marking criteria doesn’t so much help them ‘tick the right boxes’ in a spoon-feed way, but, much more importantly, understand what skills they are learning and why.

For my current module, the assessment is a portfolio, and the assessment criteria are as follows (although I do allow students to negotiate them, which I won’t dwell on here but will come back to another day):

  • Depth of understanding of how politics and power are shaped by, and shape, the natural world
  • Ability to weave together ideas from the module into your own coherent text
  • Depth and originality of critical evaluation of your own relationship with the natural world
  • Ability to argue for your perspective on how nature should be governed or cared for, by whom and in what ways, including use of reasons and evidence
  • Appropriate selection of multimedia on the portfolio
  • Ability to write appropriately for a particular audience (please specify: eg visitors to an exhibition, policy-makers, everyday readers of narrative non-fiction)
  • Creativity of your work on the portfolio
  • Evidence of learning and development over time in the module
  • Depth of critical engagement with the module materials and readings
  • Extent of additional research and further reading
  • Craft of writing, including readability, spelling and grammar
  • Accuracy of bibliographic materials

I like the approach of starting with a noun plus preposition, like ‘depth of’ or ‘ability to’, because it demonstrates that these are skills one can be better or worse at in a qualitative sense. Thus, this is not a box-ticking exercise for students but rather an invitation to engage in deep and dialogical reflection on what, for example, the ‘ability to argue’ or ‘appropriate selection of multimedia’ really looks like in practice.

It’s very important not to stop with listing the assessment criteria, of course, but rather to make them the centre of an ongoing conversation. Here is my top tip: every time a student asks a question about the assessment, or about what ‘good work’ might look like, I bring it back to the assessment criteria. So, let’s say they ask, ‘does my portfolio need to be consistent week by week?’ I will say, ‘Is that in the assessment criteria? No. So, I won’t be looking for that. If it’s something you want to learn, that is, how to create your own consistent style, that’s great – you can do so and add it to the assessment criteria for your self-assessment. But it’s not necessary from my point-of-view.’

Here is my top tip: every time a student asks a question about the assessment, or about what ‘good work’ might look like, I bring it back to the assessment criteria.

Or let’s say they ask, ‘Can my writing be more personal?’ I will say, ‘Is it in the assessment criteria?’ This is a longer conversation – the answer is, yes, I am asking them to give an account of their relationship with the natural world, so more personal writing in the first person is clearly appropriate. However, if they are using part of their portfolio to write for policy-makers, this can lead to a deeper conversation about what sort of writing, evidence and argument a policy-maker might be interested in. Distinguishing these different crafts of writing and talking about when they are appropriate, or not, is much more useful for learning than just prohibiting one of them without explaining why.

Other ways of getting students to engage deeply with the assessment criteria might include:

  1. Guided marking exercises where students mark examples of work with reference to the assesment criteria. Your aim here is to get them to focus on the criteria and not make the sorts of vague comments (‘this was not well structured’) that they have probably experienced themselves at times.
  2. Peer feedback where the focus is on giving each other feedback according to one or more of the assessment criteria.
  3. Formative feedback from the tutor where they have to tell you which criteria they want feedback on. (I have a form and they can’t have their feedback unless they tell me which criteria they are particularly interested in.)
  4. Self-assessment where students have to tell you how well they met the criteria, and where they could have done better.
  5. Any other discussion with examples of the criteria and what they mean, preferably iteratively, so they can improve over time.

Summative feedback should also, of course, refer constantly and closely to the assessment criteria. But by that point, this is just an exercise in demonstrating that you could be trusted to do what you said you were going to do. To return to the QAA discussion on racially inclusive criteria, the return of summative work should not be an opportunity to say: ‘Ta-DAH! This is what you should have done.’ What the students should have done should be clear right from the get-go, or else how can they learn how to do it?

Tools for Discussion: An Interpersonal Growth Toolkit

Today’s guest post is from Dr. Allison Anoll at Vanderbilt University!

Classroom discussion is a common pedagogical tool, but many instructors and students alike find themselves lost with the lack of structure.  Equally problematic, standard approaches to grading participation that simply count how often students speak can (re)produce racial and gender disparities.

How can we, as instructors, lead effective discussions? How can we help students grow in their interpersonal skills while also ensuring classrooms are inclusive spaces?

In my small, seminar classes I use a tool for structuring and assessing participation I call the Interpersonal Growth Toolkit.This tool relies heavily on a framework developed by Stephen Brookfield in The Skillful Teacher, where he argues that classroom discussion skills need to be taught just like any other learning goal. The Interpersonal Growth Toolkit provides students with learning goals for classroom discussion, tools for achieving these goals, and regular sources of feedback from the instructor about progress.

Here’s how it works. I start by identifying skills I want students to learn and practice in the discussion setting. In my classes, that’s: 1) humility; 2) confidence; and 3) social science thinking. I provide students with a detailed list of tools for how they can practice each of these themes. For instance, to practice humility, students can,

  • Provide credit to classmates for ideas and inspiration by using their name in comments.
  • Ask the group for a moment of silence to slow the pace of discussion and allow you (and others) time to gather your thoughts.
  • Find a way to express appreciation for what is new, interesting, or challenging in the discussion. Be specific about what has helped you understand something new.
  • Make a comment or ask questions that encourages others to elaborate on their ideas.

I then ask my students to identify which of the three areas—humility, confidence, or social science thinking— they are weakest in as a discussion participant.  My students then develop an individualized growth plan by identifying 2-3 skills in their area of weakness they want to try out in class. They write these skills on an index card that I hand back to them at the beginning of each class for the first two weeks. At mid-term, they write a brief reflection on how they think they are doing in their area of growth and what they want to work on for the rest of the semester. In combination with my own notes, I prepare mid-term feedback for each student about their performance in the three areas and tools to try in the second half of the semester. Students receive a final grade for their participation based on their growth and performance in each of the three areas.

This approach increases the quality of classroom discussions immensely. Not only do students and the instructor have a clear sense of what people are supposed to be doing during discussion time, but students themselves can become advocates of inclusivity in the classroom. Students who lean towards dominating a discussion are asked to think about how they can use their skills to draw others in; students who are nervous speaking up are provided with skills for finding their space and ways into a discussion. Using this technique, I have seen students grow immensely over the course of a semester with quieter students getting bolder and more dominant students using their skills to build bridges between other’s ideas. In my seminar courses, it is the norm to hear from every student multiple times over the course of a discussion.

This approach and assessment are well-suited for relatively small classes (less than 20 students). In larger classes, it is more difficult to build a community of trust that fosters vulnerability and to find enough time for all the students to practice their skills. However, this tool is suitable for any level of instruction: I use it with freshman, seniors, and even new graduate students. Instructors should feel empowered to adapt the skills they want students to learn in the discussion as long as they also provide specific tools to complement these skills. Want students to practice intellectual curiosity as a core skill instead of humility? Develop a list of tools that students can practice in a discussion to build this skill. You can look at chapter 8 of Brookfield’s The Skillful Teacher for more tools to consider.

To implement this assessment, remember to include a learning objective in your syllabus that highlights interpersonal growth as a key goal of the course.  I use, “By the end of the course, you will be able to:demonstrate growth in discussion techniques including close listening, speaking with evidence, challenging with respect, and summarizing others’ contributions.’’

Keeping track of student contributions and how they fit into each assessment category can be difficult to do on your own while also leading the discussion. When I use this tool, I hire an advanced undergraduate or graduate student to sit in class with me and keep track of student contributions. I use these notes to develop feedback and grades for my students.

You can find details about The Interpersonal Growth Tool Kit here. The document also includes a rubric. (In general, I’m a big fan of rubrics, but that’s a topic for another day.) Happy discussing!

On rowing and teaching

In my younger days I was a very keen rower. I caught the bug at university, and it took about another 15 years to really shake it off (and, TBH, I don’t think I really have, to this day). Towards the end of my time in the sport I lost a load of weight and switched into coxing – becoming the short shouty guy in the stern of the boat with the mic and the rudder strings – and from there got involved in coaching. And that, come to think of it, was one of my first ventures into teaching.

Now, rowing is a weird sport, in that the ‘training-to-competing’ ratio is hugely skewed towards the former. A quick comparison to rugby helps here (and may reveal some deep-seated jealousy of / animosity towards the egg-chasers). I had friends who played rugby who’d train one or two nights a week and then play a fixture every weekend – and then, true to the stereotype, spend a good few days recovering from the hangover. In contrast, we, at the rowing club, would spend nine months of the year training six days a week, then race frantically for about six weeks, and then have a month off. 

What has this to do with teaching? Well, hear me out…

Teaching is a bit like coaching sports – or rather, for the time being, coaching sport is a bit like teaching. The coach of my friends’ rugby team would be able to run them through a particular drill, or practice a particular tactic, on a Wednesday evening, and then (provided the oafs had remembered said drill / tactic) watch it in action that weekend. A review would ensue, followed by some tinkering in training the following week. The point is: the coach had a regular flow of data from his team, about how well his teaching methods were working, how well the individuals and the team were executing what he had taught them, and how to calibrate all of this in order to get the best of the team.

I, as a rowing coach, had no such luxury. We’d spent months and months training essentially in isolation, with very little contact with opposition (apart from the occasional accidental race against the much-derided Club From The Other Side Of The River), and it would be ages before we could see if the crew were able to hold their own in an actual race.

A while back it struck me that this is quite similar to our teaching-and-assessment model. In my department, and indeed, in many other political science departments around the UK, the pattern is quite similar: we spent eight to ten weeks delivering material and teaching skills, and then, after a briefing and cheery ‘Good luck!’ we send our students off to write their essays / policy briefs / to do whatever it is we ask them to do. Along the way, we have very little opportunity to see how well they’re learning the stuff we’re teaching them; and, for that matter, they have very little opportunity to find out how well the can do whatever skill it is we’re asking them to demonstrate.

(Before I get to the point of my piece, let me clarify something. There’s a conflating of ‘material’ and ‘skills’ here – and it’s deliberate. Back to my rowing analogy: I can explain to a rower, using a whiteboard and some video, how to perform a certain part of the stroke, and why it’s important that it’s done in a certain way, and that would constitute ‘material.’ That rower could understand that ‘material’, but whether or they can do actually do that thing is another matter – and that’s what I mean by ‘skill.’ TBH I’m not sure we’re clear enough with our students about this distinction, and I think we tend to assume our job is just to deliver material / transfer knowledge, and it’s their job to master the skills (subconsciously? From elsewhere? Who knows)).

There are a few fixes I can think of for this pickle. The first is to build in some form of continuous summative assessment, using spaced repetition and the frequent release of grades (and feedback) to help students learn. Although this makes me shiver and reminds me of my schooldays I see the attraction: at least that gives us a much more constant flow of data about performance (sorry, learning), and allows us to calibrate teaching accordingly.

The second is to structure the course around a portfolio of work, which is not assessed during the course, but from which students draw material when compiling their summative submission after the course has finished. This time, the ‘data flow’ comes in the form of on-the-go formative feedback, either from the teacher or from peers (or both, for that matter).

My approach harks shamelessly back to my rowing coaching, and goes like this.

Back in the day, we’d spend weeks and weeks practicing one miniscule aspect of the rowing stroke (think of those films of sprinters performing odd prancing drills in order to perfect their take-off). Then we’d switch and practice some other skill; and then much later we’d put them together, combine them, and have a crack at doing it at speed and under pressure. The point is these ‘miniscule aspects’ were the disaggregated bits and pieces of the finished article: the individual rowing stroke, to be repeated several hundred times during the race; the start, the mid-race push, and the finish; the steering of the tricky bend; and so on. When coaching – sorry, teaching – the crew how to do each element, we’d explain in great detail where it sat in the overall scheme of things, and why it was important that it was done like this and not how The Crew From The Other Club do it, and so on.

And so it is with teaching political science. I take the final thing I want the students to do well at, and I think about all the sub-tasks behind that thing: scraping the literature, presenting and justifying a position, using and critiquing evidence, even taking notes. These are my ‘miniscule aspects’, which I then blend into the design of the seminar activities. We might, then, spend a seminar discussing a particular contention in the literature about a certain topic, where my aim is both to ‘convey material’, and also to give the students a chance to learn, and refine, a particular skill – which in turn will feature in their eventual race (sorry, summative assignment). I make sure to explain how that nugget of material, and the accompanying skill, will be important further down the line.

Thinking about things this way has really helped me confront two interlocking challenges: how to handle a teaching-and-assessing pattern in which the latter is compressed into one instant at the end of term; and how to foreground the assessment without making the course all about the assessment (which, I fear, would just undermine the students’ deep learning of the material). One day I might summon up the nerve to screen a rowing race in the first week (Rio 2016 men’s four?) as I explain all this to the students…

What ChatGPT can never do

It’s a belated summer here in Belfast, where I’m attending the UACES conference. As well as a lot else, we’re running a series of L&T panels, including one on assessment.

The prompt for this was the bruhaha over ChatGPT and Why All Assessment Is Impossible (I paraphrase): while we might understand that the problems lie much more with the assessment design than the scary AI, that’s still not really filtered through to all of our colleagues.

Our discussion was really stimulating, both for its breadth and for its reflection on what we are trying to achieve in our assessment – and, by extension, in our teaching.

A concern that was raised was that if AI can produce more encompassing integration of knowledge than any human, almost instantaneously, does this mean the ‘end of thinking’ for our students? Even if the lurid framing might rile, the idea is not to be dismissed.

For me, this prompted the thought that even in an era of all-knowing technology there is still a clear role for us as teaching, instructors and facilitators: to help emancipate our students by giving them the tools to build their agency in the world.

We frequently talk about building self-reflection and criticality in our students. In an age when the challenge is parsing and navigating through too much information, getting students to make informed choices about what to use and how to use it is essential.

And we do that because it gives them a way of standing more firmly in the world, to achieve what they want to achieve.

We don’t (I think) want to produce reproductions of ourselves, but autonomous individuals who can both define their own purpose in life and find ways to realise that purpose.

In this framing of education-as-emancipation, it becomes irrelevant what AI can do, precisely because it’s AI doing it, not the student. As one colleague noted, almost all our students don’t come to university to cheat, but to learn and to develop themselves.

Put differently, if everyone can just turn to ChatGPT, then what gives you the edge is understanding that that technology can and can’t do and understanding how you can use that to your own purposes.

This points to assessment that valorises reflection and critical engagement with knowledge and with arguments, so that the student is able to apply such tools to other situations. It also suggests that we as instructors have to spend more time on assessment that is grounded in individual experience and that recognises there is as much value in being able to articulate your self-awareness as in nominal achievement of a particular task.

A case in point in our discussion was groupwork. Yes, you can mark how well a group functions together, but we know from our own lives that sometimes we have to work with people who aren’t our optimal partners [cough], so there’s as much value in understanding how to cope with and mitigate that scenario as there is in everything being super-positive.

When we can pick up a device and find out a pretty decent amount about any given subject at the drop of a hat, ‘knowing stuff’ isn’t useful in the way it used to (even if we go to lots of pub quizzes), and we need to recognise that in all of our teaching practice.

Indeed, you might argue this is a great example of being self-aware and self-critical that we should be applying to ourselves, even as we apply it to our students.

The odd situation of assessment

It’s May 2023 here in the UK, which means we are having so many public holidays that it feels like Belgium.

As a distraction from celebrating the arrival of someone else that we didn’t select into the position of Head of State, academics have been more focused on the latest stage of strike action by the main union, the UCU.

Right now, that means a Marking and Assessment Boycott (MAB). This means not returning marks/grades to the university, which in turn means there are gaps in calculating degree classifications for completing students.

Rather than get into the mechanics of all this, I’ll instead focus on the new approach that employers have taken to this.

In the past, there would be a relatively small deduction of pay for participating in a MAB: 10-20% was the typical range, reflecting the amount of time that might be involved, plus a bit for the disruption it caused.

This time around, employers seem to be going for a rather different model:

Continue reading “The odd situation of assessment”

How to debrief a simulation

Being able to see what’s happening in your session is probably also important

One of the topics that popped up at various points at TLC and ISA was the question of debriefing.

Everyone who does active learning and sims work agrees it’s important and there was lots of head-nodding whenever it was mentioned. Yes, it’s essential for reconnecting students’ learning within the activity back into their wider understanding and development, so why wouldn’t you agree?

However, at neither conference did anyone really get into what happens in a debrief.

Part of me nearly jumped straight into ‘I’ll write a blog’ mode, but then sensible me rocked up to say ‘maybe check to see if anyone’s already written something about this first’, which is good advice. Well done, sensible me.

And there’s loads of stuff. Here are some highlights:

Continue reading “How to debrief a simulation”

What to do about grading

This guest post comes from Eric K. Leonard, Shenandoah University

I had the pleasure of spending last week at the annual International Studies Association conference in Montreal.  It was a pleasure because I got to hear so many interesting discussions on pedagogy and honestly, ISA wasn’t always that way.  I attended my first ISA meeting in 1997 and the number of pedagogical panels was limited, to say the least.  

But as the years (actually decades but that makes me sound really old) have progressed, ISA has truly embraced the push by members to create space for pedagogical discussions at the annual meeting. I know for me personally, it has made attendance at this conference a joy. 

Amongthe plethora of pedagogical workshops, roundtable discussions, and panels that I attended one topic seemed to keep emerging – the assessment of students or grading. And within that topic, there was a lot of buzz around the idea of “ungrading”.  

I will admit that I generated some of the buzz since I am a vocal advocate of this method.  But in several discussions, this topic was raised without my prompting, albeit not always with the same enthusiasm and positivity that I hold for ungrading.  One thing was shared by most in the room and that was a disdain for grading. In one session, Simon Usherwood even mentioned that his appointment does not require grading!  Frankly, based on the level of jealousy that erupted from his confession, I was surprised he made it out of the room in one piece. So maybe we all can’t stop grading, but what would it mean to start ungrading? 

First, let me state that I am not a fan of the term ‘ungrading’.  As one of my students pointed out, they still get a grade at the end of the semester, so is it really ungrading? I much prefer self-reflective grading or something less clunky (suggestions are welcome). 

Second, ungrading is not one method of assessment.  It entails a lot of different methods that include specification grading, labor-based grading, minimal grading, authentic assessment, and others. The commonality amongst these methods is that students primarily self-assess through a process of self-reflection.  

The method that I employ is often considered the most extreme form of ungrading because as the instructor, I don’t grade any of my students’ work.  Because I teach in a hybrid format that includes a flipped learning model (maybe a conversation for another post) students submit a written assignment every week of the semester.  However, I don’t put a letter grade on those submissions.  

Instead, I provide students feedback to assist in their improvement of future submissions and to push students in terms of how they think about the week’s topic.  So throughout the semester, students never receive a letter grade.  But my university requires that I input a grade at the end of the semester.  

The way in which I ascertain student grades is through a process of self-assessment.  At three different points in the semester, students complete a self-assessment of their learning process.  This self-assessment includes questions about what they learned, what sparked curiosity, and questions about attendance, the number of assignments missed, if any assignments were late, participation level and quality of their comments during our classroom discussions, the effort a student has put forth, and the overall quality of their work, which can be compared to an “Answer Key” that contains examples of exemplary work from students in the course.  After students consider all of these different variables, they are asked to submit a grade and justify that grade.  

I explain to students that in most instances, I either accept that grade or input a higher grade.  And historically, that is the case.  However, I reserve the right to request a conference with a student if I am struggling to see how that student justified their grade.  In my experience, these conferences are rare (less than 1% of students per semester) and typically conclude with the student gaining a better understanding of the self-assessment both in terms of its criteria and its purpose.  In two years of ungrading, I had one student that refused to rethink their grade after a conference and therefore received a grade that I struggled to justify.  

With that brief accounting of how I employ ungrading, let me remark on a few critiques that typically emerge around this form of assessment.  

The first is that students, especially certain demographic groups, will simply overvalue their performance. In other words, students won’t truly self-reflect on their work and they will just provide themselves with a higher grade.  

I had a similar fear when starting this process but I can honestly say this has not been the case.  As mentioned above, only one student was adamant about their earned grade, despite my pointing out statements in their own self-assessment that didn’t seem to square with the perceived grade.  Because my policy is to let students grade themselves, I allowed that student’s perceived grade to stand.  But as mentioned, this has happened ONCE in two years.  Ballpark number, that is 1 in 250 students.  And a quick aside, that student would not have fit the demographic that the literature tells us would contest their grade. 

Second, as one participant stated during a panel at ISA, ungrading is “fake.”  I think what they meant by this statement is that ungrading does not accomplish what it claims to.  

An incomplete list of ungrading claims is that: 

  • Ungrading relieves the stress of grading and promotes greater learning rather than a focus on the grade.  
  • It encourages greater collaboration rather than competition among the students and between the professor and the students.
  •  It recognizes the subjective nature of traditional grading, especially in essay submissions, and allows students to agency in their learning process.

Given the context of the conversation, I believe the “fake” comment was directed at this final claim.  But what claims like this one fails to accept is that when an instructor grades an essay, research paper, or dissertation, there is a subject component to it and the student has no agency.  Thus, that is also fake.  

If we provide students the opportunity to be part of the discussion on their grades, we might get more thoughtful students that are more interested in learning rather than their GPA.  And although they do not have complete agency over the grading process, since I am creating the self-assessment form and providing an understanding of what exemplary work looks like, the level of agency is much higher than in a traditional assessment format.  I don’t understand why people see this as a dichotomous relationship in that students either have agency or they don’t.  The reality is that any increased level of agency for students in the educational process is a good thing and I believe ungrading serves to fulfill that goal.     

One final thought on ungrading.  And this is probably the most important point I can make in this post. 

Ungrading is not for everyone.  

Just like any other pedagogical method, ungrading has to fit with your teaching style, your approach to higher education, and your students.  Thus, this post is not an attempt to advocate that everyone adopt ungrading and to chastise those that refuse. Instead, it is an attempt to shed light on how one professor implements this assessment method in his classroom.  

You will notice that I didn’t discuss in any detail the literature on why we should stop grading or how it benefits the learning process.  This literature exists (in abundance) and engaging it might persuade some to give ungrading a try, but the question that every instructor has to ask themselves is whether they think this could work for their pedagogical approach and their students.  I answered that question with a resounding yes (and I have the student comments to support this claim)!  But that doesn’t mean your answer is the same.    

Notes from a conference III, or: why you hate grading

It’s probably bad to write three posts, one straight after the other, but hey: tough.

We’ve got more grading/assessment-related posts coming from people more on it than me, but a couple of times in the first days of ISA, we’ve had the classic teaching-related panel moment of everyone nodding along to ‘but everyone hates grading, right?’

[Parenthetically, one of those moments led to talk of how good it would be if our jobs didn’t including grading at all, whereupon I noted my job doesn’t include it, to many jealous looks. I’m mostly happy that I have so little grading to do (like a dozen scripts a year little), but part of me does miss it, mainly for the reasons set out here.]

The obvious question that no one really seems to keen to investigate is why we all seem to hate grading.

After all, it’s the main mechanism we have for evaluating student learning and, by extension, whether our teaching is getting the results we intend. It’s more rigorous than a chat at the end of class about learning gain or the impressions we glean from observing debates in the classroom, so why are we so down on it?

Let’s consider the usual suspects.

First up, ‘it’s repetitive’. Sure, if you have a pile of a hundred essays on the same topic, that is super-dull, especially if there’s a lot of overlap in the intellectual sources your students used (your lectures, the core textbook, the readings you suggested): it’s a rehash/mangling of what you’ve taught them and it’s probably not pretty.

So don’t make that your assessment. Give them choice, make them responsible for finding their own sources, get them to bring something more personal and individual.

My old negotiation course asked students for their only assessment point to tell me what they had learnt from the course: I was emphatic that it could be absolutely anything, as long as it was grounded in what had happened in class and was connected out to whatever literature they had found useful. Result was such a rich variety, every time I ran that course.

Second up, ‘it takes such a long time.’ Partly that’s about the time-elongating/mind-numbing of reading the same stuff a bunch of times that we just discussed, but partly it’s our demands of the process. Often, we normalise the essay of X pages or Y thousand words as proof of learning, even as we rool our eyes at ‘all that text’.

So go for concision, not length. You know it’s harder to produce a good short text than a good long one: the demand to focus much more tightly on what’s important, to lose the blah, is something we find tricky (to judge by the number of ropey abstracts you and I have read). Set a much shorter length piece, or ask for a 2-page policy brief, or a poster, or an infographic, or a 3 minute video.

Of course, shorter assessment outputs doesn’t automatically mean quicker grading, but it helps. As does trying things like videoing your feedback, or asking students to make their own evaluations of their or their peers’ work.

Finally, ‘students find it boring/unpleasant (and so do I)‘. Find me the student who loves to do assessments and I’ll just start wondering what happened to them to make it come to this. I get that everyone thinks it’s an imposition and chore.

So involve your students in finding assessments that engage them more/turn them off less. Amanda’s breakfast exercise for research methods is my go-to here: super-fun, super-engaging allowing them to really capture their experiences in an assessment.

If assessment is about cementing the bond of teacher and learner, then why wouldn’t you want to bring them in to find a mutually-satisfactory assessment regime?

Unless you’re working under a highly prescriptive institutional system, you hold the power to make your assessment and grading less boring. So use it.