Mothering Sunday lunch is apparently not the right time or place to discuss assessment regimes, my mother told me. Such are the perils of taking your work home.
In my defence, the topic had come up with another member of my parents talking about ‘the old days’, when everyone who passed their tests were not given any further specification of their pass (i.e. a degree was a degree) and when other factors in graduates’ profiles were just as important in determining their professional competence. I observed that even here there was some differentiation – the prime function of assessment – since some made the grade and others didn’t, even if those who did didn’t get further differentiated.
This all came back to me yesterday, as I marked my in-class multiple choice test. Those who are still reading this blog after a year will recall that this is open-book and students can talk and work together during the test. As last year, students did a good job in dividing up questions, but struggled with verifying whether answers were correct. And as last year, the mean grade was high (although still not as high as I might have expected).
What have I demonstrated with this test?
Firstly, I’ve checked that students have understood that there is a value in collaborative study and work: their performance as a group was stronger than it would have been if they had tackled it individually. My observation in class was that a couple of students free-rode, but to a much lesser extent than typically happens in classroom discussion.
Secondly, I’ve checked students’ substantive knowledge: the errors they made were telling of some confusions, which I can address in later classes, just as they have been able to start doing themselves, since I’ve provided an annotated answer sheet.
Thirdly, I have still identified some differentiation between students. There were two main clusters in the room, the smaller one performing a bit more stronger than the larger one. But students did also have some individual differences in response, reflecting individual judgement. While this was obviously much smaller than in a classic, individual test, it does still provide both an understanding for me of where students are, and a basis on which students themselves can engage in further development.
It would be fair to say that my test is a source of some discussion, not least by myself, and it is still as much about exam technique as it is about substantive knowledge. However, I feel that this is justified for freshmen students, who often need to work through a transition phase of practice when coming from a school environment. Seen as such, the test becomes diagnostic, as much as formative and summative.
Maybe I’ll bring that up at Eastertime…
Do you think the larger group performed less strongly because of satisficing?
I think it was more an issue of the bigger group spliting into sub-groups to tackle blocks of questions. This seemed to be done on friendship groups, rather than knowledge, so that meant that testing of knowledge wasn’t as methodical as it could have been. The smaller group sat together and all students saw all questions. Thus, it was more procedural than anything else.