A follow-up to my post from last month about changing an exam prompt:
I created two exams for this course with the same two-part design. First, answer some multiple choice questions. Second, write additions to a Twine story.
For the second exam, five out of seventeen students wrote in a style that resembled, to varying degrees, that of the story. While this marked a minor improvement over the first exam, students incorrectly applied economic concepts more frequently. The average score for the second exam was lower than that of the first exam.
While my sample size is far too small to determine whether the change was statistically significant, I would like students to do better, and I’m wondering how I might change the exam prompt yet again to facilitate this.
This sounds like a really frustrating situation. Reading through previous posts it seems like two potential interventions external to the exam prompts might prove fruitful. They both relate to practice (or repetition).
Most often it is not the exam prompt itself we must augment, but where else in the course students have a chance to practice similar types of prompts. For example, do students have an opportunity to practice using Twine stories? Are there any other portions of the course where they are asked to “add to a story?” If students who are already engaging in a high stakes activity like an exam are unaware, unfamiliar, or simply unsure about how to engage with this particular instructional technology, they might be playing it safe by trying to include as much information as possible.
Secondly, the incorrect application of concepts to these stories could be related to the format of other course opportunities to link course concepts to empirical examples. The second section of the test appears to be a really big cognitive ask of students, asking them to reach towards the top of Bloom’s taxonomy in creating something new. If they aren’t able to perform or haven’t practiced classification and application multiple times prior to the exam, they are likely going to be overwhelmed with this task on an exam.
I’ve only got some small insight into your course from these posts but I hope these comments don’t make incorrect assumptions about your course/these prompts. I hope this comment was helpful!
I think “playing it safe by trying to include as much information as possible” definitely applies to some students. Risk aversion is drilled into them in the K-12 years.