Quizzes in the Age of AI: an idea from the Teaching Day at ISA

Today is the ‘Teaching Day at ISA’, a free mini-conference held on the first day of the International Studies Association’s Annual Meeting. We’ve got four great panels going, on Creating Compelling Courses; Assignments in the Age of AI; Active Engagement; and Mentoring Early Career Instructors.

So far, the idea that has most stuck with me comes from Dr. Ronnie Olesker of St. Lawrence University from the AI panel. She’s found an innovative way to do reading quizzes that help students engage with the mistakes commonly made by AI. Reading quizzes have long been used to encourage students to engage with the assigned reading, but AI has made it possible for students to quickly generate a quiz-passing summary without actually reading. Dr. Olesker instead feeds her questions into AI, and then counts the mistakes it makes. The quiz asks students to read the AI response and ‘find the mistakes’–with an A going to students who find the same number (or more) of mistakes that she does.

This is a creative way to continue using reading quizzes to encourage student engagement, but also to teach students about the hallucinations and other issues, such as AI’s tendency to leave out important context or information. The quizzes are closed book and in-class, and because they focus on finding errors, students who rely on AI summaries won’t be able to pass the quizzes.

Looking forward to learning more great ideas at this event!

Reading (again)?

Last October, I asked the online mind hive for help because my Fall 2023 undergraduates were not reading – even though (1) I asked them to, and (2) reading – IMO – is ultimately fun and useful in life … generally. To overcome this problem, I made some small adjustments in the fall, but for the Spring 2024 semester I came out swinging and implemented Perusall in two introductory classes. As we, in the US, are slowly taking the turn towards the beginning of the next fall semester, I am once again here declaring changes to get students to read. Out with Perusall and in with reading groups. I shall explain below.

Continue reading “Reading (again)?”

“The Joy of Asking for Help: Getting students to read (anything?)”

I found myself in an all too well-known situation this week: my students didn’t read the assigned readings. In my opinion, I had set up the most fascinating set of readings to address an important issue in one of my classes. Every should want to gobble that knowledge up, said my hybris. But nada. Maybe a handful had read; the rest of the class became experts at looking straight ahead or down to their screens. I am not rediscovering the wheel with this not-reading problem, but the wheel certainly ran me over this week.  

Credit: Mercy Pilkington (Good E Reader)

I left the classroom after the lecture, wondering how I had created this environment, and how I could pivot away from this mid-way through our semester. Frustrated at myself, but knowing that somehow it had to change, I reached out for help on social media and in real life to people in academia.  I was in awe of the number of helpful responses I received. Although, I did not plan to write about it for ALPS (see Chad’s earlier work on that here), I decided to at least have – for the record – a collection of thoughts and ideas for others, who might find themselves in a similar situation at any point in their academic lives.

  1. Clarify for yourself and the students why we are reading (anything); what the purpose of the readings are; how they aid us in or learning process; and – importantly – do not assume that a one-time explanation covers that. Frequent reminders are helpful and necessary to the learning environment.
    • Here, I also received some online and offline advice about having a session at the beginning to demonstrate how to read articles. Folks have developed different techniques on how to guide students through a sample reading, including developing key questions students should keep in mind when reading (as guiding posts).
  2. I noticed an interesting debate regarding the use of reading quizzes or some sort of grading mechanism regarding doing the readings (or not).
    • I am not in favor of reading quizzes. For one, selfishly I do not want to have more grading work throughout the semester, and I am not sure how effective these quizzes are in motivating the students to read for understanding. And two, I do not want to create this sort of potentially punitive environment in our shared learning space.
  3. Nonetheless, there were some interesting grading mechanisms for readings: 
      1. This includes several recommendations for Perusall, which encourages more a communal reading practice, in which students engage with one another and can annotate readings. I am certainly not sponsored by them, and it also depends on whether your institution has a subscription to the service, but I like the idea.
        1. See similar thoughts on CritiqueIT by Dr. Colin Brown at ALPS.
      • Some faculty structure their entire lesson plan based on students’ reading reactions (required prior to class), focusing on things students did not understand or want to know more about. My planning anxiety stands in the way of this method.
  4. Similarly, the idea of cold calling was brought up. I have fallen to these temptations in the classroom, but at the same time, that does not really solve – for me at least – the reading issue. Then, I am just embarrassing folks in the classroom (if they haven’t done the readings).
    • Folks have suggested to use tools such as Menti, PollEv, or JamBoard (even though the last one is being phased out). They are interactive online boards, that you can project in the classroom. You can pose a question about the reading, and students can (anonymously) respond to that. The collection of the answers as well as the anonymity can overcome social anxiety and the fear of saying “something wrong”.
      • Anecdotally, I already tried a PollEv exercises in one of my classes this week, and I received responses from approx. 2/3 of the class vs. the usual 2-3 hands that shoot up when I start talking about the readings. I did not ask specific questions but rather focused on what stood out to them in the reading or if they had any questions (as the mind-hive suggested). Given my mid-semester pivot, I will stick with that for now.  

The beauty of having a problem with anything in this day and age is that none of us are special enough/unique enough to encounter it for the first time in all of human history. I realized relatively quickly that my problem is not just about “why aren’t they reading” but more importantly “what can I (!) do about this to improve our shared learning space”. And I think that is a better motivator than frustration. Reaching out and asking for help was probably the best way to handle this problem for myself. Aside from the fact that people provided advice/technique, what stood out to me what the fact that there are folks who care and that they care enough to help me out. Thank you!