I recently got to attend the University of Central Florida’s Teaching and Learning with AI Conference in Orlando, and it’s one I highly recommend for anyone interested in practical ideas on how to think about, resist, and use Generative AI in higher education. The presentations are all 25 minutes and the presenters make the most of their short time. I want to share some ideas from one of those presenters, Ashley Evans, professor and Program Chair of Software Development and Cloud Computing at Valencia College. While her field is obviously far from political science, her strategies for transforming assignments from AI-vulnerable to AI-resistant are easily adapted to our field, and worth considering. I want to note that these are my recollections and takes on Evans’ presentation; all ideas here are hers, not mine (excepting the adaptations to polisci). Her website is https://ashley-evans.vercel.app/ and she has told me she’s happy to talk AI with other faculty; contact her by email.
Evans’ stated goals for creating AI resistant assignments include:
Reduce student ability to get high marks from solely AI-generated submissions
Eliminate the ability of AI to complete the task in place of the student
Challenge students to think critically about AI generated output, if used.
Relieve instructors from the role of AI-detectives.
All of these seem like worthy goals to me. Most importantly in my mind, she’s interested in taking the weight off of faculty of trying to root out unauthorized AI use, not by banning it, but by revising assignments so that AI is genuinely not useful on the assignment, and by establishing grading criteria that focuses on completion of tasks that AI cannot do well. This is a tough balancing act, but one that I think faculty need to wrestle with as they figure out how much they want to incorporate AI into their classes.
This guest post is by Dr. Ruairidh J.Brown, the Head of Politics and International Relations at Forward College, where he teaches International Political Theory and International Relations. His research interests include phenomenology and hermeneutics, existentialism, the philosophy of amity and friendship, the philosophy of obligation, and developments in international political theory. Ruairidh was awarded the University of Nottingham’s Lord Dearing Award in 2019 for his pedagogical work in China. You can reach Dr. Brown at ruairidh.brown -at- forward-college.eu.
Editor’s Note (Pigeon, that is): I assess this is a useful and sober addition to the AI in the classroom debate. I’ll admit that I count myself as an AI skeptic, aligning with Ruairidh’s concerns below. Ruairidh’s article highlights the importance of expertise, as he had the necessary training and experience to evaluate ChatGPT’s efficacy in this exercise. Now on to the article!
I emailed my students last week to tell them the EU had plagiarised our Foreign Policy Analysis (FPA) seminar.
I was prompted by the story that European diplomats had used a ‘Starmer chatbot’ – an AI trained to respond like Kier Starmer – to practice and prepare for the ‘Brexit-reset negotiations’.
I was joking (in case you hadn’t guessed), but I was struck by how uncanny the EU preparation strategy was to some of my experimentations with AI this year.
I have been, admittedly, quite the AI sceptic – or perhaps even AI apathetic. A lot if this was due to how the need to adapt to AI has been framed in academia: overtly defensive, negative, and usually focused on the need to prevent students from cheating in coursework and exams. When it was suggested to me as a positive, it was usually in a way I found distasteful. In Political Theory, for example, as a means to summarise the works of Hobbes or Thucydides and thus save students the time and bother of reading the originals – something I found, personally as a political philosopher, blasphemous, and pedagogically, as a teacher, a sure way to doom future students to nothing beyond surface level understanding of the material.
Nevertheless, I have been increasingly curious about utilising AI in another module I teach, FPA. Unlike Political Theory, where a deep, nuanced grasp of philosophical ideas is essential, in FPA, a much more strategic and applied understanding is required. For example, students do not need a deep hermeneutic grasp of the concept of ‘harmony’, exploring its development across Confucian thought, as they would in political theory, but rather a grasp of how and what aspects of the idea influence and are leveraged in Beijing’s approach to other States.
Scenario role plays are thus a perfect tool for FPA as they train students not to consider ideas in abstractum but as part of a strategy. I had, in addition, been keen to introduce some digital gaming models for FPA to make these classes more dynamic, and had been inspired by articles about using such games as Diplomacy.However, I found Diplomacy interface too clunky and a game far too complex to set up.
I thus decided to embrace AI. Afterall, I thought, surely the games I loved as a youngster, like Total War used some kind of AI for their non-player characters (NPCs)? Maybe I could embrace ChatGPT as an NPC for my seminar and see what it could bring to class.
I subsequently created two classes with ChatGPT involved. The first was on North Korea and nuclear weapons. The class was split into three groups of ‘US ambassadors’ who had to try and convince Pyongyang to abandon its nuclear program. North Korea was played by ChatGPT, and whom students would subsequently have to negotiate with.
The second was on economic diplomacy, with the case study of China’s relations with Central and Eastern Europe (ECC). One group would play China and would need to get as many ECC countries to join a new fictitious economic alliance as they could. If China could get, and keep, a majority by the end of the class, they won. ChatGPT played the role of a ‘random events generator’ with the face of Sky News, announcing events such as corruption scandals, civil unrest, and responses from the US in the form of social media posts by President Trump as he reacted to student actions.
These classes, especially the second one, exceeded my expectations.
Firstly, from a technical standpoint, it was incredibly easy to set up and offered incredible flexibility in design. I told ChatGPT what I wanted and how it was to be done, which it not only understood and executed exceedingly well, but even came back which greatly enhanced the class. It was a fraction of the time and effort spent working out Diplomacy.
Secondly, it provided a solid anchor to my seminars. As anyone who has used role plays in classes will know, some student roles require more heavy lifting than others. If a student allocated this role has a certain ‘out-the-box’ perspective, or simply does not do the prework, it can sink the whole class before it sets sail. You can, of course, mitigate this by giving such responsibilities to the same ‘reliable’ student, but this becomes obvious and can create resentment. By giving a key role, like Pyongyang, to an AI however, it allows you to create a reliable anchor and control mechanism on the class and keep it on course. I, for instance, asked it to be tough, which quickly resulted in students who were more unsure or had done less prework finding themselves on the backfoot to an ‘assertive’ North Korea.
Thirdly, it gave the classes more dynamism and kept them alive. We have all designed role plays where the issue is solved, or students get bored, in five minutes. The ‘Sky News’ interventions however kept the scenario in rapid flux, teaching ‘Beijing’ it harder to hold an alliance together than make it and give students in the process a more ‘realistic’ experience of constantly adapting to a ‘live situation’.
However, what the unexpected revelation was for me was the Trump interventions. ChatGPT did this so well, you would have thought it really was Trump watching and responding to our seminars, adding an incredible level of immersion (Trump, of course, works well for such tasks, as there is so much of his rhetoric online for the AI to build a convincing persona on). Indeed, so ‘Trump-like’ was it, students would respond and react to the AI. This resulted in subconflicts developing with the US, which added another level of complexity to the main game. The level of complexity, dynamism, and immersion (as well as, importantly, fun) that resulted from this was remarkable to me.
In short, therefore, inviting AI to my class was much easier than I imagined; gave a firm anchoring and control mechanism; and provided a dynamism and immersion beyond what I expected. Responding to a live situation also gave a sense of stakes and realism to the class. That EU negotiators are also using AI to game plan real negotiations was just an added cherry of realism on top, one I could not but help pointing out to my students.
The first in maybe a few brief reflections on the recently completed semester and my twenty-fifth year of teaching as a full-time professor:
I ran a simple anonymous online survey of < 50 undergraduates on their use of AI. About two-thirds of the survey respondents indicated that they use AI only rarely if at all.
I have used AI on a recent assignment in this course.
Frequency with which I use AI for exams/assignments in my university courses.
I’ve come up with four possible explanations for my results:
Students didn’t answer truthfully even though the survey was online and anonymous.
Students don’t consider many of the tools they are using (e.g., Grammarly) to be AI.
My sample is very unrepresentative, even though the responses came from students in English, political science, psychology, and communications courses.
Students aren’t using AI nearly as much as we think they are.
“I am a big believer in not letting robots dream for us”, Nicolas Cage.
“I want AI to do my laundry and dishes so that I can do art and writing, not for AI to do my art and writing so that I can do my laundry and dishes”, Joanna Maciejewska.
“The days of spending long hours reading through extensive academic content are over.” hix.ai.
In my early days online, there was a small blog that I loved. Every week or so, the blogger would share a new film review. The review would read as entirely sincere and serious but there was a catch – the review was only based on the film’s poster. The blogger would take a look at the poster and then make sometimes reasonable, sometimes wild assumptions about what the film would be about and how the whole thing would land. It was a funny read (and unfortunately one that apparently is long gone, I couldn’t find it to share now).
Those blog posts have been on my mind recently as those vague approximations, lucky guesses and massive misfires are characteristic of a more contemporary and less often amusing part of our online architecture – Generative AI.
GenAI is now seemingly embedded into everything (why is that when I go onto the search function on Whatsapp, I am now met with suggested Meta AI search terms ‘I want to write a birthday wish’, ‘I want to write an email’, ‘I need help with my school project’? Who was asking for this?). That ‘everything’ includes many aspects of contemporary higher education. This is a phenomenon that I feel genuinely uneasy about and it’s one where I think we need to be much more willing to speak with our students about.
Today is the ‘Teaching Day at ISA’, a free mini-conference held on the first day of the International Studies Association’s Annual Meeting. We’ve got four great panels going, on Creating Compelling Courses; Assignments in the Age of AI; Active Engagement; and Mentoring Early Career Instructors.
So far, the idea that has most stuck with me comes from Dr. Ronnie Olesker of St. Lawrence University from the AI panel. She’s found an innovative way to do reading quizzes that help students engage with the mistakes commonly made by AI. Reading quizzes have long been used to encourage students to engage with the assigned reading, but AI has made it possible for students to quickly generate a quiz-passing summary without actually reading. Dr. Olesker instead feeds her questions into AI, and then counts the mistakes it makes. The quiz asks students to read the AI response and ‘find the mistakes’–with an A going to students who find the same number (or more) of mistakes that she does.
This is a creative way to continue using reading quizzes to encourage student engagement, but also to teach students about the hallucinations and other issues, such as AI’s tendency to leave out important context or information. The quizzes are closed book and in-class, and because they focus on finding errors, students who rely on AI summaries won’t be able to pass the quizzes.
Looking forward to learning more great ideas at this event!
Since my wife is spending the holidays in a military dictatorship, I’ve had plenty of time to work on my spring term courses, and I’ve made some radical changes, partly as a reaction to the growing ubiquity of AI:
A tweak to the in-class writing described in the linked post above: I won’t collect students’ notes. I’m not grading them, so why bother?
Every two weeks, students will take a pen-and-paper exam with multiple choice questions on information contained in the assigned readings on Perusall.
Although teammate evaluations will continue, I’ve turned project-related tasks like classroom presentations into ungraded activities.
These changes mark a shift away from my previous philosophy about student motivation, grading, and learning. Early in my career, I encountered students who refused to do anything that wasn’t graded — in reality, a rational response on their part to an educational system that prioritized grades over learning. My reaction? Formally assess everything that students did, to convey the message that everything was important for their learning.
The end result was an assessment regime with assignments worth in total well over 1,000 points but in which students needed only 950 points for an A. Again, I assumed this was a good thing — my intent was to give students opportunities to safely practice and improve, rather than to punish anyone who demonstrated less than constant perfection.
However, this system devolved over time into a treadmill of busywork that stripped students of responsibility over their own learning and didn’t reflect their actual performance. My grade distributions indicate that any student in one of my courses who puts in enough time and effort gets a A, everyone else gets an F. The B’s and C’s that used to be common aren’t there anymore.
So, in sum, I’m no longer going to grade a lot of the stuff that happens in the courses that I teach. Students will have greater freedom to choose what to do and not do, and to experience the consequences. Perhaps it’s a small step toward ungrading.
The fall semester is rapidly coming to a close, and, as usual, I’m already thinking about what lessons I can draw from it. Top of the list: the need to re-engineer how I teach because of AI. I get the sense that nearly all of my undergraduate students now use AI in some fashion. In many cases, they are so atrociously bad at it that, to quote former U.S. Supreme Court Justice Potter Stewart, I know it when I see it. But it’s not something I can prove in a court of law, and I don’t want to waste my time and effort trying to do so. So at minimum I’m left with altering the cost-benefit ratio for students when they blindly outsource tasks to AI.
Perusall has features that make this less convenient for students:
Settings > Advanced > Library and assignments > Do not allow students to copy from course materials — When checked, students can’t copy text from assigned documents into the clipboard.
Settings > Advanced > Library and assignments > Allow students to download all uploaded course materials — When unchecked, students can’t download assigned documents that were uploaded by the instructor.
Settings > Advanced > Commenting > Do not allow students to copy from and paste into comments — When checked, students can’t copy text from comments made by other people onto the clipboard, nor can they paste text from the clipboard when writing their own comments.
Of more consequence to my classroom praxis, I’ve thought of a new technique that might obviate the perceived advantages of AI usage while solving a far older problem — students not taking notes. Here it is: on a weekly or even daily basis, students clear their desks and handwrite notes about what they read before class or about information presented during class. I collect the notes when the prescribed amount of time has elapsed but do not grade them. Every few weeks I return students’ notes for revision in class. I then collect the revised notes, and the process repeats itself. At the end of the semester, I return students’ notes one last time so that students can use them as an aid for an in-class paper-based multiple choice or essay exam. I inform students of this system at the beginning of the course. Those who choose to complete assignments without AI will learn even more because of memory retrieval during the note-writing process, which should be reflected by their performance on the final exam. Students who rely on AI throughout the semester will suffer the consequences.
I wrote an op-ed about teaching in the age of AI that was published yesterday by The Chronicle of Higher Education. If you are interested in the topic but your university’s library doesn’t subscribe to The Chronicle, you can access the essay by creating a free account at the linked page.
2024 has been labelled a ‘historic year for democracy’ as around 1.5 billion people go to the polls in elections in more than 50 countries. Whilst the US election is likely consuming a lot of our attention and the bulk of newspaper space, between now and 5th November there are also elections in Austria, Czechia, Sri Lanka, Georgia, Mozambique, Uruguay, Jordan, Tunisia, and Botswana.
At the same time as this boom year for elections, there is growing concern about how misinformation, accelerated by the use and capabilities of generative AI, might be impacting political engagement. The risk of generative AI disrupting voters’ behaviour is something that is already being seen – in January AI-generated messages, claiming to be from Joe Biden, urged voters in New Hampshire to not vote in the state’s primary election for the Democratic Party.
Below is another example of generative AI’s rapidly advancing capabilities, via Anthropic’s Claude Sonnet 3.5. The comparison of the scholarship is accurate, but I have changed the authors’ names, so if you ever see an essay citing the Three Stooges as China experts, you know that AI wrote it.
Prompt
Write a 1,000-word argumentative essay that answers these questions: “What are the domestic sources of Chinese foreign policy?” “What advantages and constraints do they pose for Chinese foreign policy?” “Why?” Support the essay’s argument with examples from publications by the China scholars Larry, Moe, Curly, Shemp, and Joe.
Response
China’s foreign policy, like that of any major power, is deeply rooted in its domestic political, economic, and social realities. This essay argues that the primary domestic sources of Chinese foreign policy are: (1) the Chinese Communist Party’s (CCP) imperative to maintain political legitimacy, (2) the need for sustained economic growth, and (3) the management of nationalist sentiment. While these factors provide certain advantages for China’s international engagement, they also impose significant constraints. Understanding these domestic drivers is crucial for comprehending the complexities and sometimes contradictory nature of Chinese foreign policy.
The CCP’s quest for political legitimacy is perhaps the most fundamental driver of Chinese foreign policy. As Larry has argued, the Party has based its right to rule on a concept he terms “authoritarian resilience.” This involves a combination of performance-based legitimacy (primarily economic growth) and institutional adaptations that allow the regime to be responsive to public opinion without fundamentally altering its authoritarian nature. This dual foundation profoundly influences China’s international behavior.
On one hand, it pushes China to pursue a largely cooperative foreign policy that facilitates economic growth through trade, investment, and technology transfer. Moe has noted that this economic imperative has been a primary driver of China’s integration into the global economy and its generally status quo-oriented approach to international institutions. On the other hand, it sometimes necessitates a more assertive stance on issues of national pride and sovereignty to satisfy nationalist sentiments and demonstrate the Party’s role as defender of Chinese interests.
The advantage of this legitimacy-based approach is that it allows China to present a mostly non-threatening image to the world while steadily increasing its global influence. By emphasizing “win-win cooperation” and “peaceful development,” China has been able to expand its economic ties and soft power without triggering the kind of balancing coalitions that rising powers have historically faced.
However, this same focus on legitimacy also constrains Chinese foreign policy. Curly has argued that the CCP’s reliance on performance legitimacy creates a constant pressure to deliver economic results, limiting China’s ability to take actions that might jeopardize its economic relationships, even when such actions might otherwise align with its strategic interests. For instance, despite tensions with the United States, China has been cautious about using its holdings of U.S. Treasury bonds as a weapon, recognizing that doing so could harm its own economy.
Moreover, the nationalist component of the CCP’s legitimacy creates pressure for tough stances on territorial disputes and issues of national pride. This can lead to counterproductive actions that damage China’s international image and relationships. The aggressive assertion of maritime claims in the South China Sea, for example, has alarmed neighboring countries and contributed to closer security ties between these nations and the United States.
The imperative of economic growth is another crucial domestic source of Chinese foreign policy. China’s rapid development has been central to the CCP’s performance-based legitimacy, but maintaining high growth rates has become increasingly challenging as the economy matures. This economic imperative drives many aspects of China’s international behavior, from its Belt and Road Initiative (BRI) to its push for a larger role in global economic governance.
The advantage of this growth-oriented foreign policy is that it aligns China’s interests with global economic stability and development. China has become a major advocate for globalization and free trade, positions that have gained it supporters among both developed and developing nations. The BRI, in particular, has allowed China to export its excess industrial capacity while gaining political influence through infrastructure development abroad.
However, the focus on economic growth also constrains China’s options. The need to secure resources and markets makes China vulnerable to economic coercion and limits its ability to use economic leverage against other countries. Furthermore, as Chinese companies expand globally, Beijing is increasingly forced to engage with issues it had previously avoided, such as political instability in Africa or the Middle East.
Shemp and Joe have explored how the CCP has attempted to learn from other authoritarian regimes, particularly in East Asia, to maintain its rule. This learning process has influenced not only domestic governance but also foreign policy. For instance, China’s emphasis on economic performance and its efforts to co-opt potential opposition groups mirror strategies used by other authoritarian regimes in the region. In terms of foreign policy, this has translated into a focus on economic diplomacy and a resistance to Western pressure for political liberalization.
The management of nationalist sentiment is the third major domestic source of Chinese foreign policy. The CCP has actively cultivated nationalism as a unifying force and source of legitimacy, particularly through patriotic education campaigns and control of media narratives. This nationalist sentiment provides the government with a powerful tool to mobilize public support for its foreign policy objectives.
The advantage of this nationalist underpinning is that it allows the Chinese government to take strong stances on issues it deems crucial to national interests, such as Taiwan or territorial disputes. Public opinion can be leveraged to pressure other countries or to resist international criticism of China’s actions.
However, nationalism is a double-edged sword that can significantly constrain foreign policy options. Once nationalist sentiments are aroused, it becomes difficult for the government to make compromises or back down without risking public backlash. This dynamic has been evident in China’s relations with Japan, where historical animosities and territorial disputes have repeatedly flared up, damaging economic ties that are crucial for both countries.
Larry has pointed out that while the CCP has been successful in fostering nationalism, it also fears that nationalist sentiments could turn against the regime if it is perceived as weak in defending national interests. This creates a delicate balancing act in foreign policy, where the government must appear strong without escalating conflicts to a point that could harm economic interests or international relationships.
In conclusion, the domestic sources of Chinese foreign policy – the CCP’s legitimacy imperatives, the need for economic growth, and the management of nationalism – create a complex interplay of advantages and constraints. They allow China to pursue a largely cooperative international strategy focused on economic engagement, while simultaneously driving more assertive behavior on issues of sovereignty and national pride. Understanding these domestic drivers is crucial for interpreting China’s sometimes seemingly contradictory actions on the global stage.
Looking ahead, the tension between these different domestic imperatives is likely to become more pronounced as China faces slowing economic growth and increasing international pushback against its rising power. Curly has argued that the CCP’s current model of governance may not be sustainable in the long term, which could have profound implications for China’s foreign policy. How the CCP navigates these challenges will have significant consequences not only for China’s domestic stability but also for the future of the international order.
