A small communication/adaptability exercise

Some Lego, recently. And yes, I do know the worktop needs varnishing again

My neighbour (and fomer colleague) Roberta came by the other day to borrow my Lego. I’d love to say this kind of thing happens regularly, but it really doesn’t.

She had recalled an activity I’d done with various groups some years ago, to teach them about the importance of communication

The game is super simple (if you (or a neighbour) have the Lego to hand and involves trying to recreate a model, albeit with some restrictions.

Full details here.

For me, the value of the exercise is about getting students to think some more about how well they prepare for activities and how well they can make running adjustments should things turn out to be quite so simple as they thought. Which is of – hopefully – general application, not just for classrooms.

It’s also a nice ice-breaker, especially if you have multiple teams up against each other.

As I say, this isn’t a new exercise, but somehow I never really wrote it up fully. Perhaps I need to go back through my playbook and check there aren’t other things I could be sharing with you.

But do remember, you’ll have to wait for Roberta to return the Lego before you can ask for it.

The robots are coming! And they’re… writing essays!!

It’s a truism that no academic is actually interested in assessment. Sure, there are certainly academics who find assessment stimulating and and engaging as a topic, but none of us has ever met one.

Except you have: me.

I never really understood the antipathy towards assessing: maybe it’s a carryover from being a student, where being tested felt, in very large part, like being punished. And I’m not going to pretend that I really liked sitting down to go through scripts.

One big exception to that was the reflective piece that my negotiation students wrote about their work, exploring and expanding on what they had taken from the course and contextualising it in the wider literature. Every one was a fascinating insight into my students’ heads, in a way that 4,000 words on ‘IR is overly fixated on power’ never is.

Any way. The only point were colleagues do seem to get more interested is when they have a problem with their assessment.

Right now, that problem is ChatGPT.

If you’ve somehow missed this one, this is an AI text generator, capable of creating extensive – and seemingly well-written – responses to minimal prompts. Including to things like essay titles.

To say that colleagues have been concerned is very much an understatement.

Both online and in-person, I’ve seen colleagues describe the system as the death knell of the essay format in assessment. Text is both of a standard that it could satisfy criteria to get a passing grade and that isn’t going to trigger anti-plagiarism software (since it’s not cut-and-paste, but organically created).

For all the rumours that this latter software will become upgraded to pick up on such AI-generated text, the feeling is deeply pessimistic.

My own view is perhaps more measured, mainly because with all the examples I’ve seen I haven’t felt the output is that impressive, especially for any one looking to bypass they way to the kind of higher grade that so many students seek out.

As others have noted, the quality and rigour of such texts isn’t up to much, which means both that it’s possible to pick up on AI generation (even if evidencing a plagiarism case is still a massive pain in the neck) and that students get a dubious amount of return (in grade terms).

But the bigger point is that text is only part of how assessment works.

The choice of questions you ask and the requirements you impose on students also matter massively.

Take that negotiation class I mentioned. Because I knew all the students, saw all they did in class and debriefing them extensively at the time about their learning, I had a very clear idea of what might be in their reflective pieces.

So if someone tried to write about stuff they hadn’t done (and a couple tried), I knew and could mark accordingly.

More generally, this all should be making us think more carefully about what assessment is for. And part of that is acknowledging that the very large majority of students don’t want to cheat on their education: sure it’s less effort (they think), but it screws when they are out in the world, trying to use skills or understanding they don’t actually have.

So I leave you with this example of how we can get students to engage with these challenges. It’s not a whole solution, but it is a recognition that blind panic or utter despair aren’t helpful responses.

We’ll be coming back to this several more times, no doubt, but I’d love to hear your thoughts on the matter.

New Journal – Call for Papers

An announcement:

The School for International Training (SIT) is debuting an academic journal for the publication of research on the world’s most critical global issues.

The new Journal of Critical Global Issues, a peer-reviewed, open-access digital journal, will contribute to SIT’s mission to educate future scholars and professionals to address critical issues in pursuit of a more sustainable, peaceful, and just world. The journal aspires to support respectful communities, foster intercultural understanding, advocate for social justice and inclusion, and promote sustainability.

The Journal of Critical Global Issues invites proposals from researchers and scholars to contribute to an online roundtable discussion in May focused on the following areas: climate and the environment; development and inequality; education and social change; geopolitics and power; global health and well-being; identity and human resilience; and peace and justice. Roundtable presenters will have the opportunity to publish work related to their roundtable presentation in the inaugural issue of Journal of Critical Global Issues. We seek contributions from diverse theoretical and methodological perspectives to join us for this event.

Event information:

Location: Virtual
When: May 15-17, 2023
To submit a proposal for a roundtable discussion, please submit a 500-word abstract of your presentation here by February 15.

Questions? Contact university.relations@sit.edu.

Return to the source

Is that… America I can see? No. No it’s not.

It’s the new year, which also means it’s that time when several of us go “I really do need to write that TLC paper”.

[cough]

Yes, next month we have APSA’s Teaching and Learning Conference happening in the fine city of Baltimore (better than it’s reputation, I’m reliably told and who am I to argue). TLC is still the premier L&T event for polisci in North America, and is also the place where the people who give you this blog first met up.

Since those discussions back in Albuquerque a dozen years ago, a lot has changed: most of us have changed institutions and all of us have moved our practice on considerably. Indeed, it’s possible that only Victor is as he was, perennially on the verge of writing a post for us 😉

As a case in point, I’ll be attending this year in a number of capacities.

Firstly, I’ve a paper on how I’ve designed the asynchronous online simulation that I’ve mentioned here before: we’re still in the testing stage, but it’s looking good and my yes-still-to-be-written paper will talk through both the specifics and some more general theorising about how to go about such a thing. Links will follow.

Secondly, I’m part of the Journal of Political Science Education workshop on how to publish on L&T. We’ll be running a couple of sessions of this during TLC, so if you’re about do attend: as part of the editorial team and the one with responsibility for simulations submissions, I’m really keen to talk about how we can get the most out of your work for others. My own paper will probably pop up as an example of what I’m talking about, although whether as a good or a bad example remains to be seen.

Finally, I’m there generally as an ALPS blog person. This role is certainly the most enjoyable, not least because it reminds me of the most thing-that-could-be-in-a-film moment of my life.

Our blog has endured because of our excellent readers and contributors: without all you guys then what we do would be just so much hot air. So thank you all.

Chad and I have helmed this ship since the start and we’re both starting to think about the next stages of our glittering careers, so if you’d like to buttonhole me for a chat about becoming a more central and active part of the team here, then I’m all ears.

As in the past, I’ll be trying to live-blog from the event next month, but in the meantime I have a paper to write and a sudden reminder that I need to check out what Kim Kardashian is up to these days.

How Do I Get An “A”?

Last summer, when building LMS sites for my fall semester undergraduate courses, I inserted a link titled “How do I get an ‘A’,” assuming it would get students’ attention. The link was to this short video about the importance of deadlines.*

I decided to expand on this idea for the spring semester and beyond, with an LMS page that contains the link to the video and this advice:

  • The due date is not the do date. Instructions and deadlines for all course assignments are available in the syllabus and on Canvas from the beginning of the semester. Plan ahead and complete assignments several days before they are due.
  • See the syllabus for the location of reading assignments. Ask librarians how to access these materials at no cost. There are computers available for this in the library and at other campus locations.
  • Revise your writing to eliminate as many unnecessary words as possible. Bad writing is an indication of sloppy thinking. If you are not familiar with the revision process, use the Writing Center.
  • Read the feedback on the quality of your work that is contained in assignment rubrics and my comments. It is not possible for me to care more about your learning than you do.
  • Sleep, eat, and exercise. Sufficient quantities of each are necessary for learning.

While the above can be construed as facilitating more learned helplessness among students, I’m finding that my syllabus quizzes just aren’t doing the job of communicating some of the most basic academic aspects of being a college student.

*Courtesy of TikTok via Reddit. Not something I created.

The Death of Curiosity? Part 3

A final review of the previous semester, this time on my course about environmental politics and economic development. I tweak the design and content of this course every year, probably because it’s my favorite topic to teach (some prior examples of this here and here).

As in the other undergraduate course that I taught, I administered my own course evaluation. Sample is 18 out of 22 students. Here are the results for the questions with a 5-point scale of “strongly agree” to “strongly disagree”:

  • I now have a better understanding of the causes of poverty and economic growth: 4.3
  • I now have a better understanding of the relationship between economic development, environmental change, and risk: 4.4
  • The game design project helped me learn about environmental vulnerability and risk analysis: 3.6
  • I am now better able to use risk analysis as a decision making tool in my own life: 4.0
  • More courses at this university should include training in skills like risk analysis: 4.2

The relatively low score for the third question matched my observations. As in previous iterations of the course, teams of students designed games. This year I specified that the games needed to teach players about the environmental vulnerabilities faced by business owners. I devoted portions of some classes to presentations about system design and failure, and there were many writing assignments about the relationships between economic development, climate change, and risk. Yet, as in prior years, the games students built had little relevance to the design objective. In terms of mechanics, they mainly resembled Monopoly or Life.

I’m taking this as a sign that I need to impose even more limitations on the creativity students can but don’t exercise on this project. Next year I’m going to require that the games:

  • Be played on a board that is a map of the local community.
  • Have player roles that focus on a specific industry or institution threatened by climate change — such as tourism, food, or housing.
  • Contain mechanics that take into account the system components of place, people, and processes.

The good news is that I was completely surprised by answers to the evaluation’s “My favorite reading in the course?” Eleven of the respondents named the novel How to Get Filthy Rich In a Rising Asia, by Mohsin Hamid. Comments about the book included:

  • Clearly written and entertaining.
  • Nice to be able to connect with a character throughout the story.
  • Explained the timeline of a developing country through a perspective that I could visualize.
  • Unique and thought provoking.

I’ll definitely be including this novel in the course next year.

Keeping your online presence

A few weeks ago, I wrote about moving from Twitter to Mastodon, mainly because the former looked like it might fall over.

While that’s not happened (yet), it has still encouraged me to consider my digital footprint more generally.

It is both the triumph and the tragedy of the emergent technology that is the internet that how we do things is not fixed. The rise (and fall) of social media platforms, the coming and going to various formats, all contribute to needing to avoid falling back on just doing the same old.

Blogging is one area that’s been on a somewhat longer cycle: it’s taken a long decade for the burst of creativity and energy that was partly responsible for us here at ALPS Blog setting ourselves up to start to require serious attention.

Right now, blogging seems to be in an odd place, especially as RSS feeds have turned into more actively push technologies that take new content into your inbox. RSS was never that complicated, but it was both fiddly enough to stop mass uptake and logical enough to stymie innovation for a long while.

But looking around now, it seems like the big blog-ish efforts are going into platforms like Substack, even if WordPress remains a massive presence.

Of course, just because other people are doing something, doesn’t mean you have to too. As my mother likes to tell me.

However, it’s also good to try out new options, so I’ve been setting up a Substack to see what’s what.

As you’d expect, it’s very quick and easy to set up an account and start subscribing to other people’s work, pushed out to you.

It’s also been very quick to create a mirror of my other blog – OUatEU – and even import the podcast I’ve been doing on the EU – A Diet of Brussels. Creating a subscriber button was a piece of cake too.

Right now, I’m still sticking to the original platforms to post, then pulling it through, as and when. Partly that’s about retaining control of content and partly it’s about keeping historic links to content in place.

As for whether it’s worth it, I’m not sure – we’re still far too early in the process to tell. However, the range of tools in Substack is different from those on WordPress (or Audible), that opens up new possibilities, just as soon as I find the time to try them out.

In the meantime, ALPS Blog is sticking to being right here, but if you’d like to share your experiences of other platforms you’re always welcome to share them here.

The Death of Curiosity? Part 2

Continuing to review my fall semester . . .

The forecasting project might have helped students learn Middle East politics and history. I’d rate it as a success on that front. As to whether their decision making skills have improved from using the CHAMP method, who knows?

At five different points in the semester, students forecasted the likelihood of these events occurring by December 9:

  • The value of the Turkish lira against the U.S. dollar decreases to less than 22:1.
  • In Iran, the Assembly of Experts names a new Supreme Leader.
  • An anti-government protest in Cairo results in at least twenty demonstrators arrested, injured, and/or killed.
  • The president or prime minister of Lebanon is assassinated.
  • Turkey ends its occupation of Syrian territory.

None of these events happened before the deadline, but that was ok given my purposes for the project. Here are the class’s predictions, with average percentage probability on the y-axis:

I need to tweak some of the project’s components. For example, the prompt for the last individual assignment — assess how your forecasts have been affected by cognitive biases — included this statement:

“People like Daniel Kahneman, Charles Wheelan, Tim Harford, Gerd Gigerenzer, and Nassim Taleb have written about cognitive biases and how to counter their effects.”

A few students did not discuss cognitive biases at all. Others clearly did a bad job of Googling “cognitive biases” and what the above individuals have written about them. In the future I’ll need to assign a specific reading on the topic. I see this as another manifestation of student inability or unwillingness to find information that I don’t put right in front of them.

Similarly, I either need to discard the in-class team presentations or formally assess them. Overall, they were of poor quality. Students need an explicit, rigid template for constructing presentations, and students will follow the template only if the presentations are graded. Asking students to give informal, ungraded presentations simply doesn’t work. Given that this country has raised a generation of children who frequently suffer from anxiety disorders, I might need to institute a rule that credit for presentations only goes to the students who deliver them, with the condition that each member of a team can present if they so choose. I already design my courses to provide students with “multiple paths to success,” so optional-yet-graded presentations are not much of a complication for me.

I administered my own course evaluation at the end of the semester. Here are the results — from 20 out a class of 22 students — for questions with a scale from “strongly agree” (5) to “strongly disagree” (1):

  • The forecasting project improved my ability to analyze political events in the Middle East – 3.9
  • I am now better able to use forecasting as a decision making tool in my own life – 3.7
  • More courses should include training in decision making skills like forecasting – 3.4

I would like the average scores on the second and third items to be higher.

Final comment: the last two reading response assignments before the final exam asked students to respond to “Will Lebanon/Syria still be a single sovereign state in 2030?” I did not realize until the last week of classes that these questions dovetail perfectly with the forecasting project, and that I should somehow integrate the CHAMP method and reading responses so that students get more opportunities to hone their decision making skills.

It’s more complicated than that…

Surveys regularly pop into my inbox, much as I’m sure they do into yours.

I like to complete them whenever possible, partly because well-educated, white men in affluent regions would probably not get represented otherwise*, partly because I like finding out what other people are interested in finding out about.

So when I got a survey about the ‘state of poli sci’ the other day, I was all up for it. The author is someone who I know to do good work and in previous years I recall being asked about things that would be very useful to know about. No names, no pack drill.

And most of the survey was really good, and made me think about things more (always a good sign).

Then I got to a section about the use of virtual formats. I’ve screenshot some of those at the top of this post: you could slide the response scale to one of five points in each.

These feel, well, like less than helpful questions.

Let’s take Q27, for instance. I have lots of preferences in my teaching, for lots of different things, depending on what I’m trying to achieve. Plus I work in an institution that has policy about whether and when I use in-person or virtual formats: you might too.

So even if I could bundle up all my preferences, that’s unlikely to be the determining factor in what I use. Sure, I love doing in-person stuff, but for my current job it would be senseless to work to that, given my student population and profile.

It’s similar for the conference questions: there’s a lot of different things going on with events – in which the national-ness/international-ness is possibly the least of it – that mean your format depends on particular circumstances. What’s good for building up a bid for a specific funding call is probably not good for a one-off presentation or a big multi-day general conference.

On top of this, hybrid formats are not really a half-way house between in-person and virtual, but something different again, as anyone who’s been forced to teach that way can tell you.

Of course, I can see some of the intention of what’s being asked here – I managed to put something down – but all of this confuses abstracted views on format with the practice of teaching (and event organisation).

Central in this is – as ever – the combination of your learning objectives and the constraints under which you operate. I – like you – decide on what I’m trying to do, then work out how to fit that into whatever constraints I have, and use that to design a way of doing it. It’s very rare in teaching environments that those first two steps leave the option of choosing a format of the kind asked about here: even for events there’s often something that limits whether you go on- or off-line.

None of this is to dunk on the survey author, which is why I’ve not named them: the point here is more one of asking you to reflect more on how we understand teaching practice. In particular, recognising that this never happens on a blank page, but within a big pile of factors that push and pull us towards certain things.

And no, I don’t know how you’d measure that in a survey.

* – may not be actually the case.

The Death of Curiosity? Part 1

Our fall semester is nearly done, and I’ve already started mentally reviewing it. Although this might be a consequence of recency bias, the teaching-learning environment feels like it has been below average.

Given their annotations on Perusall, many students seem to still have great difficulty identifying the thesis, independent variables, and dependent variable of assigned readings — despite the accurate comments written by classmates.

Attendance in class has frequently been below fifty percent, and a greater portion than usual of the students who do come to class look like they mentally check out during discussions. For context, see my October series on physical presence in the classroom here, here, and here.

Possible solutions to the above problems? Instituting pop quizzes based on Perusall readings and class discussions, machine-graded in the Canvas LMS, comes to mind. If students don’t want to voluntarily eat the carrot of knowledge, then perhaps I should use the stick of multiple choice questions that directly affect the course grade. Two potential drawbacks to this method: first, the difficulty of guiding in-class discussions toward quiz questions that were created before the discussion occurred, and second, potential complaints about not being allowed to “make up” quizzes that were missed when absent. My general policy is not to complicate my life by scheduling alternative testing dates, granting deadline extensions, etc. As I’ve stated before, I regard students as legal adults capable of setting their own priorities.

Before classes end this week, I’ll be gathering slightly more objective feedback on the “skills” components of my two undergraduate courses via anonymous surveys. I’ll report the results in my next post.