Assessing Student Preparation

This is follow-up to a previous guest post by Joel Moore on the benefits of assessing students’ preparation for class discussion rather than their participation in it. Students are asked to rate themselves on the amount of assigned reading they have completed prior to class, and this forms the basis of their preparation grade.

He has created a web app that simplifies the process. The app is available at https://joeldmoore.com/apps/preparation/.

A video that discusses how to use the app is at https://www.youtube.com/watch?v=SQdK1FwycKw.

1st Mini-TLC at APSA and the Future of Conferences

In line with Simon’s last post, something of a continued meditation on conferences and academic disciplinary associations in the USA, relative to last weekend’s one-day TLC, which was embedded within the APSA annual meeting:

Bridge to nowhere

Conferences reflect perverse incentives that do not reflect the realities of the academic labor market. Only a small minority of people who obtain PhDs, regardless of field, end up working as tenured professors at elite research universities teaching one or two, or zero, courses per semester. Yet to have even a chance of being hired or tenured by any institution, regardless of its position in the reputational pecking order, one is supposed to present (at conferences) and publish (in journals) research. The research is almost always irrelevant to anyone outside the discipline and much of the time also irrelevant to those within it.

These norms allow academic conferences to prey financially on graduate students, who are led to believe that they must attend, to both present research and to interview. In an age of digital communication tools and decreasing numbers of tenure-track positions, neither search committees nor disciplinary associations should be encouraging graduate students to pay out of pocket to attend conferences, the costs of which can exceed $1,000 per event.

But therein lies the rub: the more people who register for and attend a conference, the more profitable the conference is to the disciplinary association that has organized it. Whether a conference enables graduate students, their advisers, or other faculty to become more effective at what most academics spend most of their time doing — teaching — is not a concern. To claim otherwise is to ignore the economics of the system.

Conference attendance by full-time faculty is subsidized by their employers in the form of professional development support. Yet the way in which most conferences are structured means that opportunities are lacking for enhancing the teaching skills used on a daily basis in the workplace. Given the declining fortunes of many colleges and universities in the USA, this subsidization is likely to decrease, and decrease substantially, at some point in the near future — or maybe it’s occurring already.

Transparent Teaching in Action: Sometimes you aren’t as good as you think you are

In the last few months I’ve been interested in transparent teaching. Basically, transparency is a commitment to all of your students to be transparent about your expectations for a class, particularly when it comes to course assignments. Transparent teaching requires being clear and precise about 3 things in an assignment:

  1. Purpose–explain to the students WHY you are asking them to do this assignment.  Don’t just assume they know how it connects to the material.  Talk to them about the skills and knowledge they will gain and how that is relevant to the course, major, program, or other aspects of their lives.
  2. Tasks–be explicit about the tasks students must follow to complete the assignment.  Sometimes we tell our students to ‘write a 5 page essay’ but don’t realize that many of our students might not know what we mean by an essay–it means different things in different fields, and not all of your students will have taken a polisci class before. Of the three areas, this is where we tend to be the most detailed in our assignments, but even here we can probably be more clear about exactly what we want than we area.
  3. Criteria for Success–the students need to know how their work will be evaluated to determine whether they have successfully completed the assignment or not.  This might mean a rubric, or just a list of what you are going to be looking at.  But one of the most important elements–and the one I messed up on–is it provide examples of successful work.  This might be student work, or something written by a professional working in the field. Want students to write a strong literature review? Show them what a strong one looks like, and talk to them about what makes it strong.  Contrast it with unsuccessful or weak work.  Better yet, give them the examples and the rubric, and have THEM score the work, so they understand how the criteria is applied.

Yes, all of that takes time.  But we owe it to our students to give them every chance to achieve success in the work we assign them.  If we think it is valuable for them to do this work, then we need to give them the detail and time it takes so that success is entirely in their hands.  Plus, doing this has wider benefits.

Research by the Transparency in Learning and Teaching (TILT) team at the University of Nevada in Las Vegas has shown that increasing transparency in our assignments doesn’t just impact their work on those assignments. It leads to big gains in student confidence and sense of belonging in college, particularly for disadvantaged populations who come to college less prepared.  At UNLV, they saw a 20% increase in retention for these students who participated in classes where just two assignments were transformed to become more transparent. I led a project at my university last spring on transparency, and while I’m still evaluating the data, those who participated indicated they found the training sessions and transformation process very useful.  If you want to attend a training on TILT, review their extensive resources, or read the ongoing research, head to their Resources page.

So here I am, someone who has trained on transparent teaching, conducted multiple trainings myself, and doing research in the area.  I’ve also used specifications grading in the past, which is in the transparent teaching wheelhouse.  I’m a pro, right?

Yeah, right.  Every time I think I’ve ‘got’ something when it comes to teaching, I end up learning how much more I have to learn.

I’m teaching a new course this semester called Sex, Marriage, and Violence, and I’m running it largely as a seminar.  Students have to write weekly papers on the readings and pose discussion questions that form the basis for class.  In the syllabus I included a purpose statement for these papers as well as a rubric, and the instructions (I thought) were clear: I wanted a full APA style bibliographic entry for each article or chapter, a 1-2 sentence summary of their main claims or findings, a 500-1000 word analysis of the themes, debates, gaps or issues raised by the readings for the week, and 1-2 discussion questions.

The first papers showed up on Monday, and all but one of them failed to follow this format.  Only a few did the bibliographic entries or summaries; some put the summaries in the main body of the paper (something I explicitly said not to do); only a handful gave discussion questions at the end; and a few wrote essentially annotated bibliographies, with no effort to compare or synthesize the readings.

A couple of students messing up is probably their fault; almost everyone messing up is clearly mine.

The good news is that this is fixable.  Because this is an assignment that starts early and repeats weekly, I was able to catch the problem in week 2.  This is an advantage of using smaller stakes assignments throughout the class–there is less harm to the student if they are completely off base in their efforts, and you have a chance to teach them how to improve. 

To fix this, I recognized a key mistake on my part: while I had provided a rubric, I had not provided an example of successful work.  So I drew up a 2 page outline of a paper.  I didn’t write an entire paper myself.  But the outline showed them the formatting I wanted to see in the paper, such as showing them that I wanted the bibliographic entires and summaries BEFORE the paper proper, and the discussion questions AFTER, not embedded in the text.  And I wrote an example of a thesis statement based on the readings for Week 2 along with a single paragraph showing them how to analyze one of the readings according to that theme.  Drawing up this document took me about an hour, but the students SO appreciated it.  They now had a visual aid to see what I wanted with examples, not just a list of requirements and criteria.  I also allowed them to resubmit their paper by the next class, and throughout the course, will ask students to keep anonymized versions of their work so I will have student-written examples to show future classes.

All of this goes to show that even when you are trying to be transparent, sometimes you might not be able to anticipate what kind of information or models the students need to really understand what you want them to do. When that happens, it is typically OUR fault as instructors, not theirs as students. Now, some students will get it wrong no matter how clear you are, but there are definitely times when we are wrong when we think we are being clear.  I’ve sometimes thought that the ‘provide examples of successful work, and show students why it is successful’ is optional, but on reflection I think this is perhaps the most important thing we can do*. This was an important wake-up call for me that even when I think I’m doing a good job, I can still get it wrong. 

*There are those who question providing examples of successful work lest students simply copy it.  I completely understand the critique, but i think with creativity on our part, this can be overcome. Often I am demonstrating formatting and technique, not content, so as long as you change the content out, there’s little risk of copying.  My go-to example is to make arguments about why cats are better pets than dogs, and I can illustrate the expected structure of an essay, a bibliographic entry or citation, a thesis statement, and using evidence to defend a claim using this very non-political example.

Another point here is that providing a structural template for students who aren’t strong writers is super useful to them.  I’m fine with them imitating my structure–I’m teaching them how to write a strong paper.  I encourage students who have more advanced writing skills to branch away from the suggested structure in my outline/template to find their own style.  

Making the most of it

Upload a video, or seek out a pastel de nata?

I’m back at UACES this week, including our regular L&T workshop, which got a really good turnout for a range of practically-directed panels and workshops. 

As always, one of our concerns in running this event is finding a format and a content that will draw people in, above and beyond the regulars who form the backbone of our work.

This time, that meant inviting applications for active workshops and a roundtable drawing in some different perspectives on engaging learners.

While this was all positively received, my mind already turns to next year’s event, in Lisbon (I know, it is a great venue) and how we can engage people.

Indeed, as we’ve moved into the general conference, I also want to make the most of the time we get to spend together.

Continue reading

Comparing American Foreign Policy Simulations

Today we have a guest post by Charity Butcher, Associate Professor of Political Science at Kennesaw State University. She can be reached at cbutche2[at]kennesaw[dot]edu.

I am always looking for new simulations – particularly ones that are easy to use and require less preparation. For my American foreign policy course, I usually use my own simulation on Iran-US relations. However, the Council of Foreign Relations’ Model Diplomacy simulations, discussed by others here and here, was an opportunity to try something new.

My simulation presents a crisis in Iran-US relations involving nuclear weapons, state support for terrorism, and/or the rivalry between Iran and Israel. In the simulation, students engage deeply with a topic, engage with a large number of state actors, and must deal with the consequences of their decisions. My ability to introduce problems in real-time creates flexibility and makes the simulation more dynamic for the students. However, my simulation requires a lot of preparation, both for me and my students.

Model Diplomacy, on the other hand, offers professors a menu of topics to choose from, and many of the simulations can be completed in a single class period. The simulations come with outstanding background material, so there is little need for students or the instructor to do additional research. However, Model Diplomacy simulations do not move past a predetermined decision point and there are no consequences to participants’ actions. Students sometimes reach a decision very quickly, which might reduce what they learn from the simulation.

I decided to use both my simulation and Model Diplomacy in the last iteration of the foreign policy course, in an attempt to capitalize on the advantages of both. Two one-day Model Diplomacy simulations served as a starting point for a longer three-day simulation. For this longer simulation, students began with the Model Diplomacy Iran Deal Breach scenario, but were provided with additional stimuli during the simulation and were able to interact with other countries. The results were quite positive, and I will continue to use both the short Model Diplomacy simulations along with a longer more interactive simulation in the course.

Extensions to a Classroom Game on War and Peace in IR Theory

Today we have the second of two guest posts by Nathan Alexander Sears, a PhD student in political science at the University of Toronto and previously a professor of international relations at the Universidad de Las Américas, Quito. He can be reached at n[dot]sears[at]mail[dot]utoronto[dot]ca.

The game that I described in my previous post can be modified to demonstrate additional theoretical concepts.

Relative Power: Changing the amount of resources each state possesses at the beginning of each round creates differences in their relative capabilities. For example, State A could begin with $10, State B with $7, and State C with $5. This change may lead to balancing and bandwagoning behavior, which is important in neorealism.

National Identities: The game can be played with actual country names, such as the United States (State A), the Soviet Union (State B), the People’s Republic of China (State C), Great Britain (State D), and France (State E). This opens up the possibility that students’ ideas about national identities or knowledge of history may influence behavior and outcomes, which is central to constructivism.

Democratic/Authoritarian States: The game can include democratic and authoritarian structures for internal decision-making. For example, one team might be required to make decisions by majority vote, while another group may have a single individual who makes such decisions with the other students acting as advisors. Regime and institution type is important in liberalism, especially in democratic peace theory. Continue reading

This year’s novelty

Come on, live a little

One of the things I like most about having an interest in learning and teaching is that it never stands still; there’s always another angle to explore.

Moreover, those angles aren’t always ones I know much about, so it’s an opportunity for me to broaden my knowledge and experience in ways that I might not even have thought possible.

For that reason, I often sign up for pilots and trials at work, because maybe it’ll prove useful.

(not always though: I once had an unhappy few weeks with an interactive whiteboard, about which I’ll say no more)

With all this in mind, this year I’m going to be trying out a tool for writing exams on laptops in exams.

Continue reading

Cultural and Historic Preservation Conference

Another announcement about an upcoming conference — in case anyone wants to enjoy southern New England autumn weather:

Registration is open for the annual conference of the Noreen Stonor Drexel Cultural and Historic Preservation program at Salve Regina University in Newport, Rhode Island, October 12-13, 2018. The theme of this year’s conference is “Community Preservation through Adaptive Reuse.”

Full details on the conference and how to register are at this link.

Call for Papers – HTA Conference

A last minute call for papers for the 40th annual Humanities and Technology Association (HTA) Conference, to be held at Salve Regina University, Newport, Rhode Island, on November 1-3. Deadline is September 1. Additional details are at this link. The HTA is an interdisciplinary organization and it welcomes submissions from all fields — from faculty, graduate students, and undergraduates.

“…were being given silly things to do…”

Plan B

So that was the summer pause, apparently, and now it’s back to the grindstone.

Luckily, one of my very first tasks is to get ready for the annual UACES Teaching & Learning workshop, which we run ahead of the main conference, this year in the fine city of Bath, England.

The event has developed over recent years into a very useful mix of activities and reflection, each time taking a different approach, to keep it fresh for participants and to attract the curious.

However, one element that seems to be a recurrent one is an opening ice-breaker activity. Last year, I found myself scurrying around Krakow to find post-it notes (and then writing a very similar opening to the current post, it seems).

As you’ll note, this year, I’m writing ahead of the activity, mainly because I wanted to reflect a bit on what the function of an ice-breaker is/might be, rather than an actual example.*

Thinking about it abstractly, we’re definitely trying to do one thing, and usually trying to do another too.

Loosening-up

The definite element of an ice-breaker is to reduce inhibitions among participants. Like the eponymous ship, the activity is intended to get us out of ourselves, feel less self-conscious and start to develop a sense of a group, within which exchange is easier.

We do that by distraction, broadly speaking. Give people a task, especially if it’s light-hearted, and they’ll be likely to get into it. I think here it’s partly about diverting attention from “here’s a bunch of people I don’t know” to “here’s a fun thing to do”, which in turn opens up a reason to talk/interact with those people we don’t know.

To flip that around, you can’t just stand at the front of a group and tell them to become less inhibited and more willing to participate. Or rather, you can, but your chances of success are slim.

And we do this dishibiting because we think it aids subsequent debate and work. Individuals are more likely to speak up, connections are more likely to be made and generally more will be got from the session, because there’s more focus. At a mundane level, that might just be because the enjoyment of it all means people are less likely to be distracted by their mobile phones.

Learning stuff

But there’s also a second element in a ice-breaker that is usually found, namely increasing knowledge.

That might be something simple, like learning the names of other people, or something about their work (as in the Krakow exercise).

But it can also be a more abstract point, such as the nature of human interactions (as in the Hobbes game), or scholastic skills (as here). Clearly, that insight can be both mixed with the more prosaic stuff, and also connected to the wider objectives of the session.

In this, we’re doing something very close to a simulation game: getting participants to have a visceral experience that feeds into their emergent understanding of a situation.

Together

Thinking about ice-breakers in such a fashion can be helpful, not least in identifying what you what to achieve from it.

In practical terms, you’re always going for the dishibition, so you need to be asking what else you want/need to achieve while you’re doing it.

In my case, given the rest of the programme for the workshop (a variety of active sessions), and the 15 minute slot I’ve got, I need to keep things simple and focused on ‘getting to know you’-type things.

Which means my big pile of blindfolds probably has to wait for another occasion.

* No, I still to sort out what’s happening in Bath. Obviously.