To Quiz or Not to Quiz, Part 2

As promised in my last post, here are the results of the survey of students’ study habits in my course on economic development. Sixteen of seventeen students in the class completed the survey (after repeated reminders); a good response rate although not statistically-significant sample.

When in class, I . . .

  • write notes on a laptop: 25%.
  • write notes on paper by hand: 75%.
  • do not write notes: 0%.

For quizzes, I usually prepare by (choose all that apply):

  • re-reading assigned articles and book chapters: 18.8%.
  • reviewing notes taken during class: 100%.
  • reviewing notes taken when I read assigned texts the first time: 12.5%.
  • doing nothing: 0%.

While I was pleasantly surprised at the number of students who said they take notes — especially handwritten ones — in class, I told the class that highlighting or underlining is an unproductive study technique. It doesn’t engage the motor and language areas of the brain like writing does. Also, when reviewing prior to a quiz or exam, often one has no memory of why one marked a sentence in a text as important. Whether students will heed my advice, I don’t know.

Continue reading

To Quiz or Not to Quiz

Earlier this year I wrote a series of posts on redesigning my course on economic development. As part of the redesign, I added ten in-class quizzes as a way to reinforce students’ learning of important concepts. From my perspective, the quizzes are simple — a maximum of four questions, many of which can be adequately answered with short phrase or sentence. For example:

Why do the rural poor tend to have more children than the urban rich?

However, students’ scores on the first four quizzes have been quite low. I started wondering if bad study habits were causing the terrible quiz performance. Not being able to read students’ minds or observe their behaviors outside of class, I decided to do something unusual: ask them. I created an anonymous online survey with Google Forms. Here are the questions:

When reading assigned texts, I . . . 

  • write notes about important ideas as I’m reading.
  • highlight, underline, or otherwise mark important ideas as I’m reading.
  • do not write notes or mark up text as I’m reading.

When in class, I . . .

  • write notes on a laptop.
  • write notes on paper by hand.
  • do not write notes.

For quizzes, I usually prepare by (choose all that apply):

  • re-reading assigned articles and book chapters.
  • reviewing notes taken during class.
  • reviewing notes taken when I read assigned texts the first time.
  • doing nothing.

I’ll publish the survey results next week.

Links to the original series on redesigning this course:

Interviewing the EU in Brussels

Today we have a guest post by Jamal Shahin and Claske Vos, faculty in European Studies at the University of Amsterdam. Jamal also works at the Institute for University Studies at Vrije Universiteit Brussel. They can be reached at shahin[at]uva[dot]nl and C[dot]Vos[at]uva[dot]nl.

Students in our one-year M.A. programme at the University of Amerstam needed to simultaneously gain knowledge of the EU and develop research skills, all in a four-week ‘skills seminar’ that runs every January. We decided to address this challenge by having students research policy by gathering data
 first-hand from EU policymakers.

We start by inviting the students to consider their ‘burning question’—what they really want to explore—in their MA theses. In the seminar’s second week, they identify methods and theoretical approaches, with a specific focus on the relationship between policy fields and research methods. Students draft deliverables, which are then peer reviewed at this stage. In the fourth week, students learn more about the practice of European policymaking by interacting with EU officials and other ‘experts’ in Brussels. To prepare for this stage, students must first:

  • Arrange the interviews.
  • Create an interview protocol—semi-structured interview guide.
  • Present a literature and policy review in written form.

These tasks are intended to give students a clearer view of the topics they wish to raise with their interviewees. At the end of the entire process, the students are expected to write up a thesis proposal to submit to their thesis supervisor.

From first contact to first meeting

Many students are daunted by the prospect of interviewing experts in fields that they feel that they are only just starting to understand. It therefore takes some effort to assure them that their requests for information will not go unheard. We help the students write the initial emails that request an interview, and provide them with a rough template that describes the etiquette to use when presenting oneself to the potential informant. Our multidisciplinary faculty, which includes anthropologists, historians, political economists, and political scientists, can help the students express themselves to their interviewees. We hold a half-day workshop with the students, in which a range of lecturers from different departments present their own interview experiences.

Bursting the ‘Brussels bubble’

‘Brussels’ is seen by many as a distant place, remote from the realities of daily life, even for students of European policymaking. This course helps students break through this perceived bubble by allowing them to engage directly with the policymakers themselves, rather than studying the policy process only through literature. It also occurs after an intensive eight-week course on European integration theories, and thus helps bring this knowledge ‘to life’—something that for students is frequently an unexpected benefit.

Negatives of Nuance

Related to Simon‘s and Amanda‘s recent posts about failing to get the results one expects, here is a very simple example.

My first writing assignment prompt this semester for my course on economic development and environmental change read as follows:

Purpose of this response: learn about the nature of poverty and perceptions of the poor.

1. Read the rubric below.

2. Read/watch:

  • MRU: Basic Facts of Growth and Development.
  • Easterly, Ch. 1 and Intermezzo, p. 5-19.
  • Emily Badger and Quoc Trung Bui, “In 83 Million Eviction Records, a Sweeping and Intimate New Look at Housing in America,” The New York Times.
  • Hans Rosling, “The Magic Washing Machine,” TEDWomen, December 2010.

Answer the following question:

  • William Easterly writes “When those of us from rich countries look at poor countries today, we see our own past poverty.” (p. 19). Is this an accurate view of poverty outside and inside the USA? Why?

My intent in asking the question — which I thought was obvious from the “statement of purpose” that prefaced it — was to get students to think about the differences between how they think about the poor at home and about the poor in countries that they’ve probably only seen on the news, if at all. In other words, I wanted them to start thinking about why our perceptions of the poor often depend on where the poor are located.

Instead students wrote about wealth and poverty from a historical perspective — that the USA once had a low level of economic development, as measured by GDP or income per capita, and that’s where other countries are today. The USA developed into a high-income, industrialized society, so probably other countries, even the poorest ones, will eventually do the same. There was no real discussion of the nature of poverty in the USA or why it exists.

So I need to a better question, something like “Are the causes of poverty in the USA the same as in other countries? Why?” or “If the USA is an economically-developed country, why does it still have poor people?”

Beyond the Essay: Briefing Memos

Today we have a guest post from Vincent Druliolle, an assistant professor at Universidad Carlos III de Madrid. He can be reached at Vincent[dot]Druliolle[at]gmail[dot]com.

Undergraduates are repeatedly told that what they study is somehow relevant for practice, yet most assignments are structured as academic essays—even though only a handful of them will end up opting for an academic career. A few years ago, I decided that my students should have the opportunity to develop non-academic writing skills, and started assigning a briefing memo about an ongoing conflict.

The briefing memo is indeed a format widely used in government, international organisations, consultancies, and NGOs. However, because of the large range of topics and theoretical perspectives covered by my module and the limited number of teaching weeks, I had to find a way of integrating such an activity into my small-group seminars. I came up with the idea of making the memo a preparatory activity for my in-class simulation on peacebuilding and transitional justice.

The briefing memo differs from the traditional essay in both content and format. It is policy-oriented, because it is aimed at practitioners and decision-makers, and it presents information in a concise and attractive manner. It requires critically analysing source material beyond the standard academic literature, selecting what’s most relevant, and presenting it in a way that can convey the complexities of the conflict analysed.

Most students have never written a memo, but I don’t give them any guidelines. Instead, I ask them to look up examples that they can use as models. I prefer to ask the students to present their memos in class and discuss the difficulties of writing it. The first seminar of the simulation is thus about comparing and learning from the work of one’s fellow classmates. For class discussion, I recommend selecting at least a very good memo, a (very) bad one, and a few with significantly different formats and/or content. The greater the variety of memos, the better. I want the students to learn from each other, so I adopt the role of a facilitator, asking them to explain why they’ve chosen a given format and/or content, and fostering a class discussion about these aspects.

Many students admit that, as I warn them beforehand, it’s difficult at the beginning to figure out how they have to write the memo. Instead of assessing it at this stage, I ask the students to submit a revised version after the simulation that reflects what they’ve learnt from their classmates’ memos. Guidelines about how to write a memo can be provided at this stage or even afterward as part of a debriefing.

While writing the memo is an activity in its own right, in this case it is also a way for the students to acquire the knowledge necessary to participate in the simulation. They learn what information the memo should include because they have to put themselves in the shoes of the actors for whom the memo is written in the first place. In this way, the memo prepares students for the simulation, while the simulation provides an opportunity for them to reflect on the writing of the memo. And for the instructor, memos are quicker (and less boring) to mark than essays.

Using Hidden Brain to Teach Research Design

Today we have a guest post by Sarah Fisher, assistant professor at Emory & Henry College. She can be reached at sfisher[at]ehc[dot]edu.

For undergraduates, the research methods course is often the most dreaded component of the political science curriculum. Students’ fear of mathematics, gaps in content knowledge, and lack of software experience (I’ve had students who have never opened Microsoft Excel, much less heard of a statistical software package) present pedagogical challenges for an instructor. 

Florian Justwan and I recently published an article about teaching social science research methods, “Scaffolding Assignments and Activities for Undergraduate Research Methods,” in the Journal of Political Science Education. The article, available here, includes instructions on how to use an episode of 30 Rock to teach content analysis, how to teach similar systems design with presidential speeches, and other activities. Teaching materials described in the article are available on my website.

One resource the article references is NPR’s show Hidden Brain hosted by Shankar Vedantam, which illustrates social science research for a popular audience with topics like the behavior of baseball umpires in extra innings and ways that colleges try to prevent summer melt. I ask students to identify the theory, hypotheses, causal mechanisms, and findings contained in different episodes and then tie this activity to longer articles assigned as homework readings. The program allows me to expand my range of examples beyond political science, important given that many of the students in my course are in majors ranging from sociology to athletic training to mass communication. This kind of content shows students that the scientific method isn’t reserved for academia; it can be used in their daily lives to interpret the world around them.

The Benefits of Recycling

Today’s post is more about career development than teaching . . .

Academia is a bureaucratic work environment. Information is constantly documented and distributed. Often this happens to the same piece of information multiple times. Consequently I began recycling my writing as much as possible several years ago, in the belief that it is better to make minor changes, or none at all, to writing upon which I have already expended mental energy. An underlying principle here is writing with an ultimate rather than a proximal use in mind. What is the most valuable end to which this writing can be eventually directed? A simple example: the proposal for your conference presentation becomes the abstract for the conference paper, which in turn becomes the abstract for the manuscript submitted to a journal.

A second and, for some, more important example: the stream of email, editorial comments, draft committee proposals, and other written minutiae that one produces — it’s all work. Don’t let it disappear into the ether. Instead, use it for future contract renewal, tenure, or promotion.

I admit that I didn’t fully recognize the potential value of this writing until my wife — also an academic — compiled her application dossier for promotion to full professor. Watching her, I realized that, in the course of my day-to-day business as an associate professor and department chair, I had generated chains of emails and memos that constituted evidence of service and scholarship à la the Boyer model. I saw that this material, if organized coherently, could form much of my own application for promotion, in many cases verbatim. Continue reading

ISA Innovative Pedagogy Conference

Registration is now open for the first annual International Studies Association Innovative Pedagogy Conference (ISA-IPC), which will be held on Thursday, November 15, 2018, in St. Louis, MO, in conjunction with the ISA Midwest meeting. This new ISA initiative marks the beginning of a series of programs to be held in conjunction with regional conferences around the United States, and beyond. The one-day event will foster a highly interactive environment to explore new ideas in pedagogy and assessment—and offers rich opportunities for professional development, networking, and classroom skills. There will be three types of sessions at the ISA-IPC: workshops, graduate teaching assistant training, and plenary meetings.

  • Workshops will be directed by leading voices on pedagogy, assessment, and professional development. Participants will attend four workshops during the day, chosen from a rich menu, to share innovations and ideas about different themes in international studies pedagogy. Among the workshop themes for 2018 are: Simulations & Games for Teaching Violence and Peace; Publishing Your Innovative Teaching Work; Research Literacy; International Studies Curriculum Design; Faculty-Led Study Abroad Programs; and Global Service Learning. These interactive short sessions will provide hands-on experience and materials for ready application, along with opportunities for professional networking and sharing of ideas.
  • Graduate Teaching Assistant Training session offer a hands-on learning opportunities for advanced graduate students who are instructing their own classes at their universities. Training will focus on running effective active teaching exercises, dealing with challenging situations in the classroom, assessment, and turning teaching opportunities into professional success. A certificate of participation will be provided for all attendees.
  • Plenary sessions will focus on best practices in innovative teaching and promote opportunities for collaboration and exchange. From the opening session to an evening networking reception, participants will share ideas with like-minded colleagues. A keynote presentation by ISA President Patrick James will foster further dialogue on best practices in active teaching and learning.

Game Theory Resource

A brief note on a free resource:

While crawling through the Intertubes, I stumbled across a twenty year old webpage on game theory by Eric Roberts, emeritus professor of computer science at Stanford. The sub-directory pages contain short, simple discussions of different theoretical models, with helpful examples — such as Rousseau’s stag hunt and versions of Prisoner’s Dilemma. Of particular interest, at least to me, is the explanation of strategies of play — given that I almost always try to minimize the maximum possible loss.

Assessing Student Preparation

This is follow-up to a previous guest post by Joel Moore on the benefits of assessing students’ preparation for class discussion rather than their participation in it. Students are asked to rate themselves on the amount of assigned reading they have completed prior to class, and this forms the basis of their preparation grade.

He has created a web app that simplifies the process. The app is available at https://joeldmoore.com/apps/preparation/.

A video that discusses how to use the app is at https://www.youtube.com/watch?v=SQdK1FwycKw.