Transparent Teaching in Action: Sometimes you aren’t as good as you think you are

In the last few months I’ve been interested in transparent teaching. Basically, transparency is a commitment to all of your students to be transparent about your expectations for a class, particularly when it comes to course assignments. Transparent teaching requires being clear and precise about 3 things in an assignment:

  1. Purpose–explain to the students WHY you are asking them to do this assignment.  Don’t just assume they know how it connects to the material.  Talk to them about the skills and knowledge they will gain and how that is relevant to the course, major, program, or other aspects of their lives.
  2. Tasks–be explicit about the tasks students must follow to complete the assignment.  Sometimes we tell our students to ‘write a 5 page essay’ but don’t realize that many of our students might not know what we mean by an essay–it means different things in different fields, and not all of your students will have taken a polisci class before. Of the three areas, this is where we tend to be the most detailed in our assignments, but even here we can probably be more clear about exactly what we want than we area.
  3. Criteria for Success–the students need to know how their work will be evaluated to determine whether they have successfully completed the assignment or not.  This might mean a rubric, or just a list of what you are going to be looking at.  But one of the most important elements–and the one I messed up on–is it provide examples of successful work.  This might be student work, or something written by a professional working in the field. Want students to write a strong literature review? Show them what a strong one looks like, and talk to them about what makes it strong.  Contrast it with unsuccessful or weak work.  Better yet, give them the examples and the rubric, and have THEM score the work, so they understand how the criteria is applied.

Yes, all of that takes time.  But we owe it to our students to give them every chance to achieve success in the work we assign them.  If we think it is valuable for them to do this work, then we need to give them the detail and time it takes so that success is entirely in their hands.  Plus, doing this has wider benefits.

Research by the Transparency in Learning and Teaching (TILT) team at the University of Nevada in Las Vegas has shown that increasing transparency in our assignments doesn’t just impact their work on those assignments. It leads to big gains in student confidence and sense of belonging in college, particularly for disadvantaged populations who come to college less prepared.  At UNLV, they saw a 20% increase in retention for these students who participated in classes where just two assignments were transformed to become more transparent. I led a project at my university last spring on transparency, and while I’m still evaluating the data, those who participated indicated they found the training sessions and transformation process very useful.  If you want to attend a training on TILT, review their extensive resources, or read the ongoing research, head to their Resources page.

So here I am, someone who has trained on transparent teaching, conducted multiple trainings myself, and doing research in the area.  I’ve also used specifications grading in the past, which is in the transparent teaching wheelhouse.  I’m a pro, right?

Yeah, right.  Every time I think I’ve ‘got’ something when it comes to teaching, I end up learning how much more I have to learn.

I’m teaching a new course this semester called Sex, Marriage, and Violence, and I’m running it largely as a seminar.  Students have to write weekly papers on the readings and pose discussion questions that form the basis for class.  In the syllabus I included a purpose statement for these papers as well as a rubric, and the instructions (I thought) were clear: I wanted a full APA style bibliographic entry for each article or chapter, a 1-2 sentence summary of their main claims or findings, a 500-1000 word analysis of the themes, debates, gaps or issues raised by the readings for the week, and 1-2 discussion questions.

The first papers showed up on Monday, and all but one of them failed to follow this format.  Only a few did the bibliographic entries or summaries; some put the summaries in the main body of the paper (something I explicitly said not to do); only a handful gave discussion questions at the end; and a few wrote essentially annotated bibliographies, with no effort to compare or synthesize the readings.

A couple of students messing up is probably their fault; almost everyone messing up is clearly mine.

The good news is that this is fixable.  Because this is an assignment that starts early and repeats weekly, I was able to catch the problem in week 2.  This is an advantage of using smaller stakes assignments throughout the class–there is less harm to the student if they are completely off base in their efforts, and you have a chance to teach them how to improve. 

To fix this, I recognized a key mistake on my part: while I had provided a rubric, I had not provided an example of successful work.  So I drew up a 2 page outline of a paper.  I didn’t write an entire paper myself.  But the outline showed them the formatting I wanted to see in the paper, such as showing them that I wanted the bibliographic entires and summaries BEFORE the paper proper, and the discussion questions AFTER, not embedded in the text.  And I wrote an example of a thesis statement based on the readings for Week 2 along with a single paragraph showing them how to analyze one of the readings according to that theme.  Drawing up this document took me about an hour, but the students SO appreciated it.  They now had a visual aid to see what I wanted with examples, not just a list of requirements and criteria.  I also allowed them to resubmit their paper by the next class, and throughout the course, will ask students to keep anonymized versions of their work so I will have student-written examples to show future classes.

All of this goes to show that even when you are trying to be transparent, sometimes you might not be able to anticipate what kind of information or models the students need to really understand what you want them to do. When that happens, it is typically OUR fault as instructors, not theirs as students. Now, some students will get it wrong no matter how clear you are, but there are definitely times when we are wrong when we think we are being clear.  I’ve sometimes thought that the ‘provide examples of successful work, and show students why it is successful’ is optional, but on reflection I think this is perhaps the most important thing we can do*. This was an important wake-up call for me that even when I think I’m doing a good job, I can still get it wrong. 

*There are those who question providing examples of successful work lest students simply copy it.  I completely understand the critique, but i think with creativity on our part, this can be overcome. Often I am demonstrating formatting and technique, not content, so as long as you change the content out, there’s little risk of copying.  My go-to example is to make arguments about why cats are better pets than dogs, and I can illustrate the expected structure of an essay, a bibliographic entry or citation, a thesis statement, and using evidence to defend a claim using this very non-political example.

Another point here is that providing a structural template for students who aren’t strong writers is super useful to them.  I’m fine with them imitating my structure–I’m teaching them how to write a strong paper.  I encourage students who have more advanced writing skills to branch away from the suggested structure in my outline/template to find their own style.  

Crowd-Sourcing and Self-Instruction

Today we have a guest post from Matthew Wilson, assistant professor of political science at West Virginia University and a visiting researcher at the University of Gothenburg as part of the Varieties of Democracy Project. He can be reached at matthew[dot]wilson1[at]mail[dot]wvu[dot]edu.

Sometimes, existing teaching materials can be too narrow or too broad. This was the problem that I encountered when designing a lower-level undergraduate course on Latin American Politics. Many textbooks on Latin American politics are organized around conceptual issues with specific chapters on topics such as economic inequality or race. However, I wanted students to also learn about the unique paths by which countries in the region developed, without limiting the discussion or using a handful of countries to characterize the rest.  Rather than cobbling together different materials myself, I saw this as an opportunity for active learning.

My idea was to create teams of students with each team seeking out information on a different country outside of class. This approach drew on distributed learning, which aims to decouple learning from the classroom constraints of time and place, by creating a learning objective outside of class that differs from but contributes to what students learn in class. The approach also reflected crowd-sourcing, where a good is produced by many people performing relatively small tasks. The assignment therefore had to involve a large number of students.

Students listed their top three preferences for countries and I matched them up as best I could. I aimed to control the quality of sources and focus of the assignment, while at the same time encouraging students to teach themselves by conducting independent research.  I personally vetted the content that students used by selecting five books that covered each of the roughly twenty countries—for a total of 100 books—and placed them on hold in the university library.  I also required each student to submit a list of ten additional online sources for my approval.

The assignment had two parts, for which students received separate grades.  First, students had one month to consult the source materials and document major events that occurred in their respective countries. I created a spreadsheet with four tabs that corresponded to heads of state, conflicts, laws, and important documents.  For each, students had to skim the respective material and fill in basic information about the event, denoted by column headings:

  • The year in which an event occurred (when)
  • The event (what)
  • The actors involved (who)
  • The source(s) consulted

I discouraged students from providing any sort of explanation.  Moreover, I was purposely vague about what constituted an event to encourage them to seriously consider what mattered. I graded students’ spreadsheets in terms of thoroughness; in large part, this was determined by comparing the spreadsheets of students who were assigned the same country.

In the second step, I grouped students into teams according to the country they had researched, and each team created a combined, revised timeline that described in only a few sentences each event that had been included. I checked the accuracy of the content in the timelines with the help of graduate research assistants, and each team received a grade on its combined timeline. At the end of the course, I consolidated students’ timelines into a single manuscript, added public domain images, and handed the final product back to them.

The results of this assignment were quite positive. It enabled me to complement the country-specific knowledge students were acquiring outside of class with lectures on more general themes. Each student became a “country expert” and therefore almost always had something to contribute to in-class discussions, which in turn improved their essay responses. Students were motivated to work on a project that was not the standard research paper. Overall, the distributed learning, self-instruction, and collaboration with teammates enhanced students’ performance in the course. I will definitely use this technique in the future. Additional details can be found in my article about the assignment in the Journal of Political Science Education.

Changing a Course on Development, Part 7

I’ve been a fan of the quality of failure essay since Amanda introduced me to it several years ago, and I’ve tweaked it several times with varying degrees of success. In an attempt to avoid a mistake I made with it last semester, I have altered the assignment yet again by shortening the instructions considerably:

Read:

In a 2-3 page essay, analyze how you learned in this course. What actions helped or hurt your learning? Which components of the course most enabled you to better understand ideas or apply them in new ways?

Note that I have moved significantly away from the assignment’s original theme of failure. I am doing this for two reasons. First, in other courses this assignment has produced a lot of commentary from students about what I will label the superficial aspects of failure — as in “at the beginning of the semester I promised myself that I would get an A++ on every assignment but I failed at this because I didn’t manage my time well.” Second, I am curious to find out whether students regard the SCAMPER-based game design exercises as worthwhile, but I’m not going to influence their thoughts by explicitly asking about it.

Changing a Course on Development, Part 6

My general approach to teaching is to emphasize the upper levels of Bloom’s taxonomy. Creation and evaluation are important. Memorization, not so much. While game design gives students the opportunity to create something connected to course content, they should also evaluate whether what they’ve created is on target. So, as promised in my last post, here is the relevant assignment, be due after students play the games that they have designed:

1. Read the rubric below.

2. In the form of a 3-4 page, double-spaced essay, evaluate the game you played that was designed by another team. How well did the game:

Work independently, do not discuss your essay with other students.

 

Changing a Course on Development, Part 5

In my last post in this series, I discussed integrating the SCAMPER technique with student game design via a writing assignment and in-class presentations. I’m a firm believer in the benefits of iteration when it comes to learning, so I’m including a second round of game design. For the second round, students will again use SCAMPER, but this time they will actually build new games. Here is the preparatory writing assignment:

Problem

People frequently do not understand the relationships between economics, politics, and the environment. Games are powerful learning tools, but there are few high-quality games about these relationships.

Solution

Design a game that illustrates a relationship between economics, politics, and the environment.

Apply SCAMPER to an existing game other than the California Water Crisis game  — for example, Risk, Mahjong, Settlers of Catan, or Monopoly — to design a framework for a new game. Choose a topic of interest. Put the game in a specific context, such as “the effects of sea level rise in Boston” rather than “climate change.”

Audience

Write a proposal to Hasbro’s Product Development Division in which you discuss the new game you have designed by using SCAMPER on an existing game. Identify the topic of the new game, what features of the existing game will change, how they will change, and why these changes are beneficial.

After students have submitted their individual proposals, I will again cluster the class into teams. The members of each team will discuss their ideas, decide on a single design to pursue, and create and deliver in-class presentations. I’ve devoted a subsequent class session for teams to physically construct the games and another one for students to actually play the games. Debriefing will occur via another writing assignment, which will be the subject of my next post.

Changing a Course on Development, Part 4

Despite varying degrees of success in my first-year seminar — which I decided to stop teaching — I’m going to again have students design board games based on course content. But I’m going to organize this process differently than before. 

There will be two rounds of game design and each will use SCAMPER, an acronym for a design thinking technique that I will demonstrate with an in-class exercise. In the first round, each student will complete a writing assignment that applies SCAMPER to the California Water Crisis (CWC) game used by Andrew Biro. Here is what SCAMPER looks like in this context:

  • Substitute: what part of the game can be substituted for some other part?
  • Combine: can two separate processes in the game be integrated into one?
  • Adapt: can an aspect of some other game be adapted for use in this game?
  • Modify: can a process that is part of the game be modified, enhanced, or simplified?
  • Put to other use: can a part of the game serve some other function within the game?
  • Eliminate: can any part of the game can be removed/omitted?
  • Reverse: what happens if some process in the game is reversed?

Here are the directions for the first round’s writing assignment: Continue reading

Changing a Course on Development, Part 3

As a follow-up to Part 2 in this series, here are specific examples of how culling learning objectives and readings led to better alignment with assignments.

My old version of the course included the topics of poverty, aid, economic growth,  economic geography, corruption, and ethnic conflict. For the new version, I abandoned the last three of these as learning objectives. This allowed me to discard corresponding chapters from William Easterly’s The Elusive Quest For Growth: Economists’ Adventures and Misadventures in the Tropics (MIT Press, 2001), plus other, shorter readings.

The old course had content organized under headings like “economic institutions” and “economic behavior.” While I am intricately familiar with these terms, students are not. As learning objectives, they are too broad. “Barriers to entrepreneurship” is more useful. As I mentioned in Part 2, students will see each of these objectives as a meta-prompts for reading responses, which are also now more specific. For example, in the portion of the syllabus where I am still using Poor Economics: A Radical Rethinking of the Way to Fight Global Poverty by Abhijit Banerjee and Esther Duflo (Public Affairs, 2011):

Old assignment A

Why don’t the poor create their own microfinance institutions, instead of “waiting” for outsiders to do it for them?

New assignment A

Purpose of this response: learn about the role of insurance in mitigating economic risk.

  • Bannerjee and Duflo, Ch. 6, p. 133-156.
  • Gardiner Harris, “Borrowed Time on Disappearing Land,” New York Times, 28 March 2014.
  • Brooke Jarvis, “When Rising Seas Transform Risk Into Certainty,” The New York Times, 18 April 2017.

Can insurance help Bangladeshis minimize economic risk? Why? What about people living in Norfolk and Houston? Why?

Old assignment B

  • Bannerjee and Duflo, Ch. 10 and Conclusion, p. 235-274.

Is the pessimism of the Acemoglu and Robinson development model correct? Why?

New assignment B

Purpose of this response: learn about the effectiveness of public policy in a changing natural environment.

  • Bannerjee and Duflo, Ch. 10 and Conclusion.
  • Richard Conniff, “The Vanishing Nile: A Great River Faces a Multitude of Threats,” Yale Environment 360, 6 April 2017.
  • Community Water Solutions, “Empowering Women Entrepreneurs to End the World Water Crisis,” 5 April 2013, https://www.youtube.com/watch?v=1zUBOLzfrQo.

Will small, decentralized, community-based changes lead to large-scale sustainable economic development? Why?

Changing a Course on Development, Part 2

Back in February, I wrote about using the question “What don’t students need?” to help me redesign my course on economic development — a guiding principle that is very different from “What do I like to teach?”

This last question got me thinking about my history with the subject and how that has affected my preferences when teaching it. I can thank Dr. William Joseph for first introducing me to the political economy of Third World development in an undergraduate course that I took decades ago at Wellesley College. His course sparked an interest that would eventually become the foundation of my academic career. At the time, the field was transitioning away from the binary lenses of modernization theory and post-colonial thought and toward new institutional economics. A standard syllabus would begin with Western imperialism, progress through post-independence struggles in a global capitalist order, and end with the success stories of newly-industrialized states in East Asia. There would be some discussion of international financial institutions — the IMF and World Bank — along the way.

For today’s War on Terror generation, decolonization, the Green Revolution, and Japan’s post-World War II industrialization are ancient history. These topics are both relevant and fascinating for a historically-minded person like me, but I only have fourteen weeks in the semester and my students will probably never take another course on economic development.

I culled the reading list a second time while trying to keep the above in mind, and more readings went into the rubbish bin. In the process I created some parsimonious alignment between learning objectives and writing assignments, which in turn led to an easy set of meta-prompts to preface those assignments. Each meta-prompt begins with “Purpose of this response: learn about . . . ” and ends with the specific learning objective that the assignment targets. Here is the list of those objectives:

  • The nature of poverty and perceptions of the poor.
  • Foreign aid.
  • Demographics and carrying capacity.
  • Causes of economic growth.
  • The role of agriculture in development.
  • Human capital.
  • The management of market externalities.
  • Effects of and barriers to savings, credit, insurance, and entrepreneurship for the poor.
  • The relationship between property regimes and natural resource management.
  • The socioeconomic consequences of urban development policies.
  • The effectiveness of public policy in a changing natural environment.

I’ll be posting more about redesigning this course in the coming weeks.

Additional Snippets from Comparative Politics

A few more quick thoughts about my comparative politics course, which just ended . . .

On the last day of class, I distributed pieces of paper and asked students to write down what they thought were the two best and two worst assigned readings, and to include brief explanations of their choices. Nine students were in the room — out a class of ten. Despite such a small sample, I will go out on a limb and draw some conclusions, all of which relate to transparency:

  • I should preface each writing assignment with a very brief overview of how assigned readings relate to the topic at hand, because students don’t automatically know this. For the question “Does Chinese culture promote authoritarianism?” the class read an interview with Singapore’s former leader Lee Kuan Yew. Two of the students didn’t understand that Yew, being of Chinese ancestry like many other Singaporeans, was discussing his views on Chinese cultural norms.
  • I should also explicitly inform students that readings are written in different styles for different audiences, which means that some of the readings will be more difficult and less enjoyable to read than others. For example, in the section of the course on revolution, I assign Theda Skocpol’s 1976 article “France, Russia, China: A Structural Analysis of Social Revolutions.” Two students said this article was a least favorite reading, because of its style. Oddly, in my opinion, a few other students complained that the long-form journalism found in some of The New York Times articles was hard to understand.
  • Students have difficulty placing unfamiliar historical events in chronological order and parsing out how what happened before might have affected what happened after (the “history just happens” mentality). For an assignment about Iran, students felt confused about the relationship between the 1979 revolution, popular support for the post-revolutionary Islamic Republic, and more recent anti-government protests. While I find it easy to connect the dots, students don’t — for them, context can be confusing. So for some assignments I need to narrow down the reading list.

After students finished identifying readings they liked or disliked, I asked them if they would have preferred formal mid-term and final exams, or regular quizzes, as an alternative to some of the writing assignments. I was surprised that they said no, since I had been under the impression that they used test scores to reassure themselves of their performance. One senior said that he didn’t remember a thing from test-based courses he had taken, while he remembered a lot from writing-based courses, and therefore he regarded the latter as much more beneficial. Two others said that, as studies have shown, they and their friends promptly forgot whatever facts they dumped into short-term memory the night before an exam, so they regarded exams as anxiety-producing events irrelevant to their learning. If next year I use quizzes as a stick to improve attendance in class, they will probably need to be the “write a paragraph that answers the question” type.

 

Changing a Course on Globalization, Part 6

Another update, this time in relation to the Place Making Essay discussed in Part 2  of this series —

To backtrack a bit, and provide some context I probably should have supplied in Part 1, the objectives of this course are to:

  1. Introduce students to concepts and methods used by social scientists and others to explain globalization.
  2. Develop the skills needed to understand complex problems related to global interconnectedness.

Students’ essays did, with varying degrees of success, thoughtfully respond to the assignment’s two prompts:

  • How does the process by which an object is made affect its ability to create a sense of place for people who use that object?
  • Has globalization altered the meaning of places or of the objects within them? Why? If so, how have meanings changed?

I did not see much discussion about the ways in which globalization affects communities, in the sense of “place making.” In retrospect, this is another example of me assuming, incorrectly, that students will follow ideas down the rabbit hole like I do — examining the more nebulous systemic implications of narrowly-defined events. If I use this assignment in the future, I might change the prompt to something like:

  • People assign meanings to the physical spaces they use. How do these meanings change when built environments and the objects within them are globalized? How are people’s spatial interactions affected? Do communities benefit? Why?

Despite horizons in students’ writing that were narrower than I would have liked, I think the essay unexpectedly hit my second course objective, through the interaction with students in the ART 202 course and the IYRS Digital Materials and Fabrication program. These interactions required students in my course to communicate effectively with complete strangers who had, in many cases, unfamiliar perspectives and different goals. A prerequisite for learning how to solve problems that arise from global interconnectedness is actually connecting with people who are different, and that happened in this assignment.

Links to all posts in this series:

Changing a Course on Globalization, Part 1

Changing a Course on Globalization, Part 2

Changing a Course on Globalization, Part 3

Changing a Course on Globalization, Part 4

Changing a Course on Globalization, Part 5

Changing a Course on Globalization, Part 6

Changing Course on Globalization, Part 7