Closing the Loop

DecisionSimon, in his post last week, explained the need to be as transparent as possible about what we do by explicitly identifying to students how our actions relate to their learning — for example, by telling students that the feedback we provide is in fact feedback. Just as importantly, students need to be more aware of how what they do affects their learning. Students need to realize, as the saying goes, that instructors can lead students to knowledge but can’t make them drink it.

I’ve been using the end-of-semester quality of failure assignment, which first came to my attention when Amanda posted about it,  in all my undergraduate courses. The quality of failure essay has been a great success; I have been tremendously impressed with students’ willingness to honestly evaluate their efforts in a course.

However, analyzing failure is self-assessment ex post facto; it would be nice to bookend this with an ex ante instrument at the beginning of a course.

Evergreen State College’s academic statement provides a model. Evergreen’s academic statement is a process of formative assessment that extends across four years of college. It asks each student to reflect annually on his or her educational choices, why they were made, and whether he or she is attaining his or her academic goals. The objective is to make students more aware of and more deliberate in the choices they make while in college.

So I’m now making plans for a similar writing assignment for the start of the semester in each of my courses. The assignment will ask students to write about:

  • What do you plan to learn in this course?
  • Why does this knowledge interest you?
  • What will you need to do to learn the knowledge that you want to learn?
  • How do your goals for this course correspond to how you are living your life?

At the end of the semester, when students are reflecting on how and why they’ve failed, and what they’ve learned from their failures, I’ll point them back to what they wrote at the beginning of the semester.

Navel Gazing

Navel OrangeAt the Edutopia parent partnership blog, Terry Heick has posted a list of nineteen questions parents should ask their children’s K-12 teachers. The list inspired me to think of ten questions that university students should ask themselves at the start of any new course:

    1. What do I want to get out of this course, and what do I need to do to achieve that?
    2. How will my academic strengths and weaknesses affect my ability to achieve my goal?
    3. What can I do to improve my understanding of the subject beyond the minimum that is required?
    4. How will I respond if I start to struggle in this course?
    5. How are the different ways that I will be evaluated in this course connected to what and how I can learn?
    6. What are the available resources for learning that I can use to my advantage? What must I do to use these resources?
    7. What are the barriers I often create that diminish my learning, and how can I change these behaviors?
    8. How can I shape my interactions with fellow students to increase my (and their) learning?
    9. What is the role of the instructor in my learning process?
    10. What am I not asking but should be?

It might be worthwhile to give students a list like this on the first day or two of class, and then give it to them again at the end of the course as a prompt for writing the quality of failure essay.

Anti-Fragile Universities

I recently had a conversation with a retired IT administrator from Boston University. He commented that people in universities, just like in most other organizations, usually operate to preserve the existing order. This reminded me of how difficult it is to alter the curriculum, whether by introducing new content, modifying its architecture, or eliminating failed programs. People who work in universities don’t like to replace something that isn’t working with something else that might work a lot better.

Yet we tell students something else entirely: that they should have the courage to fail, and to learn from it. And any system — whether a business or a living organism — grows weak if its constituent parts are not allowed to fail. It’s much better to fail often and fail fast, because learning what doesn’t work early on and responding to this information will minimize harm and generate the largest benefits.

In other words, to paraphrase a regional director from Google whom I’ve heard speak on innovation:

  • Move fast
  • Experiment, but make sure to measure what you’re doing
  • Accept that you might be wrong
  • Focus on results and use a data-driven approach to make difficult decisions
  • Cut your losses without remorse

Evolution functions according to these principles. Universities generally do not. And this idea led me to revisit Nassim Taleb, whose work, at least on the non-technical side, I was already somewhat familiar with. In the changing landscape of higher education, many universities now fit his description of a fragile system, which is represented by the graph below.

The vertical axis is relative gain or loss in whatever the system is dependent on. For businesses, and that includes universities, this is revenue. The horizontal axis is delta, or change in some input. For a standard business, that means sales. For a university, it’s basically students.   The graph shows a system that is at high risk of catastrophic failure if there is a relatively small negative change in inputs.

Fragility Graph

Let’s say a university enrolls five percent more students — move a certain distance to the right on the horizontal axis — and there’s a corresponding increase in net revenue. The green line goes up. For the purposes of example, let’s say revenue increases by $20 million. If we make delta negative — and we decrease enrollment by five percent — the revenue change is not -$20 million, it’s much greater. The red line drops farther and faster than the green line goes up. There’s a second order effect in which the downward rate of change accelerates. So the university, or whatever system you might be looking at, is at extreme risk if there is a relatively small unfavorable perturbation in the environment.

So what’s the solution? Universities should do what we tell our students to do. They should be constantly experimenting to figure out how they can become less dependent on their traditional products and services — the markets in which competition is the most aggressive  — and enter market niches that lack dominant players. This might mean continuously and ruthlessly testing new courses, delivery formats, degree programs, or even educational services that aren’t oriented around completion of a degree.

The Success of Failure

A quick note to say that the quality of failure assignment that I instituted in my undergraduate courses this semester has succeeded beyond my wildest dreams. The vast majority of my students responded to the assignment with great insight. In case others want to use it, here are the directions for the assignment (modified from the original source, which can be accessed via the link above to Amanda’s post):

This course requires realizing that progress requires curiosity, risk-taking, and failure. Making a mistake leads to the question “Why was that wrong?” and by answering this Lohan question, we are better able to develop new insights and eventually succeed. You’ll need to fail regularly to do well in this course because part of your final grade is based on your “quality of failure.” At the end of the semester, you’ll need to write a 2-3 page double-spaced essay analyzing your failures, why they occurred, and what you have learned from them. Your essay must conclude with an assessment on how you have evolved through your mistakes in the course (a grade that ranges from 0 – meaning “I never failed” or “I learned nothing from failing” to 10 – meaning “I learned in new and creative ways from my failures”).

I believe that this type of assignment could be very usefully made part of student evaluations of teaching.

Dare to Be Wrong, In Practice

Greetings from Geneva Switzerland, where I am presenting at the 18th International Humanitarian Conference, an annual event put on by my colleagues at our Geneva campus, this year on Access to Health.  I had the chance to put one of our favorite topics, teaching failure, into practice while here.

The opening night of the conference, which is aimed at students and the community, includes a roundtable conversation on campus with Webster students on a human rights topic, usually linked to the year’s chosen theme by our Institute of Human Rights and Humanitarian Studies.  This year the topic was statelessness, and I was asked to be on the panel.

The students–a mix of undergraduate and masters-level graduate students–are encouraged to participate, but in years past have usually just sat back and let the faculty talk.  This year looked to be much of the same–the chair asked a number of good questions, but for whatever reason–potentially including the presence of unfamiliar faculty–they were unwilling to do so.  I asked for the mic, then, and encouraged them to fail–to ‘dare to be wrong’, as I put it, but that joining the conversation was worthwhile, that this was a safe space to do so, and that learning only comes through being willing to fail.

I don’t know if it was my encouragement that got them talking*, but there was a fairly spirited discussion after that about issues of statelessness, identity, and nationality.  I like to think that it helped.  It also may have helped that the faculty disagreed with each other, showing the students that there is no single ‘truth’ to find.  What I can say for sure is that a roundtable with lots of student participation is light-years better than one where the faculty completely dominate the discussion.

*Edited to Add: I spoke with a couple of the students today and they said that my comments did really help them get over their insecurity about speaking.  So there’s some anecdotal data, for what it is worth.

Tinkering

When I was a doctoral student, I once spent several weeks teaching Asian history to teenagers at Barrack Obama’s former high school. I vowed never again to put myself in the position of having to prep for class at 1:00 a.m. five nights a week. This is why I’m already tinkering with syllabi for the courses I’ll be teaching in the spring.

In comparative politics, I’m going to repeat my experiment with modular architecture, but I’ve removed the globalization theme — the topic has morphed into an entirely separate course, and conveniently I’ll be able to some of the material I put together last year.

I’m also going to continue using rocket pitch competitions, but with individual rather than group presentations. I’ve noticed that teams of students haven’t figured out how to productively generate a presentation — they tend to share tasks equally across all members of the team, rather than delegate and let people utilize their strengths. The end result is four students standing in the front of the room alternately talking (this is despite my use of Shark Tank as an example of what not to do).

Based on my colleague’s recommendation, part of the final grade will be based on the student’s quality of failure. Here is the syllabus language I’m using, based on what was published in the original Inside Higher Ed column:

This course requires realizing that progress requires curiosity, risk-taking, and failure. Making a mistake leads to the question “Why was that wrong?” and by answering this question, we are better able to develop new insights and eventually succeed. You’ll need to fail regularly to do well in this course because part of your final grade is based on your “quality of failure.” At the end of the semester, you’ll need to write a 2-3 page double-spaced essay analyzing your failures, why they occurred, and what you have learned from them. Your essay must conclude with an assessment of the learning you have gained through your mistakes in the course (a grade that ranges from 0 – meaning “I never failed” or “I learned nothing from failing” to 10 – meaning “I learned in new and creative ways from my failures”).

Teaching Failure

Everyone should check out this fascinating piece on Inside Higher Ed, about how to help students learn from failure.  The instructor reserves 5% of the final grade for ‘quality of failure’, assessed by a reflective essay at the end of the course.  Students are encouraged throughout the course to try out new ideas and to fail spectacularly, and to use those moments as learning tools for the entire class.

I love this idea.  Like many of us, I struggle with getting students who feel like they don’t know anything to participate in class.  My way around it has been to find ways to teach political concepts through individual experiences, pop culture, movies, and books and to create a comfort zone for discussion and to help students see the connections of politics to their real lives.  This is easier in some courses (like methods) than others.  For content-heavy courses, we still have to deal with the actual content, and getting students to feel comfortable engaging with ideas they do not quite understand can be rough.  Creating an environment where failure is institutionalized and rewarded sounds like a great way to overcome some of these problems.

The new semester started yesterday and in 25 minutes my first methods class meets.  I am sorely tempted to reserve part of their participation grade for this ‘quality of failure’ idea.  Its certainly a risk.  But as the author of the piece says, modeling failure ourselves through our teaching choices can be rewarding to the students, so perhaps this is a risk that will work out regardless of how it goes…