Putting your money where your mouth is

One of the more useful realisations in my professional life has been that my practice often advances best when I put myself in a somewhat awkward position. Thus, by committing to a conference paper, for instance, I lock myself into producing work that otherwise might stay little more than a latent idea in my head. In short, my sense of social obligation can be hijacked, especially by myself.

And so it has been with APSA TLC, which runs this week in Long Beach, CA. I’ve written and posted my paper,  and prepared my part of the short course, and generally tried to be a good citizen.

However, in the writing of the paper I came to the conclusion that one of the things that would be potentially useful to new users of simulations was a simulation on designing simulations (nb that sentence makes more sense if you read the paper): by getting people to actively engage with the difficulties of designing simulations, they would better understand them and come to recognise how to overcome them.

Having put this out there, I now find myself feeling in some way obliged to try and make that simulation, to demonstrate its viability (rather than simply leaving it as an abstracted thought). Indeed, the act of writing this blog further suggests to me that I’m trying to lock myself into this course of action.

My hesitation comes from two sources. Firstly, there is the rather ‘meta’ issue that such a simulation is going to be tricky to produce – which is exactly what the simulation is supposed to be about – but I fear that is more my problem than yours.

The second is simply one of time. APSA TLC starts on Friday this week; I fly out tomorrow (and I have several blockbusters I need to catch up with in-flight), by which time I’ve got to leave my affairs here in the office in order. This leaves hardly any time at all to work something up by the time of my presentation on Friday afternoon, especially if we assume I can find some sights to see in LA on Thursday (not to mention In-n-Out, for reasons too complicated to go into now).

But I’m going to give it a try. Partly that’s my social obligation kicking in, so that my fellow panelists don’t think I’m just being an armchair spectator. But partly it’s my own curiosity. Even since writing the paper I’ve been turning it over in my head and I think it would do me good to try and get it out.

It might not work, but I’ll have tried. And sometimes it’s the trying that matters.

The Digital Natives Are Not Restless

Somewhat related to Simon’s post about the use of new social media: Hurricane Sandy unexpectedly provided me with experimental environment in which to test student use online communication platforms. I’m currently teaching online courses for our master’s degree program in international relations. I have students on at least three continents and in who knows how many countries. Some reside locally and lost electrical power during the storm. A few might have evacuated to higher ground. Yet all of them were in communication with each other and me through the course websites, email, and phone. Assignments were submitted in a timely fashion even though I announced an extension of deadlines due to the weather.

I’m also teaching three traditional face to face undergraduate courses. All of these courses had tasks that could have or should have been completed using either the online Canvas LMS or the Statecraft simulation. Only six of my students submitted anything. I thought this was especially odd for the thirty-five students who are using Statecraft, since a new turn in the simulation began at 9:00 a.m. yesterday morning. Although classes were canceled, students were in the residence halls and the campus network remained operational. It appears that many undergraduates — at least at my university — still think of an education as something that is dependent on a physical classroom. While these students are quite happy to socialize via text message, they are not effective at using digital communication tools for other purposes. My older students — whose occupations frequently require problem solving  — are much more capable of operating in a globalized electronic environment.

Teaching ‘difficult’ subjects

As I mentioned last week, I was in Germany for a European Studies conference, including panels on L&T. A common theme which emerged was the notion of the European Union as a ‘difficult’ subject, both for students and teachers.

The former think of it as complicated and dull, while the latter are too often complicit in this, frequently focusing on the intricacies and so raising more barriers to students’ learning. Even when lecturers are positive, this generates suspicion about normative and political agendas on the part of students, who may not be as pro-EU as the typical academic. Thus the EU joins research methods towards the bottom of the pile of preferences for all involved.

This is clearly problematic, since both these areas are integral parts of contemporary political studies in Europe. Regardless of one’s attitude to them, it is essential to have a working understanding of methods and European governance.

The panels suggested two paths towards doing this.  The first (which I have long subscribed to) is that teachers need to start talking about the subject in a positive manner. This doesn’t mean proselytising, but rather focusing on the simplicity of underlying concepts that can act as heuristics to learning. Thus, EU isn’t any more complicated than a national political system, just as the relationship between EU and national institutions is a classic two-level game. Likewise, talking about referencing as transparency of sources can often illuminate much more than a bevy of rules on plagiarism.

The second path is to reframe the subjects much more radically. One colleague talked about how presenting the EU as a site for politics allowed students much easier access to the internal dynamics. In research methods, this is equivalent to problem-based learning, where students will need to use various methods to reach conclusions.

As our discussion went on, I realised that I have actually been doing this, largely without realising, in my negotiating politics module.  Here, several of the exercises are very generic political set-ups, often with no reference to any actual case, but can be applied to the EU very simply. From a two-level game on budget cuts (see here) to an exercise on the formation of political parties within a parliament, there is scope to approach the subject in a fresh way.

All of this should remind us that even the most ‘difficult’ of subjects (and perhaps, especially these ones) can be amenable to new approaches: we just need to step back and try again.

The Word Problem Problem

Recently I was faced with the question of whether to use some of the same books in two different courses. Isn’t each course supposed to have distinctive content? But what about the unity of knowledge? My internal debate eventually led me to conclude that when it comes to curriculum design, most university faculty — at least at the undergraduate level — are like failed restaurateurs. Everyone who likes cooking in the kitchen thinks they can run a restaurant. Every faculty member thinks designing a curriculum is the same as constructing a syllabus.

We often preach about the importance of critical thinking and claim that our courses and curricula help develop this skill. Yet students are often unable to transfer analytical  reasoning techniques from one context to another. The classic example of this is the word problem from elementary school:

  • A Japanese train with seven cars travels between Osaka and Tokyo at a constant speed of 200 km/hour. How long does the trip take for someone riding in the second car?
  • A Japanese train with four cars travels between Osaka and Tokyo at a constant speed of 200 km/hour and someone accidentally drops a book from the window of the second car. How long does the book take to hit the ground?

Students who have learned how to solve the first problem will frequently fail to solve the second one, even though they are essentially identical.* Why does this happen?

As pointed out by the cognitive psychologist Daniel Willingham, the human mind is lazy — it is predisposed to look for familiar patterns, and when it thinks it has found one, its decision-making ability is often negatively influenced. To get around this habit, a person must learn to differentiate between a problem’s superficial aspects and what Willingham calls its deep structure:

When one is very familiar with a problem’s deep-structure, knowledge about how to solve it transfers well. That familiarity can come from long-term, repeated experience with one problem, or with various manifestations of one type of problem (i.e., many problems that have different surface structures, but the same deep structure).

How does this relate to the readings I assign in my courses? If I assign the same text in multiple courses, students get repeated exposure to the same problem, and their ability to solve it is reinforced, or I can use the text to demonstrate how problems with the same deep structure can manifest themselves differently in different environments. Students also get more training in the knowledge that they should look for a problem’s deep structure instead of erroneously focusing on surface aspects (like the number of cars in a train or the train being in Japan).

How does this relate to curriculum design? Many curricula follow a checklist paradigm — students must take one Western civilization history course, two math courses, one religious studies course, ten courses in a major, etc. We like to claim that problem-solving techniques learned in an economics class can be applied to situations presented in an anthropology class, and vice versa, but the checklist sends the opposite message — that knowledge resides in discrete boxes. The way we design our curricula makes it less likely that students will ever learn how to see past surface structure and become effective problem-solvers.

*The solution to both requires knowing distance; in the former, the distance between Osaka and Tokyo, and in the latter between the window and the ground.

The Premortem

I regularly teach courses on economic development and complex humanitarian emergencies, and I often employ a case study approach. When students examine past events, it is easy for them — and for me — to fall victim to hindsight bias. We assume that the prospects for failure should have been just as evident in the past when decisions were made as they are to us when we are evaluating those decisions in the present.

Even if teachers take hindsight bias into account when explaining to students why incorrect answers are incorrect, we often see students making the same mistakes over and over again.

In his book Thinking, Fast and Slow, Daniel Kahneman credits fellow psychologist Gary Klein with the idea for engaging in a premortem when making decisions. The premortem is a critique that is crowd-sourced on a very small scale. It can also be described as a quick and dirty outside review. Here’s how it works:

  • A person presents a proposed decision to a group of people who are knowledgeable about the subject matter.
  • The group is told “Imagine that we are a year into the future. We implemented the plan as it now exists. The outcome was a disaster. Please take 5 to 10 minutes to write a brief history of that disaster” (Kahneman, p. 264).

I can see this technique being applied to all sorts of assignments, especially proposals for thesis research or project design. Although many undergraduates don’t have field research or public policy experience, a classroom of students should be able to think of and write about a few ways that the best laid plans can go awry. Including experienced students as a panel of experts during the ensuing discussion  — for example, students who have already conducted thesis research while studying abroad  — might make the process work even better.

How Soon Is Now? The End of the University As We Know It

The university as we know it is headed for extinction.

Imagine the U.S. higher education system in the early 1960s – universities were awash in government money because of the Cold War; enrollments were surging because of the GI bill and the baby boom. The paradigm of undergraduate education was the four-year, full-time, residential college experience, best suited for unmarried 18-22 year olds who could delay entry into the workforce. The university was the Emerald City in the Land of Oz – a fortress of knowledge where truth was revealed to humble supplicants.

Compare that with the current educational landscape. According to the U.S. Department of Education, approximately forty percent of undergraduate  students are older than 24. About half are classified as financially independent of parents. Forty percent attend college on a part-time basis. A huge percentage of students simply don’t have the time for the four-year, full-time, residential college experience that was the norm in 1960.

Simultaneously a bachelor’s degree has gotten a lot more expensive. In inflation-adjusted dollars, the average price of a four-year college education is 2.5 times what it was in 1980. During the same period, real wages for most Americans have been flat. People in the USA now need a college education just to maintain the same standard of living enjoyed by their parents and grandparents, but the cost of that education keeps going up. So more students go into greater amounts of debt to finance their educations.

As the traditional college model has become less convenient and more expensive, the opposite has happened to information. When I grew up, newspapers printed the news on paper, radio programs were broadcast over the air, and there were three TV channels. Then cable, satellite, and the internet completely disrupted the media industry. Today anyone can access almost any kind of news or entertainment at any time, and, perhaps even more importantly, anyone can create it – much of the content is user-generated. In many ways this transformation resembles Gutenberg’s invention of the movable-type printing press, which broke the monopoly on information that had existed in Europe for centuries.

The same is now happening to higher ed.

The Emerald City is being replaced by an information ecosystem that resembles a coral reef. There’s no moat, no wall, no gatekeeper controlling how the information is accessed or interpreted. It is open source and the architecture is scalable.

According to Wael Fakharany, Google’s regional director for Egypt and North Africa, worldwide there are 4 billion Google searches and 4 billion online video playbacks every day. Every second, someone uploads an hour of video to YouTube. These webpages, videos, and the ability to search them are free.

At this point you are probably thinking that 1) the internet is an unorganized and unreliable mess, and 2) undergraduate students still need the traditional university to organize, evaluate, and deliver educational content effectively.

Wrong.

People interested in learning have been migrating from free but unorganized educational online resources – the webpages that turn up in a Google search and the videos uploaded onto YouTube – to free organized information for several years. Now the movement is toward educational systems – housed at least in part online and frequently open-source – that offer convenient, affordable, and credentialed content mastery.

The simplest example of free organized information is Wikipedia, which started 2001. Wikipedia’s content is 100 percent user-generated.

A step up in terms of quality is MIT’s OpenCourseWare, which began in 2002 and currently has material from over 2,000 courses, such as classical mechanics.

Khan Academy, which began in 2006, now has over 3,000 video tutorials. Khan Academy’s motto is to provide a “free world-class education to anyone anywhere.”

The nonprofit educational foundation TedEd, a spinoff of TED, went live last month.

People can now use this free online content to learn anything, anywhere, anytime. There’s just one problem: how do you demonstrate what you’ve learned to others? If you walk into a job interview and say “I learned chemistry from Khan Academy,” the person on the other side of the table will simply shout “Next!” You need some sort of recognized, accepted credential signifying that you learned what you say you’ve learned.

Historically credentialing has been the function of the course credit and diploma offered by the traditional university. It has enabled universities to maintain an expensive monopoly on higher education, despite inroads by lower-cost but less prestigious for-profit institutions.

Until now.

Students who take the equivalent of an online self-paced correspondence course from the non-profit Saylor Foundation (motto: “education should be free”) and receive a certificate. So what, right? Students can then take an exam administered by the for-profit StraighterLine; if they pass they receive American Council on Education (ACE) credit, which then can be transferred to other educational institutions.

Still not impressed? This month, Saylor and StraighterLine began a partnership with Northern Virginia Community College and George Mason University to make college more accessible and lower the cost of completing a degree. The plan is to allow students to begin their college education at any of these four institutions; prospective students will know upfront the benefits and costs of each starting point.

A different method of unbundling undergraduate education was pioneered at Stanford by Sebastian Thrun and Peter Norvig last fall. Thrun and Norvig taught a massive open online course on artificial intelligence that enrolled 160,000 students. Students were ranked according to performance in the course and received a certificate upon completion.

This year, Norvig and Thrun started Udacity, which currently offers six free computer science courses and enrolls a total of 200,000 students. Udacity’s plan is to offer certification to students at an affordable cost.

A similar venture, MITxoffered its first free online course, on electrical engineering, this year. MIT has now partnered with Harvard to create edX, a $60 million nonprofit joint venture, to offer free online instruction worldwide. Like Udacity, edX will offer certificates demonstrating successful completion of courses for a small fee.

At this point, you might be thinking that these developments aren’t relevant, because you teach political science, or history, or English, not computer science. These innovations will affect you, if they haven’t already, in three ways.

First, higher education is headed toward a system of price discrimination that resembles a Starbucks menu. Elite universities with established brands will continue to draw students who have the necessary time and money for the traditional full-time, four-year campus experience. The mid- and low-tier brick-and-mortar schools will be forced, due to the increasing convenience and cost-pressure of open source alternatives, to redesign their curriculums or die. They will need to abandon uniform pricing for course credit so that they can hit as many price points as possible, and they will need to go after more undergraduate students who are older, employed full-time, and unwilling to come to campus three days a week between the hours of 9:00 a.m. and 3:00 p.m.

Second, the transformation of the higher education market will force universities to better demonstrate that the product they are selling justifies the price at which they are selling it. Each item on the educational menu will need to clearly reflect “you pay for what you get and you get what you pay for” because of what open online courses will do to credentialing. Grade inflation is rampant in American higher education, and the massively scaled competition offered by open online courses can produce a credentialing system that is far better than what brick-and-mortar universities – even many of the most prestigious ones – are offering. For example, let’s say that Cal Tech gets 10,000 applications a year but admits only 1,000. You’re admitted, you enroll, and eventually you graduate at the top of your class. In the end, though, you are still 1 of only 10,000. Now let’s say that instead of going to Cal Tech, you take a computer science course from Udacity, or a whole set of them, along with 500,000 other people. You end up being the top student. Best of 500,000 is likely to be a lot more meaningful to an employer than best of 10,000, and you didn’t have to pay $200,000 in tuition, room, and board to prove it. Think what that’s going to do to the universities that aren’t Cal Tech but that (for now) charge $200,000 for a bachelor’s degree.

Third, instructors will have to pay greater attention to whether their teaching creates value for the student and the university. Massive open online courses are going to generate huge amounts of data on teaching and learning. They’ll be able to demonstrate the effectiveness of particular teaching methods much more easily than the compartmentalized classroom environment. The pedagogies that are developed and tested online will filter into the physical classroom, and anyone who is not interested in continuous improvement in their teaching will lose students to the people who are.

Perhaps more importantly, course content will move out of the physical classroom and onto the internet. Open source initiatives like MIT’s OpenCourseWare demonstrate that course content is the least valuable educational component that universities have to offer, so it’s simply given away for free. The more valuable forms of learning that cannot be easily replicated online – research, project design and execution, problem solving, peer-to-peer learning – will take over the classroom. It will be more important for students to figure out how to get from Point A to Point B than for an instructor to tell them that Point B is better than Point A. The lecture at the podium will cease to exist.

As it moves online, educational content will be created and delivered in a distributed manner. Both students and instructors will become accustomed to feeding it forward – they will locate and analyze what others have done, adapt it for their own purposes, and then make what they’ve created available for others to use in their own learning.

Instructors who are unwilling or unable to engage with content or students in this manner will be at a disadvantage. Unfortunately most PhD programs currently do not provide training in this pedagogy to the graduate students who will be entering the teaching profession.

Disclaimer: I have no financial interest in any of the entities discussed above.Thanks to Sally Gomaa, members of the PeckNet, and Chana Motobu for their helpful comments. Any mistakes are the fault of my cats, who like to sit on my keyboard.

Do I mind about mind maps?

I’ve always had an ambivalent relationships with mind maps*.  Despite the frequent suggestion from various quarters about how great they are, lots of freeware online and their obvious value in helping to visualise relationships within a subject, I have tended to find them rather problematic.

This has been for two main reasons.  On the one hand, it tends to impose a hierarchy of organisation, working out from a central node, which tends to make the incorporation of cross-cutting themes at best tricky and at worst a complete mess.  On the other, I find that my way of understanding a subject tends to be more linear: I need a starting point to build an argument and a position.  As result, mind maps haven’t figured very much in either my teaching or my research.

However, that’s been changing a bit during this semester’s teaching.  One of my classes is a Masters module on EU policies, which I teach to a small group of about half a dozen students.  This has allowed me to get us all around a table to talk with either other, rather than a more formal arrangement.  In the second half of the module, we’ve been looking at individual policy areas and this is where the mind maps have shown their value.

This is my effort for the EU’s Common Security & Defence Policy and is my copy, essentially representing my notes for the class.  As we talked about the policy area, I re-created much of this diagram with additional points, working on an A1 flip-chart sheet.  as you can see, the map allows me to piece together some ideas and gives some focus to sub-sections within the topic.

From the students’ point of view, this has helped in capturing a relatively complex field and (particularly) in getting to understand how they might tackle the second phase.  This is a student-led session where they propose changes to the policy area in order to improve its operation.  The mind map lets them get a better sense of what the problems are and how potential changes might impact.

I’m still not totally sold on mind maps, but I can see that sometimes they are a very helpful tool.

    • This is exactly the kind of sentence that gives academics a bad name with the rest of the population.

More on Project-Based Learning

I’m still gathering information on project-based learning. My colleagues have approved a redesign of an interdisciplinary major, and the “culminating experience” course for seniors is changing from the traditional thesis to a research-based project. The course requires that each student’s project be problem solving-oriented, deliver a defined outcome with measurable effects, and be presented to the public.

Edutopia, a part of the George Lucas Education Foundation, has produced a very helpful guide on how to assess project-based learning. The guide is one example of how Edutopia is generating a tremendous amount of data-driven recommendations on active learning based on field testing in the K-12 environment. A simple example is the teaching of financial literacy to socioeconomically-disadvantaged students in Chicago.

Here, briefly, are the guide’s ten tips:

  1. Authentic, reality-based final products provide students with better ways to demonstrate what they have learned.
  2. Incorporate and assess “soft” skills, such as creative problem-solving and global awareness, to better prepare students for the future challenges they will face.
  3. When incorporating project-based learning into curricula, learn what the “big thinkers” in the field have to say about assessment.
  4. Because students will often be working on different tasks at different times, use formative assessment to ensure that students are mastering content and that students’ project development stays on track.
  5. Provide students with just-in-time feedback, in part to give them frequent opportunities to reflect on their reflecting.
  6. Teach strategies for effective collaboration, because students usually don’t know how to work well as part of a team.
  7. Embed assessment methods into the students’ use of digital tools; for example, an evaluated online discussion can result in greater participation among students who are reluctant to speak in the classroom.
  8. Provide students with an external audience; they will be more motivated to produce good work and will need to respond to challenging questions and criticism.
  9. Expand your repertoire of assessment methods and strategies through the free “do it yourself” professional development that is available on the Internet.
  10. Team up with colleagues — they are facing the same assessment-related demands you are.