Effective games

I found myself in a blind panic, spraying bullets wildly into mainly civilian populations, the other day.  For all concerned, it was just as well that this was part of the gameplay in Modern Warfare 2, but I was reminded of the presentation I’d heard last week by Mary Flanagan.

Mary was speaking at a workshop on simulations and game in politics teaching.  Her core message was that games contain value systems, intentionally or not: the design of a game conveys a world-view that the player has to engage with at some level.  Mary’s work has been about building games that create spaces for critical reflection on a wide range of political and social objects, and she presented a range of examples from immunisation to community-building to the consequences of job lay-offs.

The idea that games contain values is a very useful one for us all, not least because that is precisely what we are aiming for in educational games and sims.  Therefore, it’s important to think about how to harness this most productively.  Mary pointed to a number of elements in achieving this:

  • games need to allow for the use of strategy by the player, i.e. they cannot be purely linear.
  • games need to allow the player to make meaningful choices, i.e. they need to have consequences behind them.
  • games need to create situations where ‘the mechanic is the message’, i.e. the implicit structure of the game and how one plays conveys the idea you wish to communicate.
  • games need to provide a testbed for systems thinking, by creating an integrated experience for the player to visualise and internalise an approach.
  • and finally, games need to prepare players for zombie attacks (this might not be so central).

These are all excellent points for us to consider as we develop our own activities.  Certainly, the presentation helped to bring out much of the latent thinking I have done over the years and will spur me on.

To bring it back to MW2, it helped underline the very ambivalent approach to violence that the game embodies, with its situations of deep moral ambiguity (what do you do in the Moscow airport?) and its contestation of ‘good’ and ‘bad’ (by the end, who do/can you trust beyond your own team?).  I’m finally moving on to Black Ops (since I’ve now got a machine that can play it), so soon I’ll be covered on the zombie-thing too.

Asking the Right Question Before Creating a Good Answer

Like Simon, I’ve also been wondering “what are exams for?” I want my exams to function first as learning exercises and second as a means of formative assessment. Using exams as  summative assessment instruments ranks for me a distant third.

My exams usually consist entirely of in-class essays.  I provide students with a list of possible questions as a study guide; they get a shortened list for the exam and must select one question apiece from two or three subgroups. Students typically have already encountered some of the questions as short writing assignments that are tied to course readings.

I stress the development of argumentative skills when teaching, and my exams are no different. But students believe that exams should be either-or exercises in the regurgitation of factual knowledge, and they frequently complain that my grading of the arguments they have presented in their exams is “unclear” despite my use of rubrics. I should add that these are not high-stakes events — each exam is usually worth only ten percent of the final grade in the course — but because they are labeled “exams” they loom large in students’ minds.

Time for a change.

In the fall semester, I’m thinking of devoting classroom time on question development. I tried this during the spring semester and it didn’t work very well, possibly because I didn’t have students carefully and deliberately identify what makes one question better than another.

Once students have created an appropriate exam question, they can then determine what is necessary to adequately answer it; for example, by using at least three pertinent examples drawn from reading assignments. This will hopefully lead them into the creation of a rubric, a greater sense of personal investment in and responsibility for the exam process, more frequent interaction with information they will be tested on over a longer period of time, and more learning.

Being a winner

In my mental image of me, I’m not the sort of person who wins things.  So when I do win something, I’m quite chuffed. Yesterday, I got to the status of being very chuffed indeed to win the nanoteach prize at the Higher Education Academy’s Social Science conference in Liverpool.

The nanoteach competition asked people to come with a single, short idea about Learning & Teaching, to be shared in no more than 2 minutes. The delegates then got to vote on their favourite.

It would be fair to say that I was anxious about doing something for this.  I’d never tried it before and I wasn’t quite sure what kind of thing the organisers were looking for.  So how did I manage it?

Firstly, I thought about my audience.  You and I are busy people, who haven’t got time for anything too involved or complex to remember.  So it needed to be something that spoke to that.

Secondly, I thought about the context.  It’s a conference, at the end of the day, and there’s wine on the tables, so it can’t be too solemn.

Thirdly, I thought about how I could make a connection. Most of the people weren’t political scientists, so it couldn’t be disciplinary, just as it couldn’t be university-based, since people were also coming from a wide range of sectors.

With all that in mind, I still messed up.  My original idea was to talk about the sticks as a means of randomising contributions. It is visual, it speaks to a problem that’s generic and it’s a bit quirky.  But as the day went on, I realised that it wasn’t right for this: it’s actually quite involved – certainly for a 2 minute presentation – and as I’ve noted here before, it’s not without caveats about its use.  So this is where the final element came into play: I got lucky.

The nanoteach session was ordered by lots and I was able to listen to the first couple of presentations, then pop out of the room, find three post-it notes and dash back in. I then got lucky again, as my name didn’t come up until the very end, so I could collect myself and – importantly – gauge the audience more closely.

By the time I stood up and told them about the ABC system for feedback, I knew that I was giving them a much better idea than the sticks.  It’s super-simple, very easy to remember (“as easy as ABC…” indeed), useful for everyone who needs feedback on anything, and communicable in visual form.

I’m neither going to pretend the ABC idea is my own, nor that it’s the best idea that was presented last night (I’ve got some things I’ll be trying in class when I get back), but it was pitched right for that moment.  Just as our teaching needs to be responsive to our students and their specific needs, so that rest of our pedagogic interactions need to reflect that too.

So thanks to the HEA (for the prize), to my fellow delegates (for the votes) and to the colleagues who gave me the idea in the first place.  This winning thing is rather enjoyable!

Political Scientists: Marching Toward Oblivion?

A quick blurb about something related to my post on the coming extinction of the university as we know it:

The June issue of Perspectives on Politics has an article by Lisa Anderson, president of the American University in Cairo, in which she discusses the increasing irrelevance of political scientists. I’m going to quote her at length (page 386 of the article):

The “transformation in the availability of information has eroded authority, undermined hierarchies, and upended the organizational mechanisms by which knowledge is developed, collected, and disseminated around the world. This has meant a decline in interest in and deference to universities and university-based research and has lessened popular estimates of what the university-based political scientist has to offer to students, citizens, politicians, and policy makers. If we as political scientists are to regain the esteem of the public, it will be because we have engaged them on new terms—the terms of a twenty-first century in which information is abundant and the sort of formal credentials we university-based political scientists have all worked so hard to secure are relatively inconsequential. What we must be prepared to do is engage our students and colleagues as well as our fellow citizens and policymakers, in the relatively informal, non-hierarchical networks of collaboration, reciprocity, and shared wisdom in which the next generations are already beginning to live, learn, and work . . .

As political scientists, we need to be far less attached to a view of ourselves as uniquely knowledgeable; we are no longer the repositories and carriers of rare and therefore valuable cultural and scientific information. We must be far more willing to see ourselves as guides, advocates and facilitators of learning.”

What are exams for?

For most people, the question posed in the title is either blindingly obvious or completely left-field: exams are to test people, duh.  However, you are most people so let’s try asking the question again: what are exams for?

 I ask for the simple that it’s a question which bothers me every time I encounter an exam.  It’s all the more pressing this semester, as some of my students are sitting an exam that I’ve written. It’s a bog-standard, 2-essay-questions-in-2-hours, closed book exam.

 This is perhaps a key part of my doubt about exams: they end up being a very standardised piece of assessment, relatively unvarying in format and purpose (i.e. getting students to pull together their learning into relatively concise, quick-to-mark answers). One might argue (as people have argued to me, usually when I’m their external examiner) that this predictability of form allows students to refine their technique and maximise their performance.  In brief, their attraction lies as much in their ease of administering as in their pedagogic value.

What particularly troubles me is that exams have very limited real-world utility: when do you ever find yourself stuck in a room with no access to other materials and get told to produce a reflective analysis on a topic that you previously only knew roughly which aspects to prepare beforehand?  Given that we spend as much time as we do asking students to become good managers of information and efficient and effective researchers, why then do we provide them with an assessment form that doesn’t let them demonstrate this?

Obviously, there is a baby/bathwater issue here. If we have an efficient way of assessing (that also properly captures student ability), then we should use it: none of us has unlimited time.  As I’ve discussed before, you can have open book exams, or seen exams (where the student is told what the questions will be some time in advance), or (I suppose) group exams.  These would certainly add something to the mix and encourage some additional skills and abilities.

I do recognise that this is also partly a matter of taste: as a student, I prefered writing essays to exams, even if my performance was as good in the latter.  But still we need to come back to the original question, albeit in a different way: what do we want to assess and what’s the best way to do that?

More Thoughts on Modular Course Architecture

My spring semester is over. Undergraduate students have completed an anonymous evaluation of my comparative politics course, in which I experimented with modular architecture. In this course, students chose one of five different themes to focus on for the semester — political identity, democratization, revolution, genocide, or globalization. The themes were cross-indexed by geographic region; for example, in a particular week, all students’ readings were about Latin America. At the beginning of the semester, all students read journal articles on all five themes as an introduction.

A few non-scientific conclusions about what students wrote in their evaluations:

A few students — despite regular reminders — refuse to acknowledge the importance of  the “plan ahead” concept. Yes, this book is 300 pages long. No, it’s not something you can read the night before the essay on it is due. That’s why I give you a syllabus at the beginning of the semester.

Some choice is good, but too much is bad. The reading assignments for the genocide theme were all books. I let each student in this theme choose three books to read, which meant that a given student wasn’t obligated to read anything for some of the geographic regions. A few students said they felt lost in class discussions as a result.

Groups needed to have members that were all doing the same thing. Throughout the semester, students had to do group presentations that consisted of close-readings of assigned texts. Students did not select the themes in equal numbers, so some of the groups that I created contained students who had chosen different themes. Students in these mixed groups said this hampered collaboration — one or two members of the group would contribute nothing. My expectation was that students would teach content to each other while they were putting their presentations together. So that didn’t work out. In the future I will either need to drop this kind of collaborative task or figure out a way to ensure relatively equally-sized groups, each containing students who have chosen the same theme.

Class discussions helped students see connections between different historical events and apply theoretical concepts to new situations.  One of my personal goals as a teacher is to facilitate students’ ability to  integrate knowledge, so I would like to develop more formal ways of doing this — graded writing assignments, if possible — in the future.

Sticking it to the man

We’ve already reached the last week of teaching this semester, so it’s been time for me to round up my feedback from students.  In particular, I’ve been keen to find out what they thought of both my use of sticks to randomly select seminar presentations and the open book multiple-choice exam.  As before, I started from an ABC exercise to generate some data to open the conversation.

In both cases, the students were very happy to discuss the pros and cons, which suggests that they have at least engaged at that level.

With the sticks, there was a considerable amount of unhappiness about their use. “It’s unfair”, “they make people not attend”, “if people haven’t prepared very well, then we just sit around doing nothing” were some of the comments.  At the same time, several people had made positive comments about the random nature of the system and the way it had made them prepare more than they otherwise might have done.

Reading more widely across the comments, a couple of people noted that seminar leaders hadn’t been consistent in ejecting those who hadn’t prepared and my feeling from the discussion was that this might have been a key part of it all: if there was strong confidence about others preparing, then students would themselves prepare.  This is certainly one point that I will be taking into the next iteration.

On the MCQ, the picture was less clear.  Some pointed to all the assessment being test-based, when they would prefer at least one essay.  Others liked the MCQ, but not the open-book part of it: “it’s not fair that I got a high mark” was one telling comment.  Others still obviously enjoyed the experience, although how much that is a reflection of the high marks I cannot tell.  The argument about essay-based assessment is a fair one and fits my general tendencies in this, but for now my inclination is to try it for another year, perhaps without the use of computers or smart-phones, or even more conventionally closed-book.

Beyond this, some of the comments seemed to suggest that these (first year) students hadn’t all grasped that they are not expected to be active learners, rather than people to be taught.  They enjoyed my lecturing style and the (metaphorical) ‘love’, but wanted more detailed powerpoint slides and discussion focused on revision/assessment topics, as well as more direction on what they needed to know.

In all of this there is the central difficulty of balance, between giving students want they want and giving them want they need.  As I said in the class, I’m always happy taking on feedback, because it’s the best way for me to square that particular circle.

How Soon Is Now? The End of the University As We Know It

The university as we know it is headed for extinction.

Imagine the U.S. higher education system in the early 1960s – universities were awash in government money because of the Cold War; enrollments were surging because of the GI bill and the baby boom. The paradigm of undergraduate education was the four-year, full-time, residential college experience, best suited for unmarried 18-22 year olds who could delay entry into the workforce. The university was the Emerald City in the Land of Oz – a fortress of knowledge where truth was revealed to humble supplicants.

Compare that with the current educational landscape. According to the U.S. Department of Education, approximately forty percent of undergraduate  students are older than 24. About half are classified as financially independent of parents. Forty percent attend college on a part-time basis. A huge percentage of students simply don’t have the time for the four-year, full-time, residential college experience that was the norm in 1960.

Simultaneously a bachelor’s degree has gotten a lot more expensive. In inflation-adjusted dollars, the average price of a four-year college education is 2.5 times what it was in 1980. During the same period, real wages for most Americans have been flat. People in the USA now need a college education just to maintain the same standard of living enjoyed by their parents and grandparents, but the cost of that education keeps going up. So more students go into greater amounts of debt to finance their educations.

As the traditional college model has become less convenient and more expensive, the opposite has happened to information. When I grew up, newspapers printed the news on paper, radio programs were broadcast over the air, and there were three TV channels. Then cable, satellite, and the internet completely disrupted the media industry. Today anyone can access almost any kind of news or entertainment at any time, and, perhaps even more importantly, anyone can create it – much of the content is user-generated. In many ways this transformation resembles Gutenberg’s invention of the movable-type printing press, which broke the monopoly on information that had existed in Europe for centuries.

The same is now happening to higher ed.

The Emerald City is being replaced by an information ecosystem that resembles a coral reef. There’s no moat, no wall, no gatekeeper controlling how the information is accessed or interpreted. It is open source and the architecture is scalable.

According to Wael Fakharany, Google’s regional director for Egypt and North Africa, worldwide there are 4 billion Google searches and 4 billion online video playbacks every day. Every second, someone uploads an hour of video to YouTube. These webpages, videos, and the ability to search them are free.

At this point you are probably thinking that 1) the internet is an unorganized and unreliable mess, and 2) undergraduate students still need the traditional university to organize, evaluate, and deliver educational content effectively.

Wrong.

People interested in learning have been migrating from free but unorganized educational online resources – the webpages that turn up in a Google search and the videos uploaded onto YouTube – to free organized information for several years. Now the movement is toward educational systems – housed at least in part online and frequently open-source – that offer convenient, affordable, and credentialed content mastery.

The simplest example of free organized information is Wikipedia, which started 2001. Wikipedia’s content is 100 percent user-generated.

A step up in terms of quality is MIT’s OpenCourseWare, which began in 2002 and currently has material from over 2,000 courses, such as classical mechanics.

Khan Academy, which began in 2006, now has over 3,000 video tutorials. Khan Academy’s motto is to provide a “free world-class education to anyone anywhere.”

The nonprofit educational foundation TedEd, a spinoff of TED, went live last month.

People can now use this free online content to learn anything, anywhere, anytime. There’s just one problem: how do you demonstrate what you’ve learned to others? If you walk into a job interview and say “I learned chemistry from Khan Academy,” the person on the other side of the table will simply shout “Next!” You need some sort of recognized, accepted credential signifying that you learned what you say you’ve learned.

Historically credentialing has been the function of the course credit and diploma offered by the traditional university. It has enabled universities to maintain an expensive monopoly on higher education, despite inroads by lower-cost but less prestigious for-profit institutions.

Until now.

Students who take the equivalent of an online self-paced correspondence course from the non-profit Saylor Foundation (motto: “education should be free”) and receive a certificate. So what, right? Students can then take an exam administered by the for-profit StraighterLine; if they pass they receive American Council on Education (ACE) credit, which then can be transferred to other educational institutions.

Still not impressed? This month, Saylor and StraighterLine began a partnership with Northern Virginia Community College and George Mason University to make college more accessible and lower the cost of completing a degree. The plan is to allow students to begin their college education at any of these four institutions; prospective students will know upfront the benefits and costs of each starting point.

A different method of unbundling undergraduate education was pioneered at Stanford by Sebastian Thrun and Peter Norvig last fall. Thrun and Norvig taught a massive open online course on artificial intelligence that enrolled 160,000 students. Students were ranked according to performance in the course and received a certificate upon completion.

This year, Norvig and Thrun started Udacity, which currently offers six free computer science courses and enrolls a total of 200,000 students. Udacity’s plan is to offer certification to students at an affordable cost.

A similar venture, MITxoffered its first free online course, on electrical engineering, this year. MIT has now partnered with Harvard to create edX, a $60 million nonprofit joint venture, to offer free online instruction worldwide. Like Udacity, edX will offer certificates demonstrating successful completion of courses for a small fee.

At this point, you might be thinking that these developments aren’t relevant, because you teach political science, or history, or English, not computer science. These innovations will affect you, if they haven’t already, in three ways.

First, higher education is headed toward a system of price discrimination that resembles a Starbucks menu. Elite universities with established brands will continue to draw students who have the necessary time and money for the traditional full-time, four-year campus experience. The mid- and low-tier brick-and-mortar schools will be forced, due to the increasing convenience and cost-pressure of open source alternatives, to redesign their curriculums or die. They will need to abandon uniform pricing for course credit so that they can hit as many price points as possible, and they will need to go after more undergraduate students who are older, employed full-time, and unwilling to come to campus three days a week between the hours of 9:00 a.m. and 3:00 p.m.

Second, the transformation of the higher education market will force universities to better demonstrate that the product they are selling justifies the price at which they are selling it. Each item on the educational menu will need to clearly reflect “you pay for what you get and you get what you pay for” because of what open online courses will do to credentialing. Grade inflation is rampant in American higher education, and the massively scaled competition offered by open online courses can produce a credentialing system that is far better than what brick-and-mortar universities – even many of the most prestigious ones – are offering. For example, let’s say that Cal Tech gets 10,000 applications a year but admits only 1,000. You’re admitted, you enroll, and eventually you graduate at the top of your class. In the end, though, you are still 1 of only 10,000. Now let’s say that instead of going to Cal Tech, you take a computer science course from Udacity, or a whole set of them, along with 500,000 other people. You end up being the top student. Best of 500,000 is likely to be a lot more meaningful to an employer than best of 10,000, and you didn’t have to pay $200,000 in tuition, room, and board to prove it. Think what that’s going to do to the universities that aren’t Cal Tech but that (for now) charge $200,000 for a bachelor’s degree.

Third, instructors will have to pay greater attention to whether their teaching creates value for the student and the university. Massive open online courses are going to generate huge amounts of data on teaching and learning. They’ll be able to demonstrate the effectiveness of particular teaching methods much more easily than the compartmentalized classroom environment. The pedagogies that are developed and tested online will filter into the physical classroom, and anyone who is not interested in continuous improvement in their teaching will lose students to the people who are.

Perhaps more importantly, course content will move out of the physical classroom and onto the internet. Open source initiatives like MIT’s OpenCourseWare demonstrate that course content is the least valuable educational component that universities have to offer, so it’s simply given away for free. The more valuable forms of learning that cannot be easily replicated online – research, project design and execution, problem solving, peer-to-peer learning – will take over the classroom. It will be more important for students to figure out how to get from Point A to Point B than for an instructor to tell them that Point B is better than Point A. The lecture at the podium will cease to exist.

As it moves online, educational content will be created and delivered in a distributed manner. Both students and instructors will become accustomed to feeding it forward – they will locate and analyze what others have done, adapt it for their own purposes, and then make what they’ve created available for others to use in their own learning.

Instructors who are unwilling or unable to engage with content or students in this manner will be at a disadvantage. Unfortunately most PhD programs currently do not provide training in this pedagogy to the graduate students who will be entering the teaching profession.

Disclaimer: I have no financial interest in any of the entities discussed above.Thanks to Sally Gomaa, members of the PeckNet, and Chana Motobu for their helpful comments. Any mistakes are the fault of my cats, who like to sit on my keyboard.

More Lego-based learning

This weekend I’ve been learning about learning from a young boy of my acquaintance.  Upon arrival at a weekend of wedding celebrations for his aunt, he discovered that while he had brought his recently-acquired Lego X-Wing starfighter – dismantled into its 400 or so individual bricks – he had left the instructions back home.

After some flumping about, he was encouraged to try building it from memory, since he has already built it and dismantled it (with instructions) about ten times already.

2 hours later, he was finished.

I talked to him about this, since I was intrigued as to his approach, not least since he had clearly not been specifically prepared for such a task (a common issue when learning).

He explained that some parts were very easy/memorable and didn’t require any thought as to their construction.  Secondly, he noted that there were sub-assemblies he could work: blocks of bricks that naturally belonged together and which could be dealt with, then put to one side until they could be joined on.

However, the main part of the model didn’t work this way, since it was an integrated design that needed to be done in a particular order. Here was where I found the most interesting insight, namely that he had thought about the intent of the finished piece: what was it that each piece contributed to the overall design?

As always, this made me think of how our students acquire and use their learning.  the key difference here is that students don’t get asked to exactly reproduce an output (or rather, they shouldn’t be).  But they do have to engage in novel exercises that might be out of their direct experience, extrapolating from what they have previously done.  Just as this boy progressively moved away from the instruction booklet as the weekend progressed on his iterations of building, so students develop their confidence in using knowledge and skills through use.  And as instructors, we need to think about how we can facilitate this.

Googling

The importance of information literacy can be difficult for undergraduates to grasp. For most of them, all information is created equal and it comes from Google. Now Google is helping to educate students out of these habits. The website includes sample lesson plans and videos for instructors who want to integrate information  literacy training into face-to-face or online courses.

Instructors can also use A Google A Day to test students search skills — it’s fast, fun, but not necessarily easy.