Using Sherlock to Teach Sources v. the Literature

One of the challenges with students is that they often aren’t trained to recognize the difference between sources and the literature.  We may attempt to teach them the difference between primary and secondary, or scholarly and non-scholarly, but even amongst solid scholarly sources there are differences: there are the key works that make up the core of the literature on a debate, and there are the sources that sit on the fringes of the debate or never even quite enter into it.  Students think sources are something to use to prove their point, rather than a place to start out their process to discover what others have already discovered.  Even if you can get them to accept that scholarly sources are superior to, say, Wikipedia (a daunting task!), it can be difficult to explain to them that there is a difference between the literature and random sources, and that its important.

Enter Sherlock Holmes.

I’m a bit obsessed with the new BBC series called Sherlock, which if you haven’t seen it is a clever modern retelling of the stories.  Imagine Sherlock Holmes with a smartphone, and you have the heart of this series.  There is one particular scene in the first season that is useful in helping students understand the core of the above lesson.  In episode 3, starting at 19:53 (per Netflix instant), Sherlock asks Watson’s opinion of a pair of sneakers.  With encouragement, Watson notes some facts about them and their owner from the worn sole, clean appearance, and writing on the inside.  He asks Sherlock how he does, and he replies ‘Well, John.  Very well.  Of course you missed everything of importance.”  Sherlock then figures out the ‘important’ details–that the child had a skin condition, that he was from Sussex, and that he loved the shoes dearly.

I used this in my methods class today to get students thinking ‘What would Sherlock Do?’.  In other words, finding facts is not particularly difficult, but it also doesn’t tell us the whole story, and without the right facts, we are left completely in the dark about the phenomenon we are studying.  The right facts–or the literature–put us on the path of discovery.  Finding the right facts also means that Sherlock Holmes wouldn’t think of us as idiots…so again, ‘What would Sherlock Do?’  An amusing way to teach a simple but incredibly important lesson, and usable not only in methods, but any class where you expect students to use scholarly sources or do a literature review for a research paper.

“What’s that got to do with anything…?”

One of the constant challenges one faces in teaching is one of relevance.  Curricula are designed by committees of academics, who tend to work on the basis of “what does a student need to know in order to understand our subject?”  However, the number of students who learn a subject just in order to become invested in that subject just for itself is small: that many of those few that do, typically go on to become faculty in the future merely perpetuates the cycle.

Instead, university study is a means to an end, a necessary stopping point to some other putative career.  And this is where relevance really comes in: I have to recognise that very few of my students are going to become either political scientists or politicians.  This means that unless they are able to see that what they are learning has some relation to their lives, it is difficult to engage properly. Obviously, in PolSci this is often a comparative simple task, since the world around us provides ample justification: there’s always something we can draw by way of illustration.  But even then, I get students asking the question of the title: my relevance might not be theirs.

Part of this is timeliness – apposite examples do not always appear just when one is running a class.  With this in mind, I have tried a couple of different technologies that can serve to bridge this time gap.  Firstly, delicious offers a communal bookmarking system, to allow a collection of relevant material to be built up.  However, I have found it just that bit too involved to really get into, so I can’t really comment further.  By contrast, since becoming a twitter user late last year (@usherwood), the ease of posting up links to useful material, and the scope for interaction with other users has proved very good at flagging up timely and relevant content to students (and anyone else who follows).  By tagging tweets with module codes one can further direct people to what is relevant.

Again, twitter isn’t a panacea, but it does help to close some of the gap between what we think is important and relevant and what students think.

Incentivizing Active Teaching

Although I’m not an economist, I’m quite interested in identifying incentives, and faculty usually have few to no material incentives to experiment pedagogically.  Occasionally someone might receive a stipend or grant to vary one’s teaching methods, but these rewards are one-shot deals. Sometimes merit pay exists, but frequently it’s based on student evaluations of teaching, which is a recipe for disaster. This situation is particularly disturbing given the findings in Academically Adrift (Richard Arum and Josipa Roksa, U. of Chicago Press, 2011) that certain writing and reasoning skills fail to improve for over one-third of  students during four years of undergraduate education. Students aren’t learning, and faculty have no incentive to change that.

I’ve been thinking about this subject recently because I’m involved in an effort to redesign an interdisciplinary major. Team teaching has been proposed as a way of delivering content that crosses disciplinary boundaries — something that I wholeheartedly agree with — but at my university there are no incentives for it. The credit hours for a course that is team-taught are regarded as shared between instructors, and any teaching duty that equates to less than a complete three credit hour course is compensated as overload at a drastically reduced pay rate. Unless a full-time faculty member is really desperate for money, the cost of team teaching to faculty in time and effort is greater than the financial reward.

Soomo’s Supplements

Soomo Publishing has some neat online tools that can supplement courses in international relations and American politics.  They have a US Politics textbook, and then for both fields that have a neat set of supplementary exercises that are worth considering for intro classes.  For IR, they have ‘Between Nations’, a collection of video- and primary source-based assignments that can be mapped to a number of leading textbooks.  Each assignment includes a video clip or two or an excerpt (from the Leviathan, or the Melian Dialogue, etc) relevant to a major concept or issue in international relations.  The students then answer questions about the assignment–either multiple choice or short answer.  There is a gradebook that records their answers.  The assignments are also fully customizable and you can add your own into the system.  It costs $25 for students.

I’m trying it out for the first time in in my introductory class.  Each week the students have to complete one or two of them prior to class, and my hope is that the exercises will help familiarize them with the concepts so that class discussion is richer.  While in general I think that the short answer questions are better, I’ve chosen to use the multiple choice questions because they are automatically graded with no effort from me.  I’ve assigned 15 of these throughout the semester, but included a few extra that students can do for extra credit if they wish.

My hope is that, as a supplement to simply reading about IR, these exercises will result in a better learning process for the students in addition to enriching class discussion, but I will report back in a few weeks on whether my expectations are met or not.

Externalising knowledge

One of the main elements that we tend to stress to students is the process of internalisation, of bringing knowledge and skills inside oneself and of making one’s own.  At the same time, we also stress the ability to externalise, in the sense of being able to articulate that knowledge- and skill-set to others, most obviously through assessment.

This week, I have been finding myself in a similar situation, as I prepare my teaching notes for the coming semester.  Since I am of the opinion that teaching should not bear much resemblance to a murder mystery, I like to make my powerpoint slides for lectures available at the start of semester, so students can know roughly what I’m covering.  However, since both my modules this time round are new, I still don’t know exactly what I’m going to say. 

Indeed, in recent years, since I’ve gone further into different pedagogies – all of which stress student-led learning – I’ve found lectures more and more problematic.  Last semester’s use of podcasts was very good for that particular situation (allowing me more time to have practical activities) and highlighted the benefits of moving away from the passive classic lecture model.  In future years, I’m certainly going to look at moving all my lectures to podcasts.

This semester I have a somewhat different situation, since my Intro to the EU module is split into separate lecture and seminar slots.  While I know what each week will cover, I’m also aware that this has historically been a subject that suffers from a poor image and which students often struggle with basic concepts. While I will be using some activities such as ‘EU with t-shirts’ (below), I also want to retain some flexibility in my lectures to match student progress.

The upshot of this is that my lecture notes have become ever briefer.  While this merely confirms my status as the faculty member with the most telegraphic powerpoints, I’d like to think that it will also benefit students.  Instead of feeling that they can just take my powerpoints and use them for revision, they will instead need to attend my classes in order for those keywords to make real sense and for them to be able to internalise that for themselves.

That’s the theory: only time will tell if it pays off.

Buying in buy-in

After a very pleasant Christmas and New Year – marred only by over-eating on my part – we’re back in the saddle here, although our students now have four weeks of revision/exams before we start our new semester in February, so it’s a still rather quiet time.

Today is the deadline for the reflective portfolio that is the sole piece of assessment for my negotiating politics module.  In the UK, because of the quality assurance systems we have to use, formal assessment is subject to both internal second-marking (by another member of faculty) and external marking (by faculty from another university), to ensure national parity of grading.  This has meant that there is a strong pressure towards a small number of pieces of assessment for any given module, not least because more pieces also often means more regression to the mean, so a lot of the micro-assessments (as I’d see them) that we’ve seen discussed on this blog have not been viable options for me to use.

However, this doesn’t mean I can’t work quite flexibly within my one piece of assessment.  As I’ve noted in previous postings, I’ve set informal formative pieces of work to help students get into the reflective mode and, more importantly, I’ve talked consistently since October about the way in which the assessment can most simply be completed by working on it throughout the taught sessions, little and often.  Particularly for a subject such as this, timely reflection is essential.  Much as I’d love all my students to want to engage for the love of the subject, I’m enough of a realist to know that doesn’t always happen, so incentivising behaviour that works towards my goals is a key strategy.

My problem is this.  In the past week, I’ve had more queries about the portfolio than in the past two months (i.e. after the introductory session for the module).  This suggests two things: firstly, that in some cases students have left their work until the last moment, and secondly, that my incentivising hasn’t been incentivising enough.  Neither of these statements will cause too much surprise to anyone, but still it rankles that people don’t always choose the path of least resistance.

With this in mind, I’m going to be going back to the drawing board on assessment.  In particular, once I’ve marked the portfolios, I’ll be looking at the grades to see if any of the various things I tried through the semester can be picked up, in terms of improved (or worsened) performance.  Then I will have to decide whether having an intermediate submission point during the semester might be a helpful device in encouraging students to get work when they might best be able to the most out of their reflection.

Using Oral Exams in Political Science

Happy New Year to all.  I hope our collective breaks have been enjoyable and that we are reinvigorated for the start of spring term!  I know I could use a couple of extra weeks, but instead I’m finishing up a paper and getting ready for SPSA next week.

I’m a fan of oral examinations in our field.  One of the challenges students face in a written exam is interpreting exactly what we mean in the questions that we ask.  Language that, to us, appears clear and obvious, to them is vague and confusing.  This is even more the case for multiple choice questions, when there is not only a stem to interpret, but four or five possible choices.  No matter how clear we are, students–even, perhaps especially, those who are very prepared for the exam–can overthink the options and choose the wrong answer.  Another problem with written exams is that its not exactly a skill set that students need after graduation.  I can’t imagine many situations where, outside of formal education and standardized entrance/exit exams, they will be asked to prove their knowledge on the basis of prewritten questions that they do not know ahead of time.  I doubt many of us actually grade students on the sentence structure and grammar, so as an exercise in writing its not particularly useful either.

Oral exams, on the other hand, can accomplish the same goals as written exams–testing students’ knowledge and understanding of the material–but have some additional benefits.  As an interaction between professor and student, they can help us dig through the BS that fills exam answers and figure out whether the student really knows what they are talking about.  It also can help us clarify the question or ask it in a different way to help a struggling or confused student better understand what is being asked.  Finally, its pretty decent training for the post-school world, where you may have to defend a political argument before peers or supervisors in the course of a conversation.

We use an oral exam as the capstone project for our majors, called the Overview, so I instituted a mini-overview in my required methods class.  The Overview requires students to sit a 45 minute oral exam with two faculty members on a scholarly book, and face questions about the authors argument, values, and methodology and are asked to synthesize those with ideas and issues confronted in their coursework.  The mini-overview is a miniature version of that.  It acts as the midterm in my methods class, and students read a book chapter or article that I assign (the same one for each student) and sit a 15 minute oral exam with me.  In the course of the mini-exam, we focus almost entirely on methodology and analyzing the argument and I have students break down the variables, measures, methods, sources, etc.  Students also turn in an annotated abstract of 300 words for the piece, with the annotations noting the X, Y, theory, methods, and findings.

I only had 25 students in this class, so it is perhaps easier for me to do individual oral exams than it would be if I had 100 students.  Most of the exams were held during what would have been class time, so it did not take up too much of my time, and grading occurred immediately. If you have larger classes, you could consider training TAs to give these exams, as we trust them with grading anyway.

In general students performed better on this oral midterm than my previous group of students had on their written exam, and this was at least partly due to my ability to probe their responses for understanding, and encourage them to make vague responses concrete.  My first crop of students that underwent mini-overview take their capstone Overview this spring, so I will report back on its longer-term impact then.

The Results Are In

As I’ve discussed here and here, this past semester I ran a role-playing simulation for undergraduates on international relations in Europe on the eve of World War I. Previously I had tested for the simulation’s possible effects on student learning by comparing essay exam grades from a “regular” class that participated in the simulation to grades from an honors class that had not. Grades from the honors class were higher, and in most cases the differences were statistically significant, but the possibility existed that the higher scores were due to the honors students supposed better academic ability.

This semester I finally was able to teach an honors class again and do a more valid comparison. Some preliminary data crunching indicates that the simulation had no effect on essay assignments and exams, not what I’d expected. However, in a 2009 study, Stroessner,  Beckerman, and Whittaker found that students’ “writing ability, at least extemporaneous writing, was not affected” by Barnard’s Reacting to the Past role-playing simulations.* This makes me wonder what can be changed about history-based role-playing simulations so that students become better able to connect their simulation experiences to course content and demonstrate that connection through their writing.

I know someone reading this is thinking “it’s quite possible that your assignments and exams aren’t measuring what you think they are measuring.” Yes, possible, but logic dictates that if (a) we think it’s important for students to learn how to use theory to create a coherent and persuasive written explanation of an event, and (b) a question asks, for example, “did the international system in Europe prior to World War I best reflect liberal or realist IR theory, and why?” then (c) assignments and exams indicate how well students can accomplish (a). In other words, I’m testing for what I think students should know, and the simulation that I’ve been using doesn’t seem to have any effect on how much knowledge students acquire over a semester.

I will be presenting on this subject at the APSA Teaching and Learning Conference (TLC) in February. I’ve been attending the TLC  since 2007 and it’s been remarkably rewarding — lots of interesting and pedagogically practical information. I encourage you to attend.

*Stroessner, Steven J, Laurie Susser Beckerman, and Alexis Whittaker (2009) “All the World’s a Stage? Consequences of a Role-Playing Pedagogy on Psychological Factors and Writing and Rhetorical Skill in College Undergraduates,” Journal of Educational Psychology 101(3):605-620.

Return of the Sticks

It’s nearly the Christmas break here in England, but before I jet off to my chalet for a few week’s powder-skiing with some minor royals, (inevitably) Kim Kardasian and (oddly) Piers Morgan, I’m going to look forward to next semester’s teaching.

In particular, I’m looking at a new module that I have, an introduction to European integration for first-years/freshmen. My central aim in designing this module was to avoid the usual problem with this subject, namely a perceived high level of difficulty and an associated problem of engagement.  With this in mind, I’ve aimed to create learning spaces that privilege student activity.

Firstly, seminar activities are centred around the creation of one-page summaries of key concepts and events: students come with materials, then spend their time in the active construction of a collaborative piece, which is then shared with the rest of the students in other seminar groups.  The material will all be included in some form in the final exam, so giving students added incentive to produce their best work.

Secondly, to try and reduce free-riding, I’m getting my seminar leaders to use the old stick technique.  Each student has their name on a lolly-stick, and the seminar leader will randomly pick someone from the pile of sticks to write the summary, and another person to lead the discussion with a 5-min presentation.  This means everyone needs to be prepared to make that contribution.

Thirdly, and largely because I didn’t pay enough attention at the time, there is a mid-term multiple-choice test.  However, because I’m me, we’re doing it in class, not only open-book, but also allowing students to talk to each other.  With 45 questions in as many minutes, my anticipation is that students will discover that even in such benign conditions, there is no substitution for proper revision and preparation: weak students will spend so much time looking for answers that they can’t finish the test.  I can also see it becoming a test of altruistic behaviour.

So, plenty to look forward to and plenty to need on-the-fly adjustment.  I’ll be reporting back during the spring on this and other developments.  Until then, happy holidays!

What students want

It’s the final teaching week of the semester here at Surrey, so it’s wrapping-up time.  As well as trying to get students to connect their class-based work with events in the wider world, I was also interested in using the final session with them to get some constructive feedback on changes for the next time I teach my module on negotiation.

I’ve talked about the A-B-C feedback system I use before, so I’ll concentrate on what they’ve said.  Essentially, students are asked to suggest one thing to abandon, one to begin and one to continue.

If we remove what might be politely termed ‘outliers’ (e.g. classes in the pub, no assessment at all), then it is very interesting to see what came out.  While each of the individual games I ran had at least one student wanting to abandon it, there was a big group who liked the range provided, especially since it allowed them to practise a range of skills.  Similarly, opinion appeared to be split on the use of a reflective portfolio for the assessment, with some students wanting more structure to it, and others liking the flexibility.

In short, it shows that Lincoln’s adage about pleasing people is just as true in this context as any other and that even in a very student-led learning environment, it is essential for the teacher to provide direction and guidance.

Having said this, the session did bring forward one very good idea that had previously only nestled, half-formed, in the back of my mind.  This is simply to link together all the games into a coherent story (or arc, as I think we have to refer to it now in the age of the box-set), so students can develop into roles over time.  This would neatly solve the disjuncture between games, improve the capacity for preparation, bring out issues of iteration and, as such, was enthusiastically received by the class when we talked about it.  Of course, it also implies another complete re-write of my materials, but that might not be a bad thing.