Perils of Small (Online) Courses, Part 2

How I see my course design
How I see my course design

Two weeks ago I wrote about the lack of discussion in two online graduate courses with a combined enrollment of eleven students. These two courses ended yesterday and I’m now taking a break from grading final exams. Discussion decreased to zero in the last week of one of the courses and remained paltry in the other.

I created an ungraded, anonymous survey to try to find out what students thought was most useful about these courses. Perhaps how I think about course design, which includes weekly discussion, is very different from how students think about it. However, only two people responded to the survey, so I have to guess both about the causes of the problem and about potential remedies.

None of these remedies are very attractive to me. I could increase the weight of discussion participation in the final grade to something like 20 percent. But as I wrote previously, my online courses are populated primarily by adult students who have competing demands on their time — such as family and full-time careers. I do not want to penalize people too severely if writing about the course readings is all they have time for, especially since it’s my belief that students learn more from this than from participating in discussions with classmates who often have erroneous or biased points of view.

A second option is to require that each student submit a certain number of responses to other students’ discussion comments each week. But with only a half dozen or so students in a course, I suspect they will quickly just be repeating what others have already posted. 

How students might see my course design
How students might see my course design

A third possibility is to alter the discussion medium itself by requiring video rather than text posts.  Video comments were actually suggested at the blended learning conference that I recently attended as a way of enhancing online discussion, and so I inserted brief introductory videos into discussions beginning with the fourth week of class. But there are major drawbacks to this technique, too. Videos aren’t captioned, which contradicts universal design principles. Listening to comments made on video takes longer than reading text when the instructor is grading the quality of the content. And some students will invariably be incapable of producing video of sufficient quality or in a usable format.

The last option is to just abandon online discussions altogether — one less thing for me to grade — when a small number of students enroll in a course. Given students’ lack of engagement with online discussion this year, they must not have found it very important to their learning or to their grades, so eliminating it might not have much of a negative effect.

The Perils of Small (Online) Classes

Shrinking ManA few months ago I wrote about a few of the effects of small class sizes in my undergraduate courses. I’m experiencing related problems in my online graduate courses, where per-class enrollments have dropped to only about a half dozen students.

With so few students, there is no actual conversation in weekly discussions. I fully understand and accept the fact that several factors probably diminish student contributions to discussions — these courses are online, asynchronous, and convenience for students is paramount. However, people did respond to each others’ discussion comments in previous semesters when enrollments were higher.

This is the first time that I’ve explicitly tied discussion posts to a rubric — at least two posts per week, with the first before midnight on Wednesdays, with cited examples from the reading assignments, to earn full credit — rather than vaguely categorize them as evidence of course participation. But making my assessment of students’ contributions to discussions more transparent should encourage rather than discourage posts, if it has any effect at all.

In sum, students are occasionally acknowledging the existence of their classmates but decide to forego interacting with them. They might be reading each others’ discussion posts, but there is no way for me to tell if this is in fact happening; if they are, they aren’t writing responses. Overall it seems unlikely that much social learning is occurring.

Back to the Future with Blended Courses

I recently returned from the Online Learning Consortium’s conference on blended learning. Blended, or hybrid, means a course in which lecture content has been moved online, and less-frequent classroom sessions focus on higher-order tasks of application, evaluation, or synthesis.

Here is the advice that veterans of blended course design gave at the conference:

  • Set student expectations in advance. Students who are new to blended courses frequently conclude that they are a bad combination of the online and face-to-face worlds. It’s up to instructors to frame the experience as one that provides greater access to and more effective interaction with faculty. Pitching the course as an experiment is probably the worst message to send.
  • Online content and face-to-face exercises must correspond to but not duplicate each other. Students’ classroom participation in team- or project-based activities, for example, needs to align with the key concepts of the online content so that both sides of the course unfold in a coherently-scheduled, mutually-reinforcing manner. A frequent method of assessment that prevents non-proficient students from progressing through the content is highly useful in this regard. If online replicates what happens in the classroom, or if they are not integrated with each other, students will either stop engaging with the former or stop being physically present in the latter. 
  • Students need to understand that “online time” does not replace “homework time.” They will still need to devote significant effort outside of class to research, writing, or the completion of problem sets. This message can be highlighted as part of the orientation to using online content that students will need at the beginning of the semester.
  • Conversely, instructors need to be careful not to overwhelm students with material in excess of what students would encounter in the course’s traditional version. 
  • Online video should be in 5-10 minute pieces with Goldilocks-style assessment exercises after each piece — something not too easy nor too difficult. This fosters students’ engagement with the content by giving them the feeling that they’re being fairly challenged. If the assessments are perceived as too difficult or as irrelevant busy work, student motivation to access the content will decrease.
  • When producing video, don’t be afraid to be a real human. Students are not looking for a Taylor Swift-level of production value.
  • Use replicable tools, methods, and content to drive down the financial and emotional costs of creating additional blended courses in the future.

Grading Discussion in Online Courses

Cat ErrorIf enrollment holds steady, on June 29 I will start teaching two seven-week online graduate courses.* I’ve been teaching these courses every summer for several years, and I’ve decided to experiment this summer with a different system for grading student discussions.

I incorporate student discussion into all my courses, whether they are on campus or online, because I believe it fosters student engagement. But–yet again–discussion in these two courses last year demonstrated that there is often a difference between my beliefs about what students should do and how they decide to achieve whatever objectives they have set for themselves.

The shift was also prompted by the adoption of a different instructional tool. When I began teaching these courses, my university used Blackboard as its course management system. Anyone who has used Blackboard knows that it lacks an intuitive user interface and requires that both students and instructors click through innumerable screens. I created this rubric for class discussion, but there was no way to easily link it to what students were writing. Also the rubric was much too complicated to use to evaluate every discussion post by every student. My assessment of discussion defaulted to digging into the student analytics feature after the mini-semester had ended, to weigh the total number of a student’s posts against a scale I had created. Students got little direct feedback from me on how well they were performing in this component of the course while it was still running.

Last year a few students did not participate at all in the weekly discussions. Because of how I structure my courses, they were able to exercise other options and still perform well in terms of their final grades. But their absence from the discussions meant that their peers were not learning from them and they were not learning from their peers. And it looked to me that the lack of transparency in how I evaluated discussion made this outcome more likely.

This time around the courses will be delivered via Canvas instead of Blackboard. Canvas allows the instructor to create interactive rubrics that can be linked to specific assignments or posts in a discussion. The instructor clicks on the rubric’s boxes and the resulting grade is generated. Students see how their work will be assessed without having to click through a myriad of webpages, and they get immediate feedback from the instructor. 

So I created this new rubric, simpler than the old one but still containing the criteria that I think are most important for peer learning in a professional environment, for grading each student’s discussion posts on a week-by-week basis.** I’ll let you know how it works.

*The courses are the politics of the Middle East and comparative political development, part of an M.A. program in international relations. If you’re interested in acquiring some transferable graduate credit hours, learning about a new subject, or learning how to design and teach online course on a compressed schedule, get in touch–you don’t need to be admitted to the degree program to  enroll in either course.

**My wife/colleague showed me how to do this.

Poverty Games, Part 7: Against All Odds

Against All Odds is an online game hosted by the Office of the United Nations High Commissioner for Refugees (UNHCR) that lets players experience what life is like for a political refugee. The game has twelve parts, divided into three sections: War and Conflict, where you play through levels that require you to give up all your beliefs and identity markers in a military police interrogation (or you are thrown in jail indefinitely), choosing which of your many possessions to take with you when you leave (including choosing whether or not to take your dog), and navigating out of the country without being seen; Border Country, which focuses on trying to get asylum and figuring out who to trust; and A New Life, where you have to get a job and go shopping in a climate of suspicion and in some cases, hostility.

There are some great things about this game. Each level only takes a few minutes to play; the entire game can be finished in 30 minutes. The level one initial interrogation is interesting, because you cannot proceed in the game without agreeing to some pretty harsh statements, such as ‘I give up my faith’ or expressing a willingness to give up your language. Students will probably struggle with agreeing to these statements, which can lead to interesting conversations about life in an oppressive environment. Several of the mini-games are fun to play, such as the one where you have to navigate out of town without being seen. You can access any of the twelve levels at any time, which makes it easier to focus on a particular lesson with the students, and each level can be replayed as many times as necessary without penalty. UNHCR also provides a series of web facts, with stories of people who lived through the levels in the game, as well as teaching resources related to the game, although much like the game itself the target audience is younger than college.

There are some downsides. First, some of the levels are a bit too high-handed, such as the ones in the A New Life segment where you have to go from apartment to apartment and learn about your neighbors’ prejudices. Its also frustrating that a single incorrect answer prevents you from directly moving onward in the game (you can always go back to the start screen and choose a later level, but its an inconvenient extra step). For example, one mini-game involves correctly identifying the origin of various inventions like the bicycle, chocolate, gum ball machines, and insulin. A single incorrect answer prevents you from moving on. In one way this is an asset, as replaying it helps you learn, but it’s still cumbersome. Also, the game ends rather abruptly and anti-climatically.

Overall, though, Against all Odds is a pretty neat way to teach students about the plight of refugees, focusing not only on the original oppression that drives refugees out of their home country, but also on the difficulties they face in seeking asylum and building a new life. I would recommend assigning it either as homework the night before a lesson on refugees, or with a small class and access to a computer lab, having them play through a level or two as a class exercise, followed by discussion.  This would also work very, very well in an online, hybrid or blended learning environment.

Previous Posts in this series

Part 1: Ayiti, the Cost of Life

Part 2: 3rd World Farmer

Part 3: Free Rice

Part 4: Spent

Part 5: Inequality Monopoly

Part 6: Papers, Please!

There goes the neighbourhood

Who can you trust these days?

Over the past few months, I’ve been playing an online, asynchronous game with a couple of different groups.

One of these is the INOTLES trainers that I’m training in the use of simulations, while the other is my final year undergraduate students here at Surrey.

As we come towards the end of the INOTLES cycle, I thought I’d finally share my game with you (Neighbourhood Game) and would talk about how it worked.

The game is essentially a simple model of the EU’s Eastern relations, with a three country Union, a big unhelpful country that likes to exert its influence, and a couple of smaller states in-between. In the attached version, I’ve put two players per country, but that can easily be expanded.

The gameplay is very open, with lots of options for activity, coupled to a long period in which to play (using a weekly cycle of posting on a forum).

The aim is to get students thinking about collective action problems in an international context, as well as to cope with the vagaries of online discussion: the INOTLES version has been running with players from six countries.

So how’d it go?

In a word, fitfully.

Online games require a hook, to keep people involved and active. My ability to do this with the INOTLES group was less than with my students (who I see every week for class), but in both cases the level of engagement has been less than with comparable face-to-face exercises. In short, the ability to coerce action is very limited.

That in turn highlights the importance of the game leader in motivating players: if you’re not pushing, then who’s going to do it for you? Again, the online nature of the game means that it’s easy to forget to prod people, and this is something that I’d say needs careful thought.

A second issue seems to be linked to the openness of the game play. Because players can do pretty much anything from writing a terse communique to launching world war 3, I’ve noted a certain hesitancy about doing anything. This ‘jam choice overload‘ problem is well-known in psychology, but raises an interesting problem for us. Too much choice might be inhibiting, but it also reflects the real world, so we might want players to feel inhibited. As always, this will depends much on what you aim to achieve.

Thirdly, while this is a fictional situation, it is also obviously close to the real-world (I took most of the data from real countries). This adds a different dimension to the game, as people apply what they know of that real-world into their actions. Thus Novy Putonova acts rather like Russia, Bigistan like Germany and the Squashed Republic like Ukraine. To be more accurate, people acted like they thought those countries act like. This offers lots of opportunity to get into a discussion about how we understand the real world: are Russians really that sneaky, to take one obvious example? Either way, it opens a door to discussion of the substantive material.

On the level of skills, there is also a lot to think about. How did people work in their teams? How did they deal with each new development? How much did they try to take control of what was happening? How did they cope with some groups being very passive/silent? Again, in all of this, the large range of possible actions meant that there was also a big question about why they chose to do what they did, and not something else?

Next steps

This game has been a trial for me (in at least one sense of the word). It’s been my first effort in this type of game and, as always, I’m not totally happy with it.

On the plus side, players have played, and it’s shown that one can model an international system with some quite simple elements. Feedback to date from both groups has been positive and – importantly – I can see how I would change things in future.

On the down side, engagement has been relatively low (compared to face-to-face) and my input has been more than planned. The inhibition to take more drastic action in either game (they’ve largely been polite and pretty constructive) means that I don’t know how the more radical options might play out.

With my students, I now plan to use our final session before the Christmas break to play the game in class for a couple of hours, to connect it more strongly to the rest of the module, and to let me see how that changes the interactions.

If you’d like to use the game, please do – I’ve popped it up on my other website already. If you’d like to feedback on how it works for you, then I’d also love to hear about that.

Do we still need face-to-face interactions?

Alfred wasn’t sure that the ticker tape was properly conveying the emojis

Yesterday prompted this question from a variety of different directions, but it feeds into a bigger question about the changing nature of Higher Education, especially in light of MOOCs. Face-to-face, physically-co-located meetings are relatively resource- and labour-intensive, so can we get the same benefit without them?

First prompt was the third in a series of webinars that I’ve been involved with for our INOTLES project. Using Abode Connect, we brought together educators in 4 time zones and 7 countries for a ninety minute chat about assessment and feedback. And it worked really well. The mixture of video, audio and text debate produced a rich discussion, from which I think everyone got something. Indeed, precisely because it was one in a series, we were able to focus on the matter in hand, rather than feel – as we have when meeting physically – that we need to cram in everything we can.

Second prompt was my continuing uncertainty about whether I will be joining my fine ALPS-blog colleagues (and Victor) at APSA’s TLC in January: they’ve heard, I’ve not.*

TLC was of course the occasion that first brought us together, and I’ve tried to go every other year, so that we can spend some time catching up and generating new ideas. By necessity, our group is one that interacts almost exclusively online, so we obviously can cope without the meeting, but it’s still good to see each other.

Third prompt was my first serious use of the online marking and feedback system that I have introduced to to our Faculty this year. Quite aside from the technical aspects of this, I’ve also come to realise that it further changes the interaction with students, since it invites you to channel much more feedback through the online system: no handing back of essays in class, with an opportunity for a quick chat.

So, in three very different ways, I have been thinking about the role of face-to-face interactions, both with students and with colleagues.

Now, you might expect that – given my work on simulations – I’d be unreservedly in favour of face-to-face, because of the richness of that mode of interaction. And you’d be right to an extent.

In all three of the prompts I’ve just mentioned, face-to-face modes can bring something that is probably impossible to achieve otherwise. The scope of deep engagement, the building of soft elements of relationships, the unverbal cueing – none of that works in the same way elsewhere (if at all).

But at the same time, I can see a huge value in online interaction: indeed, without it, this blog and the INOTLES project wouldn’t work at all. And the process-tracing and transparency of the online marking/feedback system makes for a more manageable experience than its predecessors (transition costs notwithstanding).

In short, it’s that old chestnut: it’s not better or worse, just different.

Personally and professionally, we need to get students exposed to different kinds of interaction, since they will not always be able to choose what situation they find themselves in.

Seen as such, the important factor is to develop everyone’s reflection about the nature, benefits and limitations of those interactions; and that includes us too. It’s not only useful as a skill in of itself, but it’s also useful in making sure that we all get the most of how we deal with other people.


* – If someone from the TLC committee is reading, I’d really like to go, so let me know.