A Quick Exercise on Confirmation Bias and Hypothesis Testing

This neat exercise featured on the New York Times takes a few seconds to play and includes a neat set of examples of how confirmation bias impacts government policy and corporate America.

Basically, you are presented with 3 numbers in a sequence, and asked to guess the rule that governs the sequence. You can enter in any 3 numbers you like, and the system will tell you whether or not your sequence follows the rule or not. When you are ready to guess, you enter it in but you receive no second chances. Apparently 78% of people make a guess without getting a single ‘no’–and most get the rule wrong.

The example in the NY Times is ‘2, 4, 8’. A number of possible rules could come to mind–must contain multiples of 2, or even numbers, or that the number doubles the one before it. The actually rule in this case is even simpler: the number must be larger than the one before it, meaning that ‘4, 8, 16’ works, but so does ‘1, 10, 3593’.

They don’t mention it in the article, but this exercise can adapted to teach hypothesis testing. Used in class, you can put the sequence on the board and have students suggest other sequences, which you then judge as either following or not following the rule. They have to use this information to come up with the right answer.

If this sounds familiar, it may be because one of the very first entries on this blog was about the board game Zendo, which does precisely this, only with physical pieces rather than numbers. I still use Zendo on day 1 of my methods class, and find it a really useful tool for teaching a variety of methodological skills. This numerical version is a great, easy activity to pull out for a quick fix on helping students with their logical thinking.

Avoiding Feature Creep, Part 1

This post is inspired by the disappearance of some of the last chalkboards on my campus because of a building renovation. I regard the chalkboard as one of the best teaching tools ever invented. They are absurdly easy to use. They have no moving parts and no need for electricity, so they always work. They are relatively inexpensive and never need upgrading.

The new classrooms that are being constructed will invariably get a combination of whiteboards and multimedia equipment. I find the former to be a pain to erase, and with whiteboards I produce noticeably worse penmanship and diagrams than I do with chalkboards. The latter allows for the presentation of information in a variety of ways — video clips, Powerpoint slides, the display of documents being edited in real time  — but only when all the complex pieces are working properly. The fact that we are able to continue teaching and teach well when this equipment fails demonstrates that the computer, the projector, the speakers, and everything else are actually not essential to doing our jobs.

Generally I want a tool that does one thing or a very small combination of things well, like a pen that makes writing on a piece of paper feel good or a chair that doesn’t make my back sore when I sit in it. Or a cell phone that functions as . . . a phone. Whether I’m in the classroom or online, the tool should make me more effective at teaching because it simplifies the process of learning for students.

Unfortunately many of the people who design tools ignore the fact that the utility of any tool usually decreases in proportion to the tool’s number of features and the manner by which those features are accessed. For example, compare the screens of these two online learning management systems:

Blackboard — Chernobyl control room?

Canvas — I can fly this.

Canvas Screenshot

Or, if selecting a simulation for the classroom, the difference between the Survive or Die game:

Deck of Cards

and Statecraft:

Statecraft

I’ll talk about why avoiding feature creep is important for overall course design in a future post.

 

Back to the Future with Blended Courses

I recently returned from the Online Learning Consortium’s conference on blended learning. Blended, or hybrid, means a course in which lecture content has been moved online, and less-frequent classroom sessions focus on higher-order tasks of application, evaluation, or synthesis.

Here is the advice that veterans of blended course design gave at the conference:

  • Set student expectations in advance. Students who are new to blended courses frequently conclude that they are a bad combination of the online and face-to-face worlds. It’s up to instructors to frame the experience as one that provides greater access to and more effective interaction with faculty. Pitching the course as an experiment is probably the worst message to send.
  • Online content and face-to-face exercises must correspond to but not duplicate each other. Students’ classroom participation in team- or project-based activities, for example, needs to align with the key concepts of the online content so that both sides of the course unfold in a coherently-scheduled, mutually-reinforcing manner. A frequent method of assessment that prevents non-proficient students from progressing through the content is highly useful in this regard. If online replicates what happens in the classroom, or if they are not integrated with each other, students will either stop engaging with the former or stop being physically present in the latter. 
  • Students need to understand that “online time” does not replace “homework time.” They will still need to devote significant effort outside of class to research, writing, or the completion of problem sets. This message can be highlighted as part of the orientation to using online content that students will need at the beginning of the semester.
  • Conversely, instructors need to be careful not to overwhelm students with material in excess of what students would encounter in the course’s traditional version. 
  • Online video should be in 5-10 minute pieces with Goldilocks-style assessment exercises after each piece — something not too easy nor too difficult. This fosters students’ engagement with the content by giving them the feeling that they’re being fairly challenged. If the assessments are perceived as too difficult or as irrelevant busy work, student motivation to access the content will decrease.
  • When producing video, don’t be afraid to be a real human. Students are not looking for a Taylor Swift-level of production value.
  • Use replicable tools, methods, and content to drive down the financial and emotional costs of creating additional blended courses in the future.

Grading Discussion in Online Courses

Cat ErrorIf enrollment holds steady, on June 29 I will start teaching two seven-week online graduate courses.* I’ve been teaching these courses every summer for several years, and I’ve decided to experiment this summer with a different system for grading student discussions.

I incorporate student discussion into all my courses, whether they are on campus or online, because I believe it fosters student engagement. But–yet again–discussion in these two courses last year demonstrated that there is often a difference between my beliefs about what students should do and how they decide to achieve whatever objectives they have set for themselves.

The shift was also prompted by the adoption of a different instructional tool. When I began teaching these courses, my university used Blackboard as its course management system. Anyone who has used Blackboard knows that it lacks an intuitive user interface and requires that both students and instructors click through innumerable screens. I created this rubric for class discussion, but there was no way to easily link it to what students were writing. Also the rubric was much too complicated to use to evaluate every discussion post by every student. My assessment of discussion defaulted to digging into the student analytics feature after the mini-semester had ended, to weigh the total number of a student’s posts against a scale I had created. Students got little direct feedback from me on how well they were performing in this component of the course while it was still running.

Last year a few students did not participate at all in the weekly discussions. Because of how I structure my courses, they were able to exercise other options and still perform well in terms of their final grades. But their absence from the discussions meant that their peers were not learning from them and they were not learning from their peers. And it looked to me that the lack of transparency in how I evaluated discussion made this outcome more likely.

This time around the courses will be delivered via Canvas instead of Blackboard. Canvas allows the instructor to create interactive rubrics that can be linked to specific assignments or posts in a discussion. The instructor clicks on the rubric’s boxes and the resulting grade is generated. Students see how their work will be assessed without having to click through a myriad of webpages, and they get immediate feedback from the instructor. 

So I created this new rubric, simpler than the old one but still containing the criteria that I think are most important for peer learning in a professional environment, for grading each student’s discussion posts on a week-by-week basis.** I’ll let you know how it works.

*The courses are the politics of the Middle East and comparative political development, part of an M.A. program in international relations. If you’re interested in acquiring some transferable graduate credit hours, learning about a new subject, or learning how to design and teach online course on a compressed schedule, get in touch–you don’t need to be admitted to the degree program to  enroll in either course.

**My wife/colleague showed me how to do this.

Would You Like Coffee With Your Freshman Year?

Starbucks BaristaIf you’re in a radio market like mine, you get to hear ads for the dozens of online education programs offered by Arizona State University (ASU). ASU also garnered publicity not too long ago for its collaboration with Starbucks that provides the company’s customer service employees with access to an affordable college education.

If you’re a regular reader of this blog, you’re also familiar with my prognostications about the technology-driven transformation of post-secondary education — those depressing posts about the financial unsustainability of small colleges with mediocre reputations, the inevitability of a tiered pricing system that conforms to market demand, and the rise of convenience-based learning.

As of today the stars have aligned: ASU has partnered with edX to offer, according to The New York Times, “an online freshman year that will be available worldwide with no admissions process and full university credit.” Cost? Two hundred dollars per credit hour, payable upon passing each course. The total price for the whole program is about $5,000. 

According to MIT professor Anant Agarwal, the CEO of edX, the full complement of courses for the new Global Freshman Academy is projected to be live within twenty-four months.

Say goodbye to your general education requirements.

Digital Skills Quiz

Digital SecretaryI try to model the information literacy skills we think are important for students to acquire. I also heed Nina’s caution against fostering learned helplessness. Yet I’m seeing students who lack even the most rudimentary technological skills like knowing how to download a document file so that they can read it off-line. This puts me in a bind.

My first thought was to specify the fundamental digital knowledge and skills that students must possess to get through any of my courses:

  • Regularly access your university email account and the online course management platform with your username and password.
  • Download files to a device, with names and to locations that enable you to easily find them later.
  • Save/save as/export files in required formats, with files appropriately named, to personal computing devices and the course management platform.
  • Proofread your writing without using automated spell-check features.
  • Back up work on external devices (USB drives) or cloud services like the course management platform, Google Drive, or DropBox.
  • Set the preferences in your course management platform account so that you are notified of course updates in a timely fashion.
  • Use the library’s website and databases to locate assigned readings.
  • Contact the IT Help Desk, librarians, and other campus technology support services when you have questions about any of the above.

I could put that list on the first page of all my syllabi, but that essentially goes against what we at ALPS stand for. Reading a list once doesn’t change behavior, and my students don’t really read the verbiage on a syllabus anyway. The real reason for me to include such language in a syllabus (which is a perfectly valid one) is that it helps protect me when a student fails a course.

A better approach might be to format the list as a quiz at the beginning of the semester that is worth a tiny fraction of students’ final grades. The quiz questions could require some action by the student that demonstrates that he or she is actually able to perform the skill that is referenced by the question; for example, “download this one-page file, insert your name at the top, and then upload the new version as a pdf to this file folder.”

The above option assumes that students actually take such a quiz, and that they try to remedy any skill deficiencies that the quiz identifies. This may not happen. I re-instituted an open-book syllabus quiz in all my courses this semester and a few students chose not to complete it, while others scored badly. Given that I already see a loose correlation between students’ scores on the syllabus quiz and their overall course performance, a quiz on digital skills might provide additional support for my hypothesis that I can accurately predict any student’s final grade after only the first two weeks of a semester — but that’s a subject for another post.

Using Webinars in Political Science Education

As part of our continuing series of guest contributions, John FitzGibbon (Canterbury Christ Church) talks about his experience with webinars:

I’ve been using Webinars for just over a year in my teaching here at CCCU and I think it is time to evaluate them as a form of teaching practice.

What you may ask, is a Webinar?  Generally it is understood as an online ‘event’ hosted by a particular organisation and broadcast to a select group of users.  This typically takes the form of a live video broadcast.  Obviously given different disciplines there can be huge variety in content for this live video and this blog post will focus on how they can apply for Politics/International Relations and the Social Sciences more generally. But the potential wider educational impact of using Webinars is clear.

philippe-legrain
Philippe Legrain webinar

 

I  have been using them in two main formats: 1. Hosted and organised by external partners, 2. Hosted and organised by myself.  There have been mixed results with both approaches but overall the use of Webinars has been viewed positively by my students.  Additionally, it has exposed both them and myself to new teaching methods and to the power of the internet as a learning tool.  Much is made of the potential of the internet to enhance student learning but there is still the need to fit specific internet based usage into defined pedagogic strategies.  This blog post aims to tease out some of the potential use of Webinars as part of a typical Social Sciences undergraduate module.

 

External Partner Led Webinar

I  have a great working relationship with the Global Net 21 NGO.  Their goal is to link the community with politicians and other decision makers in a constructive narrative.  What they bring is technology, contacts and experience.  My students and I shot a series of videos (available here) on the issue of food scarcity.  Global Net 21 were able to edit them altogether into a coherent whole.  This takes both time and technical training which most educators simply don’t have, to make the videos look somewhat professional.  As Global Net 21 have been hosting Webinars for many years, they have developed the technology to stream two live video feeds, with a comment box for invited guests on a single web-based platform.  This allows for a diverse audience to participate remotely in the Webinar.  But perhaps the most important element that an external partner brings is contacts.  My students have been able to participate in Webinars with many MPs and former government minister Peter Hain MP.  Ordinarily it is difficult to get MPs to commit time to a Webinar but they have been happy to give time to Global Net 21.  An educator can face a struggle in getting even 30 minutes from a leading figure to participate with a class but an outside partner, particularly an NGO, brings cache.  Moreover, a well-established external partner can allow the educator and students to interact with a wide variety of participants, enhancing the learning experience and facilitating networking with more potential partners.

cccu-student-on-webtv
Video Reports from our students broadcast on a Webinar

 

The clear drawback in working with an external partner is that your pedagogical goals for the class might not always align with the outcomes the topics the partner wants to focus on.  This means compromise which generally results in discussion that may not be closely related to the module.  While an external partner with excellent technology skills and contacts can make a Webinar happen in the first place, the main cost is a lack of control.  With Webinars this means a loss of interaction.  The external partner hosts the event, asks the questions, refers or does not refer to you and generally guides the discussion where they want.  This might lead to an overlap in interests but more often than not in my experience leads to students becoming bored by a discussion lacking the required academic rigour that does not include them.

 

Educator Led Webinars

In response to this reduced level of interaction my students experienced in previous Webinars, I sought to organise my own.  Lacking the equipment and time to work with an online interactive Webinar platform I decided to just use Skype and contact potential Webinar participants myself.  Though there are many specific Webinar platforms available but they need to be bought and many universities have issues over downloading, installing and purchasing software that are more hassle than they are worth.  Using this excellent Logitech conference cam there were no technical difficulties with Skype. Participants could hear us, even questions from the back of the class room and we could hear them.  I invite individuals that I want the students to interact with meaning that there is no need for compromise on the themes to be discussed and the Webinar fitted in seamlessly with the module learning goals. For my most recent Webinar on 5th of February I circulated articles written by the individual, Philippe Legrain an expert on the European Financial Crisis, and asked the students to draft up questions for Philippe.  In the previous week we had covered the role of finance in the global economy and the first hour of class was focused on the development of the Eurozone.  I asked Philippe to participate as not only did he write an excellent book on the Eurocrisis but he has plentiful media experience and so can communicate his ideas succinctly.  For the lesson plan I envisaged the second hour of class been given over for the Webinar, allocating Philippe  20 minutes to speak on the reasons why he thought the Eurocrisis happened, and then for a 20 minute question and answer session with students.  Philippe spoke so well and with such completeness that he actually answered all the students questions.  Students recorded his discussion on their phones and several bought his book on Amazon as he was speaking. The students were extremely positive on the experience.  They were impressed to have such an authority, he had been on BBC Newsnight the week before, talk to them and answer all the questions on a deeply complex contemporary issue.

philippe-legrain-ii
Philippe Legrain Webinar Students buying his book online

 

From this I took that an educator led Webinar is far more work than bringing in an external partner.  However, it allows for a better pedagogical outcome.  This is due to greater levels of control.  Paramount in this is that the educator can chose the participant and frame the Webinar to directly encompass learning goals.  With a good Webcam and internet connection there are no technical issues.  The only significant problem is to locate a participant with the required skills, strong personality and time to be involved.

 

The Pedagogic Usefulness of Webinars

Concluding on the pedagogic usefulness of Webinars I have found them to be a potentially excellent tool for educators. An interesting finding from participating in 12 Webinars is how their informal nature lends itself to an excellent flow of discussion. This is in contrast to invited lectures where the guest gives a formal lecture involving a huge time commitment on both sides on a theme that though suggested by the educator, might deviate from the module content. A Webinar gives the opportunity to guide a guest on to themes that are directly relevant to the module and allow for students to interact more intimately with them in a less formal setting. Taking control of the process does require some degree of organisation in finding a suitable participant and finding a suitable time but they, and the accompanying technological issues, are not insurmountable. Using Webinars I have found brings new voices and insight into the classroom and facilitates student interaction with outside experts. It also encouraged me to interact with NGOs and other members of the community. The more advanced elements of Webinars, contextualizing video, require too much preparation time for an individual educator. Access to a dedicated and skilled Teaching and Learning Team would enable a much more advanced Webinar process like this to be developed but many do not have such help available.  To educators I would advise to try a Webinar. Find a suitable participant, brief them on what topics you want them to discuss, give guidance to your students to prepare questions and let the debate flow!

This post was originally posted on the Canterbury Christ Church Politics & IR blog.