Specifications Grading, Attempt 1, Day 0

Hello, ALPS readers! I’m back after a long summer and spring sabbatical, and am eager to get back in the classroom and talk all things pedagogy here on ALPS. I’m starting a new series where I outline in excruciating detail my experiences using Specifications Grading.  I’ll be sharing my materials, talking about the ups and downs, and reflecting on this unique grading system throughout the semester.

We’ve given quite a bit of attention to specifications grading in the past few months. I did a presentation on it at the ALPS workshop at the University of Surrey in May as I started working on adapting one of my own courses to this new system. I also consulted several former students and children-of-friends about what they thought of the system in abstract, and the general consensus ranged from “shrug” to “that might be cool.” Experts in analysis, my young consultants.

In a nutshell, Specifications Grading is a system where all assignments are clearly linked to course learning outcomes, given clear specifications on what students need to do to earn a passing mark, and graded on a pass/fail style system, where a pass is a high bar (typically a B). Assignments are bundled together by learning outcome, and course grades are assigned based on the bundles that students complete. So, higher grades go to students that either complete more bundles (achieving more learning outcomes) or higher-level bundles that demand students complete more complex work. The course also employs flexibility mechanisms such as tokens to let students revise or reattempt a failing assignment, forgive a course absence, or gain some other kind of benefit. This system is supposed to ensure that all students who pass the class are achieving the minimum learning outcomes for the course, but also puts their grade into their hands by removing the mystery behind grades (no longer 170 out of 200 points, but ‘excellent’ ‘satisfactory’ or ‘unsatisfactory) and letting them choose which grade bundle to achieve.

Check out our previous posts for more general information on Specs Grading, or check out this great community of scholars working with the system.. For this new series, I am going to write throughout the semester about my experience in adapting and teaching my research methods course to this system.

Continue reading

Teaching Methods: Reliability and Validity at the AP Reading

Last month I participated in the annual Reading of the Advanced Placement US Government exams. In May, over 300,000 students in US high schools took this exam, consisting of a series of sixty multiple choice questions and four free responses, usually in the hope of earning college credit or placement in a higher level course. The multiple-choice items are easily dealt with, but someone has to score the free response questions. Enter the AP Readers, a collection of several hundred college professors and high school teachers who voluntarily spend a week sitting in a convention center, scoring essays for eight hours a day, seven days straight. Sound awful? It gets worse: we actually score responses to the same essay question, over and over again. On a good day, I score about 450 essays, all answering the same question.

So why have I put myself through this tedious exercise for nine of the last ten years?

Continue reading

Grading on Spec III

The first-year seminar I mentioned in my last post will have twenty-two students; each will be (or should be) submitting at least thirty-five reading responses during the semester. While the reading responses make students read and write about what they’ve read, I have to grade them within twenty-four hours of submission if my feedback is going to have any effect. To make grading these writing assignments as easy as possible, I’ve used a form of specifications grading with a 2 X 2 rubric.

I’ve noticed that any long-form comments I make about what a student submits — besides compliments like “Good work” — almost always relate to mechanical errors in the student’s writing. I see spelling mistakes, disagreement between the singular and plural, incorrect verb tenses, and other problems that detract from the ideas students are trying to express. My rubric hasn’t included a criteria for writing mechanics, hence the perceived need for me to type something in the tiny comment box when grading.

I’ve decided to add that criteria to the rubric to further reduce the amount of time spent grading. The rubric now looks like this:

30 point rubric

Since each reading response will be worth 30 points rather than 20, I will probably also boost my grading scale for the course from 1,000 to 2,000 points.

Previous posts on specifications grading herehere, and here.

Specifications Grading

The Chronicle of Higher Education had a great post this past week about Specifications Grading, a style of grading that moves away from holistic (“too light–feels like a C”) approaches into the world of contract grading, where specific criteria are set that, if met, earn students a particular grade.

Thorton Mellon definitely believed that college instructors grade by weight.

I hate grading, so I was eager to read more.  This style of grading sets clear, achievable metrics for success on a given assignment, and ties that success to the learning outcomes for both the assignment and the course.  Grading itself becomes simple–either students met the metrics, or they did not.  Students are given more choice in how to approach their work for the course, but at the same time, must complete enough work at a satisfactory level in order to pass the class or get their desired grade. This system, if done correctly, can eliminate student grade complaints, put students in charge of their own learning, AND reduce faculty grading time (YES!),  What’s not to love?

Continue reading

Authenticity in Classroom Presentations

Bored AudienceNina’s recent post about students’ tendency to default to infopedia-style presentations reminded me of my own horrible experiences with students incoherently regurgitating information. It occurred to me that many of us — because of the nature of our academic experiences — incorrectly regard writing and presentation design as unrelated processes. They both are forms of communication, so why not apply the principles of one to the other?

I’ve written previously about how a good writing assignment clearly specifies the role of the author, the audience for whom the author is writing, and the format that the author is supposed to follow. We can construct guidelines for student presentations the same way:

  • What role are you performing when giving this presentation? What’s the presentation’s purpose?
  • To whom will you be delivering this presentation?
  • What is the format with which you will deliver the presentation?

If your instructions for presentations allow a class to answer the above questions with “student, other students, data dump,” then that’s what you’re going to get.

I find it easiest to grade format, so I assess that first. Student teams upload files of their presentations before they deliver them, and I grade the files using this rubric:

Presentation Rubric

After my feedback via the rubric, teams have time to improve format if they wish before delivering the presentations in class.

The audience for the presentations is the rest of the class, not me, so I use peer evaluation. Teams compete and whichever team students select as the best earns points. For a sense of how these competitions have evolved over time, check out my posts here, here, here, and here, and there’s also Amanda’s and Simon’s big-picture posts on the subject. Or just click on the Presentations category in the right-hand sidebar column on your screen.

As for role, I’ll be the first to admit that it can be very difficult getting students to step outside of themselves and take on the identities of government officials, corporate executives, or journalists. But providing clear expectations about audience and format goes a long way toward helping students achieve this third goal.

More on Twine

After my last post on building an entire course around the creation of interactive digital texts, Jason Enia asked if I had additional materials to share. Here they are:

Three Wise MonkeysThe first-year seminars, of which this course is but one example, are part of my university’s recently-revised general education curriculum.  The following learning outcomes are specified for the seminars:

  • Utilize the liberal arts skills to analyze and evaluate significant texts and investigate mathematical and scientific processes.
  • Speak about significant issues in a cogent, analytical, and persuasive manner.
  • Write about significant issues in a cogent, analytical, and persuasive manner.

At the end of the semester, I am expected to assess students’ work on the course’s final Twine text against these learning outcomes with this rubric. I don’t use this rubric for grading. Instead, as I mention in my previous post, student teams grade each other’s work using a different rubric that I’ve designed specifically for this type of assignment.

My syllabus contains these passages:

Course Description

In a disaster, who lives, who dies, and why? This course examines the life and death decision making of individuals who have survived hurricanes, earthquakes, civil war, and genocide. With story-boarding and free software, you will learn how to design digital interactive stories on this subject that evolve according to the decisions made by the reader. The texts that you read and create will allow you to explore ethically complex decision making in high-risk, worst-case environments—the same kinds of situations faced by humanitarian aid workers, diplomats, military personnel, and, all too frequently, ordinary individuals.

Course Objectives

• Practice recognizing and creating thesis statements.
• Analyze arguments.
• Create and communicate persuasive, evidence-based narratives orally and in writing.
• Reflect on different cultures by examining biographical depictions of historical events.
• Evaluate the work of oneself and others.

I ensure that students steadily read the book associated with each Twine with frequent but brief writing assignments.

That is basically everything — anything else specific to the Twine process is described in my last post.