Hello, ALPS readers! I’m back after a long summer and spring sabbatical, and am eager to get back in the classroom and talk all things pedagogy here on ALPS. I’m starting a new series where I outline in excruciating detail my experiences using Specifications Grading. I’ll be sharing my materials, talking about the ups and downs, and reflecting on this unique grading system throughout the semester.
We’ve given quite a bit of attention to specifications grading in the past few months. I did a presentation on it at the ALPS workshop at the University of Surrey in May as I started working on adapting one of my own courses to this new system. I also consulted several former students and children-of-friends about what they thought of the system in abstract, and the general consensus ranged from “shrug” to “that might be cool.” Experts in analysis, my young consultants.
In a nutshell, Specifications Grading is a system where all assignments are clearly linked to course learning outcomes, given clear specifications on what students need to do to earn a passing mark, and graded on a pass/fail style system, where a pass is a high bar (typically a B). Assignments are bundled together by learning outcome, and course grades are assigned based on the bundles that students complete. So, higher grades go to students that either complete more bundles (achieving more learning outcomes) or higher-level bundles that demand students complete more complex work. The course also employs flexibility mechanisms such as tokens to let students revise or reattempt a failing assignment, forgive a course absence, or gain some other kind of benefit. This system is supposed to ensure that all students who pass the class are achieving the minimum learning outcomes for the course, but also puts their grade into their hands by removing the mystery behind grades (no longer 170 out of 200 points, but ‘excellent’ ‘satisfactory’ or ‘unsatisfactory) and letting them choose which grade bundle to achieve.
Check out our previous posts for more general information on Specs Grading, or check out this great community of scholars working with the system.. For this new series, I am going to write throughout the semester about my experience in adapting and teaching my research methods course to this system.
Day 0: Adapting Research Methods from Traditional Grading to Specs Grading
I chose my introductory research methods class for a few reasons. First, it is the course I teach the most frequently. It turns out that if you like teaching methods, your colleagues are generally happy to let you teach it as often as you wish. For me that means at least once a semester, sometimes twice. Unlike many of my other courses, I also teach it in the traditional sixteen-week daytime format, giving students (and me!) more time to adjust to a new system. The classes are small (~15 students, usually a combination of sophomores and juniors with the occasional senior) which lets me use lots of assignments. The course has clear learning outcomes, a variety of detailed assignments, and previously prepared rubrics, so I had already done some of the necessary legwork for switching to a system of clearly defined specifications. Finally, this course is the subject of many previous experimentations on my part, so it felt…right to start here.
Step 1: Review and revise the learning outcomes
Here are the learning outcomes for the course:
- Identify the fundamental components, approaches, methods, and common practices of scientific social research.
- Evaluate the quality of scientific and non-scientific research claims and distinguish between the two, with a special attention to claims in the “real world”.
- Articulate the process of crafting an excellent research design and paper.
- Analyze published work in political science, identifying the components (including thesis, methodology, assumptions, and data) and evaluating its claims.
- Justify theses and conclusions through the use and analysis of evidence.
- Craft a research design or paper to explore a social problem, carry out the design, and evaluate the resulting data.
- Reproduce standards of professional behavior in a college classroom.
The first six have been on my syllabus for years. I added the 7th, because I realized during this adaptation that I do expect professional behavior. Jason Mittell in his use of Specs Grading noted that he realized he was grading students on skills and attitudes that they had coming into the class that existed outside of the learning outcomes. I wanted to maximize how well my outcomes match my expectations for student work and behavior. Since I expect students to show up on time and not use their phones or laptops, I made it a learning outcome.
Step 2: Determine assignments and match them to learning outcomes.
I made no changes to the assignments in the course. This was important to me for a few reasons. First, I’ve been pretty happy with the assignments in past terms, and felt no need to make changes. Second, I was making so many other changes that I wanted to keep anything that didn’t feel broken. Finally, I wanted to be able to reassure students that they were doing no more work (and in some cases, less) than previous terms—in other words, that specs grading wouldn’t unfairly demand more of them.
The assignments in the course are:
- Required Homework Modules
- Optional Homework Modules
- Ethics Training Assignment
- Deconstruction Oral Exam (Midterm)
- Deconstruction Abstract (Midterm)
- Perfect Paper Assignment (includes revision and workshop attendance)
- Best X in Town data collection assignment
- Best X in Town final paper
- Take Home Final Exam
- Professional conduct adherence
This step was also relatively easy because I already had linked assignments to outcomes as this is encouraged in my department. The main challenge was taking all of my problem sets, dividing them into modules based on individual concepts (so, ‘central tendency’ and ‘frequency’ and ‘variation’ rather than ‘Descriptive Statistics Problem Set”) and deciding which ones should be required v. optional. The chart below shows how each assignment (or group of assignments) matched up to learning outcomes. Specific assignments will also clearly indicate in their ‘specs’ which learning outcomes they assess.
Step 3: Determine bundles
This is the step that took up the most of my time by far. I had to clearly indicate which assignments were required to earn each possible grade. This is the heart of specs grading: what do students need to do to pass the course, at a minimum, and what does a C, B, or A mean in terms of achievement of the learning outcomes? Ultimately I decided that all students have to complete, at a minimum, satisfactory work on the required homework modules that stretch across the entire course. Since a ‘satisfactory’ on these assignments requires what would normally be C+ or B- work, I know that even a student who earns a D in my course will have demonstrated their facility with all of the major course concepts. In this way, even though these students do not have to turn in a final paper or take the final exam or part of the midterm, I am actually expecting MORE of them than before. A D no longer means 60% on everything, but 75+% on the work that matters most.
I’m still very iffy on my decision to not require the final paper of D students, though. I struggled a lot with that one.
Careful readers will note that, yes, part of this system means not every student is completing every assignment. This is one of the flexibility features of specs grading: certain assignments are limited to higher-end bundles, meaning that students comfortable with a lower grade never have to do them. This is such a change to my usual system that I keep wanting to go back and make everyone have to do everything—but that defeats the purpose of specs grading, as the entire point is that A students have to do more work at a higher level than B, C or D students. That means that A students have to take the final exam and complete the deconstruction abstract (an annotated abstract of a scholarly article where they note the independent and dependent variables, the thesis, methodology, theory, etc) and get an Excellent on their final paper and oral exam, while B students need only a satisfactory on those two assignments, have fewer homework modules, and no final exam, and C students do not even need to complete the abstract, a perfect paper revision, or optional homework modules at all.
| Summary of Assignments, Deadlines, Grades, and Learning Outcomes | ||||||
| Assignment | Deadline | A
Requirements |
B
Requirements |
C
Requirements |
D
Requirements |
Learning Outcomes |
| Required Homework Modules | Varies | 21
|
21 | 21 | 21 | 1-6 |
| Optional Homework Modules | Varies | 6 | 3 | 0 | 0 | 1-6 |
| Excellent HW Modules | Varies | 10 | 5 | 0 | 0 | 1-6 |
| Ethics Training | 9-1 | Satisfactory | Satisfactory | Satisfactory | Satisfactory | 1 |
| Deconstruction Oral Exam | Week of
10-24 |
Excellent | Satisfactory | Satisfactory | Satisfactory | 1, 2, 4, 5 |
| Deconstruction Abstract | 10-25 | Satisfactory | Satisfactory | n/a | n/a | 1, 4 |
| Perfect Paper Assignment | 10-11 | Satisfactory | Satisfactory | Satisfactory | Satisfactory | 3, 5 |
| Perfect Paper Revision | 11-10 | Satisfactory | Satisfactory | n/a | n/a | 3, 5 |
| Perfect Paper Workshop* | 11-1 and
11-3 |
Attendance | Attendance | Attendance | Attendance | 3, 5 |
| Bxit Data | 11-3 | Satisfactory | Satisfactory | Satisfactory | Satisfactory | 6 |
| BxiT final paper | 12-6 | Excellent | Satisfactory | Satisfactory | n/a | 3, 5, 6 |
| Take Home Final Exam | 12-15 | Satisfactory | n/a | n/a | n/a | 1, 2, 4 |
| Violations of professional behavior allowed (absences, preparation, late arrival, tech use, etc) | n/a | 6 | 8 | 10 | 12 | 7 |
| * Attendance at the perfect paper workshop is only required of students that do not score an Excellent on their perfect paper assignment. | ||||||
Step 4: Token economy development
Flexibility is at the core of specs grading. Students choose which bundle to complete, knowing that if they complete that set of assignments, their grade is assured. The token economy is another key point of flexibility. Students are given virtual tokens at the start of the semester that they turn in for certain course benefits. No longer do I have to determine which excuses are ‘good enough’ to earn an extension or forgive an absence. Instead, students simply turn in a token as needed for such accommodations, and every student starts with the same number of opportunities for flexibility, regardless of their ability to argue their point or whether they know the hidden rules of the classroom. They also have opportunities for earning more throughout the semester by doing additional work that would typically have earned them extra credit in the past.
Here is what I decided:
One token can be traded in for any one of the following benefits:
- The opportunity to re-do a single Unsatisfactory required homework module. Each module can only be re-attempted once.
- To cancel out a single course absence, late arrival, or early departure. A canceled-out violation will not count against your total for the purpose of calculating your grade bundle.
- To cancel out a single violation of the professional conduct policy. This can only be done twice in the semester, regardless of the number of tokens you earn.
- To earn a 24-hour deadline extension on any single assignment (homework module set, perfect paper, ethics training, BxiT paper, deconstruction abstract, or take home final exam). The Deconstruction oral exam cannot be postponed via token. No more than 3 tokens can be spent on any individual assignment.
- To retake the Deconstruction oral exam. Only one retake is allowed, regardless of the number of tokens you earn.
- Save them! Whoever has the most unused tokens at the end of the course will receive a prize.
New tokens can be earned by:
- Attending a campus or community event or talk related to politics, international relations, or research methods and writing a summary and analysis of the event (per assignment specifications).
- Excellence in in-class or online participation may be rewarded with a token at the instructor’s discretion.
- On BxiT group assignments that require individuals to take on the responsibility of coordinating their group and doing extra work, a token may be awarded to the students who volunteer and complete the additional work.
- Earning a grade of ‘Excellent’ on the Perfect Paper assignment will result in TWO tokens.
Note: any course can incorporate a token system, whether or not you use specs grading.
Step 5: Determine Assignment Specifications
This is the hardest part and the most time-consuming. Every assignment has to have clear specifications for what earns a ‘Satisfactory’ or ‘Excellent’ grade. This can range from ‘score above a 75% on the test’ to ‘complete these seven specific things’ or ‘earn 2000/2500 points on the rubric’. For the two biggest assignments in the course (the final paper and midterm), I already had clear guidelines and rubrics so the work was minimal. But I have to do this for every homework assignment, the ethics training, the perfect paper—everything. It’s a bit of a pain, but worth it: I only have to do it once and will be able to reuse them moving forward, and it’s forcing me to clarify—to myself as well as the students–exactly what I expect in a given assignment.
Here are the specs I wrote for one assignment—the reaction paper that can earn students additional tokens.
- Paper must be submitted via Canvas no later than one week following the event, by midnight CST of that day. They must be uploaded into an unused ‘Token Assignment-Reaction Paper’ assignment.
- Paper must be submitted in .doc, .docx, or .pdf format. Corrupted files will automatically be graded ‘Unsatisfactory’.
- Paper must clearly indicate at the top the following information: your name, the title of talk/event; name, title and affiliation of all speakers; and location, time, and date of the event;
- Paper must summarize the key points of the talk. This should be complete enough that a non-attender would understand the main gist of the talk.
- Paper must include an analysis of the talk, with emphasis on the strongest and weakest points in the argument, application to related issues or events, and/or relevance to other texts, talks, or concepts of study in political science, international relations, or research methods. This section should be the bulk of the paper (at least 300 words).
- Paper must be at least 500 words, written in a professional style that has limited use of the first person, and no more than ten mistakes in grammar, spelling, or structure.
Step 6: Syllabus and Website Clean Up
Almost done! The last step is doing a final proofread of the syllabus o make sure that the policies and procedures reflected the new system. Policies on course absences and late work, for example, needed to be replaced with reminders about the token system. In the schedule, “problem set 1 due date’ changed over to ‘Required Homework Modules 1, 2, and 3’ due.
I also had to set up the learning management system (LMS) to work for the new class. Canvas does not let me copy over a previous course, so I set it up from scratch each term. Everything was pretty much the same as in previous years—I post the syllabus, assignment information, readings, etc.—except that now I have many, many more assignments to create—27 just for the homework modules.
Overall this entire process took about a day of work to complete. I’m fairly satisfied with the results. My biggest worries right now are 1) if I got the balance right in the bundles, requiring enough of D students and not too much of A students; 2) whether I missed any small details that will force me to make tough decisions later; and 3) whether the students will like the system or completely rebel.
That last question will be the subject of my next post, where I talk about Day 1 reactions of the students.