Does the Question Determine the Answer?

Regular readers of this blog know that I sometimes ponder the clarity of my assignment and exam prompts (some past posts on this subject are here, here, and here). Students sometimes don’t hit what, in my mind, the question targets, so I revise in the hopes of creating a prompt that is more transparent. But I don’t want prompts to be answerable with a Jeopardy-like regurgitation of facts. I want students to exert some cognitive effort to figure out how to apply concepts that are relevant to the question at hand.

Usually this situation occurs with my undergraduates, but I’m noticing it more frequently with master’s degree students. A recent example is an assignment from my graduate-level introduction to comparative politics course:

Continue reading

What Do Grades Mean?

What do grades actually mean? I began pondering this question while designing a course for the fall semester. Theoretically a grade indicates the amount of knowledge or skill that a student possesses. But really? Those of us working in the USA are quite familiar with grade inflation. A final grade of C today probably doesn’t indicate the same level of knowledge or skill proficiency as the C from fifty years ago. There is also the persistent problem of knowing whether our assessment tools are measuring the types of learning that we think they are/want them to. And it is probably safe to assume that, both in and out of the classroom, there is a lot of learning happening but we just aren’t interested in trying to measure it. The situation gets even more complex given that — again, in the USA — a “learning activity” often won’t function as intended if students believe that it has no discernible effect on their course grades.

I structure my syllabi so that the sum total of points available from all assessed work is greater than what it needed for any particular final grade. For example, a student might need to accumulate at least 950 points over the semester for an A, but there could be 1,040 points available. I do this to deliberately create wiggle room for students — with so many assignments, students don’t need to get perfect scores on, or complete, all of them. While this leads to higher grades in my courses than if I graded strictly on a bell curve, I want to give students plenty of opportunities to practice, fail, and improve. And I firmly believe that sloppy writing indicates slopping thinking, while good writing indicates the converse. So in reality what I’m doing with most of my assignments is evaluating the writing abilities of my students.

This system often produces a bimodal grade distribution that is skewed to the right. Expend a lot of effort and demonstrate a certain level of proficiency, and you will get a grade somewhere between an A and a B-. Choose not to expend the effort, or consistently demonstrate an inability to perform at a minimum level, and you will get a D or an F. I’m comfortable with this result, in part because I know from the cognitive science research on learning that repeated exposure and frequent testing builds long term memory.

This leads me to the reason for doubting that grades my courses mean the same thing as they do in courses where the only assessment is done through mid-term and final exams composed of multiple-choice questions. Yes, the proportion of A’s in the latter might be lower than in the former, but I bet on average my students are retaining more. At least I like to think that’s the case. There is no way for me to be sure.

Call for Proposals

The next New England Faculty Development Conference will be held on November 8 at the College of the Holy Cross in Worcester, Massachusetts. The deadline for proposals is August 17. Full details are here. The NEFDC is totally teaching-oriented and interactive workshops are encouraged.

As the new Director of Faculty Development at my university, and managing editor of this blog, please get in touch if you would like to publicize a teaching-related conference or event.

What Sticks?

Inside Higher Ed recently published a column written by a community college dean on the most important subjects one took or could take in high school — part of a larger conversation that originated on Twitter. Responses to the column mentioned:

  • Theater productions, to learn how to work with other people who have different perspectives and objectives.
  • A foreign language, to learn principles of grammar that allow one to become a better communicator in English.
  • Typing, to learn how to communicate more quickly with less effort.
  • Bookkeeping, to learn how to manage one’s personal finances.

Comments also referenced the processes through which the learning occurred. For example, one person mentioned that he gained a better understanding of the here and now when a history teacher worked backward from the present instead of using the traditional method of moving from the distant past toward today (which in high school is almost never reached).

The column and the comments got me thinking about the same question as applied to college. What undergraduate course was the most useful to you, and why?

As I wrote a few years ago, I generally don’t remember anything about the content of my college courses. Sorry, James Clerk Maxwell, I’ve forgotten how to use your equations. But I do have memories of what actions I performed when I originally learned the content and how I felt when that happened. The general process stuck. The specific outcome did not.

While I have tried in my own teaching career to better emphasize process over content, I still don’t get the kind of feedback contained in the Inside Higher Ed piece. My university doesn’t collect data on this level from alumni. So maybe it’s time I started doing it myself with a survey.

Research the Write Way

Credit for the subject of today’s post goes to Lindsay LaChapelle, writing center acting director, and Alicia Vaandering, education and instructional design librarian, at my university. They can be contacted, respectively, at lindsay [dot] lachapelle [at] salve [dot] edu and alicia [dot] vaandering [at] salve [dot] edu.

I attended a presentation by the above-mentioned individuals on leveraging the expertise of academic support staff when using research-driven writing assignments. Undergraduates trying to complete these assignments typically lack proficiency in critical skills like:

  • Identifying peer-reviewed journals for literature reviews.
  • Knowing when and how to cite sources.
  • Decoding the rhetorical techniques of scholarly literature.
  • Applying the authentic writing principles of role, audience, and format.

For example, students often locate a pile of sources that they believe are pertinent, and only then begin writing. Or they do the writing first and then seek out what they think are a sufficient number of not-really-that-relevant sources to cite in the text.

Incorporating instruction from your university’s writing and research specialists can help solve these problems. The first step in the process is to consult with writing center and library staff, to share assignment information, identify your goals and expectations for the assignment, plan classroom workshop content, and schedule the workshops in relation to assignment due dates.

Workshop topics should reflect the four phases of the research writing process: exploration, reflection, revision, and becoming a creator of information. Each workshop can target one or more of the specific objectives for each phase, as shown in this table. “L” indicates a practice that normally falls within a librarian’s area of expertise, while “W” indicates a focus on writing. Think of the table as a menu from which to choose the skills in which your students most need training.

The classroom workshops allow both academic support staff and the course instructor to interact with students about their work as they proceed through the research and writing process. They can include probes of students’ pre-existing knowledge of college-level research and writing principles and the review of students’ work as they complete different components of the assignment.

Collaborating with librarians and writing specialists benefits both me and my students because it:

  • Reduces redundancy in instruction.
  • Shares the workload for lesson planning, delivering instruction, and assessment.
  • Reduces the instructor-to-student ratio in the classroom.
  • Models the collaborative nature of academic research.
  • Allows writing center staff to identify research-related problems that require librarian support, and vice versa — instead of leaving students in the dark about the kind of specialized help they might need and who can provide that help.

Emotional Labor in the Digital Age

Katherine Wright’s recent post on Twitter as a hostile environment caught my attention, especially her point about employer expectations that faculty “engage” with social media.

If you’re like me, you never received any formal training in using social media for professional purposes. Whatever you see from me on this blog, it’s self-taught. A learn-as-you-go process where mistakes, whether technical or editorial, get made. But the consequences of these mistakes are quite small given that the format allows me to retain most of the privileges (male, Caucasian, overly-educated, etc.) that I enjoy in so-called meatspace. Comments are few, come from like-minded colleagues, and in the end, moderated. The content doesn’t attract hackers, doxxers, or nuisance lawsuits.

That is not the digital world that many of us live in. We know that some people, because of their identities, are regularly subjected to negative biases in the work environment. Expecting them to communicate in mediums where such biases can be infinitely amplified, while not preparing them for what they are likely to encounter, is simply disastrous.

Making matters worse: employers that want their employees to act as semi-public figures for marketing purposes often expect those employees to endure the harassment and abuse that they subsequently receive, or, as if this were somehow possible, prevent the harassment and abuse from occurring in the first place by tailoring their communication to the needs of an anonymous and dysfunctional audience.

Cause for Pessimism?

A bit of self-promotion, which most academics need to become better at . . .

Yesterday Inside Higher Ed published an op-ed of mine on a simple metric that helps identify which small colleges and universities in the USA are at risk of failure. I wrote a bit about the metric on this blog in December. If you work at a private, non-profit college with fewer than two thousand undergraduate students and an endowment of less than half a billion dollars, you will probably want to read the op-ed. Feel free to add to the comments.

More Changes to a Course on Development, Part 3

The final exam for this course last year asked each student to write an economic rationale in support of one of two policy options, using information from course readings as evidence. Generally students students did not do well on the exam, mainly because they did not discuss applicable concepts like moral hazard and discounting the future. These concepts were found in several course readings and discussed in class. While I didn’t explicitly mention these concepts in the exam prompt, the benefits of including them in the rationale should have been obvious given course content.

Now I’m thinking of a question like this for the final exam:

What has a greater influence on economic development in Egypt: law (institutions) or geography (luck)? Why?

In your answer, reference the items below and relevant course readings listed in the syllabus:

The downside here is that I’m giving up an authentic writing exercise in favor of (I assume) even more transparent alignment with course objectives.

More Changes to a Course on Development, Part 2

My original design for this course included a design thinking component organized in two stages. In the first stage, teams applied SCAMPER to California Water Crisis, a freeware board game. Although the subject of water scarcity was quite relevant to the course, the game’s mechanics were not the most engaging. This should have made it easy for students to think of significant SCAMPER-based improvements, but their recommended changes were relatively superficial. The graded writing assignment tied to this activity also left much to be desired.

In the second stage, students were asked to apply SCAMPER to an existing game other than California Water Crisis. Two problems popped up here. First, teams chose very simple games to modify — think Chutes and Ladders (and without even any awareness of its Indian origins or its connection to British imperialism). Second, although I specifically directed them to place the new game in a specific context, like a city, this didn’t happen.

This time around, I’ll be having students play Stop Disasters and Wingspan. Teams will have to apply SCAMPER to one of these two games. Although they both connect well to the course’s subject, neither game is ideal. Stop Disasters is problematic because it is Flash-based. Wingspan requires, where I work, a significant departmental budget outlay of $100 per game, and I have to purchase five of them. Given the dimensions of Wingspan’s box, transporting all five at once could be a problem. The campus building in which I work is not ADA-compliant (my office, perhaps appropriately, is at the top of what originally was the servants’ stairwell).

Instead of selecting something different for the second design round, teams will stick with whichever of the two games they chose for the first round. While students will be free to choose any subject related to the course for the new game they are designing, it will have to be set in the city in which the university is located. I hope to locate some online data visualizations — maps of flood zones, public transportation routes, property tax assessments, etc. — to help students with this.

After the initial SCAMPER-based redesign, each team will play another team’s game. In an individual writing assignment, students will evaluate the games they played according to the game design principles referenced in the same assignment from last year. I will provide each team with the feedback it receives from the other students.

For the next phase, teams will, I hope, use SCAMPER as a means of applying feedback to improve their game designs. Then there will be another round of play testing, with another written evaluation. I might make this second evaluation a mechanism by which teams earn points on the quality of their games, as assessed by other students. That could heighten students’ investment in the design process. I will probably also need to include a means for students to evaluate the work of their teammates on this project over the semester — something I do regularly in my other courses.

The Business of Small Colleges Is . . . Not Business

Four recent news items that I believe are additional evidence of a trend that is going to poke a large hole in the finances of many small, private colleges and universities in the U.S. — a hole that is going to help make some of these institutions insolvent:

First, an online program will replace the residential MBA at the University of Illinois at Urbana-Champaign.

Second, Iowa Wesleyan University has signed an articulation agreement with Kishwaukee College, a community college located three hundred miles away in another state, whereby students complete three years of study at Kishwaukee and obtain a bachelor’s degree in business from Iowa Wesleyan with a fourth year of online courses.

Third, Linfield College is shedding approximately one-fifth of its tenured and tenure-track faculty positions. Undergraduate enrollments at Linfield’s main campus and in its online programs have declined precipitously, while the size of its nursing program in Portland has been flat. My guess is that the college’s leadership has decided to radically shrink its liberal arts curriculum and focus on offering degrees in business in addition to nursing.

Fourth, as described in this New York Times article, data indicates that there is often an inverse relationship between the cost of a graduate program and its quality.

Business programs, both undergraduate and graduate, have been cash cows for small U.S. colleges and universities for decades. But in 2019, why pursue this field of study at a small, struggling, largely-unknown college when one can get the same degree online at less expense from a much larger, better-resourced, nationally-reputable state university? Iowa Wesleyan has been in a death spiral for some time. The chances of its articulation agreement attracting enough business degree-completion students to pull the school back from the brink are slim to none. The same is true of an institution like Linfield College. Offering business degrees at a higher price than more-prestigious competitors does nothing to differentiate either institution in the market.