Does the Question Determine the Answer?

Regular readers of this blog know that I sometimes ponder the clarity of my assignment and exam prompts (some past posts on this subject are here, here, and here). Students sometimes don’t hit what, in my mind, the question targets, so I revise in the hopes of creating a prompt that is more transparent. But I don’t want prompts to be answerable with a Jeopardy-like regurgitation of facts. I want students to exert some cognitive effort to figure out how to apply concepts that are relevant to the question at hand.

Usually this situation occurs with my undergraduates, but I’m noticing it more frequently with master’s degree students. A recent example is an assignment from my graduate-level introduction to comparative politics course:

Continue reading

What Do Grades Mean?

What do grades actually mean? I began pondering this question while designing a course for the fall semester. Theoretically a grade indicates the amount of knowledge or skill that a student possesses. But really? Those of us working in the USA are quite familiar with grade inflation. A final grade of C today probably doesn’t indicate the same level of knowledge or skill proficiency as the C from fifty years ago. There is also the persistent problem of knowing whether our assessment tools are measuring the types of learning that we think they are/want them to. And it is probably safe to assume that, both in and out of the classroom, there is a lot of learning happening but we just aren’t interested in trying to measure it. The situation gets even more complex given that — again, in the USA — a “learning activity” often won’t function as intended if students believe that it has no discernible effect on their course grades.

I structure my syllabi so that the sum total of points available from all assessed work is greater than what it needed for any particular final grade. For example, a student might need to accumulate at least 950 points over the semester for an A, but there could be 1,040 points available. I do this to deliberately create wiggle room for students — with so many assignments, students don’t need to get perfect scores on, or complete, all of them. While this leads to higher grades in my courses than if I graded strictly on a bell curve, I want to give students plenty of opportunities to practice, fail, and improve. And I firmly believe that sloppy writing indicates slopping thinking, while good writing indicates the converse. So in reality what I’m doing with most of my assignments is evaluating the writing abilities of my students.

This system often produces a bimodal grade distribution that is skewed to the right. Expend a lot of effort and demonstrate a certain level of proficiency, and you will get a grade somewhere between an A and a B-. Choose not to expend the effort, or consistently demonstrate an inability to perform at a minimum level, and you will get a D or an F. I’m comfortable with this result, in part because I know from the cognitive science research on learning that repeated exposure and frequent testing builds long term memory.

This leads me to the reason for doubting that grades my courses mean the same thing as they do in courses where the only assessment is done through mid-term and final exams composed of multiple-choice questions. Yes, the proportion of A’s in the latter might be lower than in the former, but I bet on average my students are retaining more. At least I like to think that’s the case. There is no way for me to be sure.

Research the Write Way

Credit for the subject of today’s post goes to Lindsay LaChapelle, writing center acting director, and Alicia Vaandering, education and instructional design librarian, at my university. They can be contacted, respectively, at lindsay [dot] lachapelle [at] salve [dot] edu and alicia [dot] vaandering [at] salve [dot] edu.

I attended a presentation by the above-mentioned individuals on leveraging the expertise of academic support staff when using research-driven writing assignments. Undergraduates trying to complete these assignments typically lack proficiency in critical skills like:

  • Identifying peer-reviewed journals for literature reviews.
  • Knowing when and how to cite sources.
  • Decoding the rhetorical techniques of scholarly literature.
  • Applying the authentic writing principles of role, audience, and format.

For example, students often locate a pile of sources that they believe are pertinent, and only then begin writing. Or they do the writing first and then seek out what they think are a sufficient number of not-really-that-relevant sources to cite in the text.

Incorporating instruction from your university’s writing and research specialists can help solve these problems. The first step in the process is to consult with writing center and library staff, to share assignment information, identify your goals and expectations for the assignment, plan classroom workshop content, and schedule the workshops in relation to assignment due dates.

Workshop topics should reflect the four phases of the research writing process: exploration, reflection, revision, and becoming a creator of information. Each workshop can target one or more of the specific objectives for each phase, as shown in this table. “L” indicates a practice that normally falls within a librarian’s area of expertise, while “W” indicates a focus on writing. Think of the table as a menu from which to choose the skills in which your students most need training.

The classroom workshops allow both academic support staff and the course instructor to interact with students about their work as they proceed through the research and writing process. They can include probes of students’ pre-existing knowledge of college-level research and writing principles and the review of students’ work as they complete different components of the assignment.

Collaborating with librarians and writing specialists benefits both me and my students because it:

  • Reduces redundancy in instruction.
  • Shares the workload for lesson planning, delivering instruction, and assessment.
  • Reduces the instructor-to-student ratio in the classroom.
  • Models the collaborative nature of academic research.
  • Allows writing center staff to identify research-related problems that require librarian support, and vice versa — instead of leaving students in the dark about the kind of specialized help they might need and who can provide that help.

When Education Harms

Alternative title for this post:

Why Many Teacher-Training Programs Should Be Abolished

Perhaps some of you — at least in the USA — have noticed the phenomenon of college students using multi-syllabic words out of context. The student clearly does not know what the word he or she inserted into the sentence actually means.

I used to think this was an attempt to impress me in the hopes of getting a higher grade on the writing assignment — pull a complicated-sounding but inaccurate word from an online thesaurus instead of using something simpler. But perhaps the behavior is really a sign that the student is deficient in some basic literacy skills.

As pointed out in this National Public Radio story, millions of children in the USA do not learn how to read well at an early age because of the unscientific and ineffective methods used by their teachers. If children fall behind in the first few years of primary school, it’s probably difficult for them to become proficient readers later on. I’m now wondering if these deficits in literacy persist all the way into college.

Negatives of Nuance

Related to Simon‘s and Amanda‘s recent posts about failing to get the results one expects, here is a very simple example.

My first writing assignment prompt this semester for my course on economic development and environmental change read as follows:

Purpose of this response: learn about the nature of poverty and perceptions of the poor.

1. Read the rubric below.

2. Read/watch:

  • MRU: Basic Facts of Growth and Development.
  • Easterly, Ch. 1 and Intermezzo, p. 5-19.
  • Emily Badger and Quoc Trung Bui, “In 83 Million Eviction Records, a Sweeping and Intimate New Look at Housing in America,” The New York Times.
  • Hans Rosling, “The Magic Washing Machine,” TEDWomen, December 2010.

Answer the following question:

  • William Easterly writes “When those of us from rich countries look at poor countries today, we see our own past poverty.” (p. 19). Is this an accurate view of poverty outside and inside the USA? Why?

My intent in asking the question — which I thought was obvious from the “statement of purpose” that prefaced it — was to get students to think about the differences between how they think about the poor at home and about the poor in countries that they’ve probably only seen on the news, if at all. In other words, I wanted them to start thinking about why our perceptions of the poor often depend on where the poor are located.

Instead students wrote about wealth and poverty from a historical perspective — that the USA once had a low level of economic development, as measured by GDP or income per capita, and that’s where other countries are today. The USA developed into a high-income, industrialized society, so probably other countries, even the poorest ones, will eventually do the same. There was no real discussion of the nature of poverty in the USA or why it exists.

So I need to a better question, something like “Are the causes of poverty in the USA the same as in other countries? Why?” or “If the USA is an economically-developed country, why does it still have poor people?”

Beyond the Essay: Briefing Memos

Today we have a guest post from Vincent Druliolle, an assistant professor at Universidad Carlos III de Madrid. He can be reached at Vincent[dot]Druliolle[at]gmail[dot]com.

Undergraduates are repeatedly told that what they study is somehow relevant for practice, yet most assignments are structured as academic essays—even though only a handful of them will end up opting for an academic career. A few years ago, I decided that my students should have the opportunity to develop non-academic writing skills, and started assigning a briefing memo about an ongoing conflict.

The briefing memo is indeed a format widely used in government, international organisations, consultancies, and NGOs. However, because of the large range of topics and theoretical perspectives covered by my module and the limited number of teaching weeks, I had to find a way of integrating such an activity into my small-group seminars. I came up with the idea of making the memo a preparatory activity for my in-class simulation on peacebuilding and transitional justice.

The briefing memo differs from the traditional essay in both content and format. It is policy-oriented, because it is aimed at practitioners and decision-makers, and it presents information in a concise and attractive manner. It requires critically analysing source material beyond the standard academic literature, selecting what’s most relevant, and presenting it in a way that can convey the complexities of the conflict analysed.

Most students have never written a memo, but I don’t give them any guidelines. Instead, I ask them to look up examples that they can use as models. I prefer to ask the students to present their memos in class and discuss the difficulties of writing it. The first seminar of the simulation is thus about comparing and learning from the work of one’s fellow classmates. For class discussion, I recommend selecting at least a very good memo, a (very) bad one, and a few with significantly different formats and/or content. The greater the variety of memos, the better. I want the students to learn from each other, so I adopt the role of a facilitator, asking them to explain why they’ve chosen a given format and/or content, and fostering a class discussion about these aspects.

Many students admit that, as I warn them beforehand, it’s difficult at the beginning to figure out how they have to write the memo. Instead of assessing it at this stage, I ask the students to submit a revised version after the simulation that reflects what they’ve learnt from their classmates’ memos. Guidelines about how to write a memo can be provided at this stage or even afterward as part of a debriefing.

While writing the memo is an activity in its own right, in this case it is also a way for the students to acquire the knowledge necessary to participate in the simulation. They learn what information the memo should include because they have to put themselves in the shoes of the actors for whom the memo is written in the first place. In this way, the memo prepares students for the simulation, while the simulation provides an opportunity for them to reflect on the writing of the memo. And for the instructor, memos are quicker (and less boring) to mark than essays.

The Benefits of Recycling

Today’s post is more about career development than teaching . . .

Academia is a bureaucratic work environment. Information is constantly documented and distributed. Often this happens to the same piece of information multiple times. Consequently I began recycling my writing as much as possible several years ago, in the belief that it is better to make minor changes, or none at all, to writing upon which I have already expended mental energy. An underlying principle here is writing with an ultimate rather than a proximal use in mind. What is the most valuable end to which this writing can be eventually directed? A simple example: the proposal for your conference presentation becomes the abstract for the conference paper, which in turn becomes the abstract for the manuscript submitted to a journal.

A second and, for some, more important example: the stream of email, editorial comments, draft committee proposals, and other written minutiae that one produces — it’s all work. Don’t let it disappear into the ether. Instead, use it for future contract renewal, tenure, or promotion.

I admit that I didn’t fully recognize the potential value of this writing until my wife — also an academic — compiled her application dossier for promotion to full professor. Watching her, I realized that, in the course of my day-to-day business as an associate professor and department chair, I had generated chains of emails and memos that constituted evidence of service and scholarship à la the Boyer model. I saw that this material, if organized coherently, could form much of my own application for promotion, in many cases verbatim. Continue reading

Back End Skills

Most ALPS posts deal with the front end of teaching — the stuff that eventually turns into the student experience. Today I’m going to talk about the back end of the job: skills that are beneficial for one’s career because they have applications far beyond the classroom environment. Here are the skills that I now wish I had acquired while in graduate school:

Writing for the Audience

As I’ve mentioned occasionally in the past, the vast majority of academic writing is terrible. It is produced to be published, not to be read. Important ideas are not communicated well, if at all. For example, compare the writing of Anatol Lieven in Pakistan: A Hard Country to any journal article or multi-authored volume about that country. Or read Sarah Kendzior‘s The View From Flyover Country. These people can write well, a lot of people read what they write, and they have benefited professionally as a result.

Academics need to reach different audiences, and that requires learning how to write for those audiences. Take courses in journalistic or creative writing. Write memos. Submit op-eds to your local newspaper. Get feedback from people who write better than you do. Write a lot, even though it takes time. Use the process of writing as a tool to refine your thinking. Practice what we preach to students.

Graphic and Web Design

I’ve written about this before too — messages can and often should be communicated visually. But the message is lost if the visuals are bad. I’m often shocked by the inability of faculty members to display information in a manner that is easy to understand — whether for other academics or a curious and reasonably intelligent public. Creating simple but effective charts with Excel is not that difficult. Yet training in this basic skill was not part of my graduate program — I had to learn it on my own. Others probably never bothered.

My doctoral studies began just before the Web sprang into existence. Since then, I’ve been struggling to catch up with the digital revolution. This blog is one small tangible result. Don’t be left behind like I was — learn how to build websites. The more proficient at this you become, the more of an advantage you will have.

Data Literacy

Related to the above is the ability to work with data. Can you easily mine data by creating longitudinal analyses and calculating percentages? Do you know how to determine whether your data and conclusions are meaningful?  I am constantly amazed by what I can learn and communicate by making those simple Excel charts. I dream about what I could do if I knew R.

Stage Presence

Let’s face it: teaching is performance. As are committee meetings, admissions office recruitment events, and board meetings. Elocution and body language can make or break a conference presentation. Don’t be the person whom everyone immediately tunes out. Take a course in public speaking, acting, or musical theater.

People Management

We have to interact with others as part of larger organizations, and I bet every person who reads this has encountered at least one toxic colleague in their careers. Some of us end up with managerial duties, as research team leaders, department chairs, and administrators, yet we’ve never been trained for these roles. I recently attended a workshop on how to manage difficult conversations in the office, and it was eye-opening. Find out how you can become better at working with people. Then do it.

Another Example of What You Say Is Not What They Hear

Probably how students perceive me

Back in April, I gave an example of students comprehending a question differently than I did. In that case, I identified what I thought may have caused the miscommunication — the question needed to be worded slightly differently.

I now have another example, in an online graduate course. The question was “Of the different political and religious reactions to European imperialism by Middle Eastern societies, which was the most effective? Why?” This writing prompt corresponded to chapters from No God but God by Reza Aslan and The Modern Middle East: A History by James L. Gelvin. Students referenced information from these books, something I require, so I know that they actually read the assigned material.

Instead of writing about the ways in which Middle Eastern societies responded to colonization, several students submitted answers that discussed: Continue reading

Teaching with Trump: A Challenge and an Invitation to Problem-Based Learning

Our series on “teaching Trump” continues today with a guest post by Martin S. Edwards, Associate Professor and  Director of Graduate Studies at the School of Diplomacy and International Relations, Seton Hall University. He can be reached at Martin [dot] Edwards [at] shu [dot] edu.

Talk to the hand.

Teaching in a professional international affairs program offers some unique challenges. The focus at the graduate level is on professional skill development rather than preparing students for comprehensive examinations, and our students come from a wide variety of disciplines rather than as political science undergraduates. The election brought a unique challenge for those of us that teach International Organizations. It made clear that we need to better train students to speak up in defense of multilateralism, especially at a time in which the White House is suspicious of it. So, given the current climate, in which the White House is openly dismissive of the value of the United Nations, the World Trade Organization, the International Financial Institutions, and NATO, how can one rethink an established course so as to demonstrate its relevance? My approach to “Teaching with Trump” during the Spring semester was to refocus my graduate course on problem-based learning. Continue reading