More Pre-Post Post-its

A follow-up about asking students why they do what they do . . . For the second stage of this data-gathering exercise, I had students use Post-its to anonymously answer three questions at the beginning of class:

  • How are you feeling right now? (the one-word check-in)
  • Why are you feeling what you’re feeling?
  • Why did you come to class today?

Nineteen out of twenty-three students, or more than eighty percent, reported feeling badly — the same proportion as last time. Of the nineteen, ten referenced being tired while four wrote “stressed.” Only one wrote “hungry.” The overwhelming majority of people in this group attributed their feelings to too little sleep and too much work.

The other four students felt “happy,” “good,” “relaxed,” and “chill.” Three of these students attributed their feelings to having had time to eat, buy coffee, or otherwise get ready before class. One of them mentioned sleeping comfortably, while another wrote “not super-stressed . . . trying to stay calm for the day ahead.”

I sorted answers to the third question into a few different categories, which are shown below, along with their frequencies. A few students’ comments fell into more than one category.

  • I had to; attendance is mandatory: 7
  • Get a good grade: 5
  • I am paying for the course: 3
  • Learn something: 3
  • Participate in discussion: 1
  • Collaborate with teammates on an upcoming assignment: 3
  • Miscellaneous reasons — “My roommate told me I couldn’t skip,” “I was awake so I figured why not,” “Because I didn’t go to the last one,” “I try to go to all of my classes,” “Didn’t want to miss anything,” “To avoid falling behind”: 6

In sum, only seven students, or thirty percent, indicated that they had been intrinsically motivated to attend class that day; i.e., they came to learn or participate in a learning-oriented activity. More than half of the students indicated that they were extrinsically motivated by the fear that their grades would be harmed if they did not attend. What I think is interesting here: I do not penalize students for being absent from class — I regard them as legal adults, free to suffer the natural consequences of their actions. I do not grade on attendance or class participation. Only students’ written work, submitted before class, gets assessed.

More thoughts on this subject in a future post . . .

Information Literacy as Research Methods III


R M Media Ltd, CC BY-SA license

An important component of both statistical and information literacy is the ability to recognize the difference between correlation and causation. Teaching this skill is made even more difficult by cognitive biases that lead to errors in probabilistic thinking.* So I decided to hit my students over the head with Chapter 4 from Charles Wheelan’s Naked Statistics and, from Tyler Vigen’s Spurious Correlations website, an image of the 99.26% correlation between the divorce rate in Maine and margarine consumption.

The assignment asked students to submit a written response to this question:

Why are these two variables so highly correlated? Does divorce cause margarine consumption or does margarine consumption cause divorce? Why?

All the students who completed the assignment answered the question correctly: neither one causes the other. In class, students identified several possible intervening variables, including:

  • People eat margarine and margarine-laced products as an emotional comfort food when relationships end.
  • Divorce leads to a greater number of households, with each household purchasing its own tub of margarine.

Students’ ideas led in turn to a discussion of how to appropriately measure these variables and construct new hypotheses.

*An excellent overview of this topic is Jack A. Hope and Ivan W. Kelly, “Common Difficulties with Probabilistic Reasoning,” The Mathematics Teacher 76, 8 (November 1983): 565-570.

Links to all posts in this series about information literacy:

Information Literacy as Research Methods II


R M Media Ltd, CC BY-SA license

Another post about the methods course that I’m now teaching. Chapter 3 of Naked Statistics is about deceptive description. So here is the accompanying assignment . . .

Many high school seniors are interested in attending Southwest America State University for college. Before 2015, applicants to this university had to submit high school transcripts that include average GPA scores, SAT scores, and an essay. In 2015, the application process changed; applicants had to submit high school transcripts with average GPA scores and two essays, while submission of SAT scores became optional. In 2019, the university claimed that the academic quality of its students had increased since 2011 given this pattern in the average SAT score of each year’s incoming class:

  • 2011 – 990
  • 2012 – 1130
  • 2013 – 1090
  • 2014 – 1150
  • 2015 – 1160
  • 2016 – 1185
  • 2017 – 1170
  • 2018 – 1190

Is this claim deceptive? Why?

Continue reading

Pre-Post Post-its

Sometimes the best way to find out why students do what they do is to ask them.

During a recent lunchtime conversation with a colleague, I learned about the “one-word check-in” — asking students to each describe, with a single adjective, how they felt at that moment. I decided to incorporate this into a data collection exercise that I hoped would demonstrate one benefit of taking notes in class — a problem for which I still haven’t figured out a solution.

My hypothesis: students who took notes — a more cognitively-engaging activity than just listening — would be more likely to feel better by the end of class.

I collected data in my course on globalization, which meets twice a week in seventy-five minute sessions from 9:30 a.m. to 10:45 a.m. The class, when everyone attends, has only twenty-five students, so my results are not statistically significant.

As students were entering the classroom and settling into their chairs, I gave each person three Post-it notes, along with a playing card dealt from a stacked deck (more on this further down). I told everyone to marked their Post-it notes with the suit and number of the playing card each had received. This allowed me to sort the Post-its by individual student afterward. Students should also number each Post-its with a 1, 2, or 3, to simplify keeping them in the correct sequence after class. I didn’t think of this at the time, but luckily I kept each pile of Post-it notes separate after they were collected.

The data:

  • At the beginning of class, students wrote a one-word check-in on Post-it #1.
  • After the discussion of that day’s reading response, students wrote on Post-it #2 answers to “Have I written any notes during today’s class?” and “Why?”
  • Students then clustered into teams to discuss plans for an upcoming project assignment. Note that this introduces a methodological flaw in my research design, but it turned out to be irrelevant.
  • At the end of class, students wrote a one-word check-out on Post-it #3.

A different randomly-selected student collected each set of Post-it notes after students had finished writing on them, which he or she placed face down on a table. The goal here was to make it obvious that I was trying to preserve the anonymity of students’ responses. However, I had dealt cards from a stacked deck (low value cards on the bottom) so that I could identify which responses were from men and which were from women — because I expected that women would be more likely to take notes.

Now for the results. Out of 23 students who were in class that day . . .

Continue reading

Another One Bites The Dust

Postscript to my February 14 post on colleges and universities in financial trouble: the College of New Rochelle, whose enrollment woes I profiled in 2017, will probably close this summer. Details are contained in this Inside Higher Ed story.

According to its federal tax filings, the College of New Rochelle had a negative net operating margin for every single fiscal year from 2011 through 2016. During this period its operational expenses per FTE undergraduate increased by almost 45 percent. The increase for fiscal years 2012 through 2016 was nearly 53 percent.

New Rochelle is yet another example of a private, non-profit college that did not sufficiently reduce its operational costs when enrollment plummeted after the Great Recession. Expenses per student ballooned until the college became insolvent.

Information Literacy as Research Methods I

Image by R M Media Ltd under a Creative Commons Attribution-ShareAlike license
R M Media Ltd, CC BY-SA license

One of the joys of being department chair is creating a curriculum map for information literacy learning outcomes — as part of a five-year program review for a department that is only two years old. Since I’m teaching research methods, a requirement for students in all three of the department’s interdisciplinary majors, I decided to make information literacy a focus of the course. I designed several brief assignments based on chapters in Charles Wheelan’s Naked Statistics that pertain to evaluating information sources for authority, reliability, and relevance. These tasks in turn complement, in my mind at least, two larger assignments: Amanda’s Best Breakfast in Town project and writing a research proposal.

I thought I’d post some of those assignments here on the blog along with an assessment of how well students did on them. First topic on the list is hypothesis construction:

Given the availability of mobile phone coverage in Ghana, Nigeria, and Tanzania, how can we infer which country is the most violent? Why? (Generate a hypothesis about a relationship between mobile phone coverage and violence.)

Students did a good job thinking of possible causal relationships between mobile phone use and violence. Class discussion included ways to operationalize the concepts of violence, wealth, and happiness, which we did with some quick internet research. Students did not find an association between homicide rate and the amounts of mobile phone coverage in Ghana, Nigeria, and Tanzania, which then led to the topic of sample size. The assignment seemed to work as I had intended.

Continue reading

That Giant Sucking Sound

Something of an update to my last post on the slow-motion tsunami in U.S. higher education: Green Mountain College and Oregon College of Art and Craft will close at the end of this semester. Essentially the same fate will befall Hampshire College, because its board of trustees has limited the Fall 2019 incoming class to only about seventy deferred and early decision admits. Few of them will enroll and current students will transfer out, hastening Hampshire College’s impending insolvency.

Applying my measurement of change in annual total expenses per FTE undergraduate from fiscal years 2011 to 2016 to these schools, I get the following percentages:

  • 27: Green Mountain College
  • 24: Oregon College of Art and Craft
  • 25: Hampshire College

Note that these figures are far lower than those for several of the colleges and universities listed in my last post. Does an increase of 25 percent or more over a six-year period in the average cost per full-time undergraduate indicate that a private, tuition-dependent, small-enrollment institution is at high risk of closure? I’ll say, “Yes.”

What’s the figure for the college or university at which you work?

Out of Control

Some comments on a recent study of active learning published in the journal PLOS One — “Knowing is half the battle” by Shaw et al. The study reports on data gathered in an introductory biology course that was taught with active learning techniques from 2013 to 2016. Post-course scores on a concept and skill inventory were significantly higher than pre-course scores, which the authors take as an indication that students learned. Inventory scores from traditionally-taught iterations of the course are not reported. Without a control group, we have no idea whether the new pedagogy is more effective at generating desired learning outcomes than the old one. This is the typical flaw in research on active learning.

But there is a silver lining to this study. The researchers also measured student perceptions. Over time, students increasingly attributed their learning to course structure and pedagogy. Student course evaluations usually correlate with grades, but in this case, grades did not significantly change from year to year. So it appears that students’ expectations about the course eventually aligned more closely with how the course was taught.

This points to a phenomenon that I have noticed anecdotally: if you suddenly adopt an active learning pedagogy, prepare to be criticized initially by students, especially if all the other instructors that students encounter continue to teach in a traditional way.

When Education Harms

Alternative title for this post:

Why Many Teacher-Training Programs Should Be Abolished

Perhaps some of you — at least in the USA — have noticed the phenomenon of college students using multi-syllabic words out of context. The student clearly does not know what the word he or she inserted into the sentence actually means.

I used to think this was an attempt to impress me in the hopes of getting a higher grade on the writing assignment — pull a complicated-sounding but inaccurate word from an online thesaurus instead of using something simpler. But perhaps the behavior is really a sign that the student is deficient in some basic literacy skills.

As pointed out in this National Public Radio story, millions of children in the USA do not learn how to read well at an early age because of the unscientific and ineffective methods used by their teachers. If children fall behind in the first few years of primary school, it’s probably difficult for them to become proficient readers later on. I’m now wondering if these deficits in literacy persist all the way into college.

The Pigeon’s Checklist for Classroom Game Design

Today we have a guest post by Lt Col James “Pigeon” Fielder, USAF, Associate Professor of Political Science at The U.S. Air Force Academy. He can be reached at http://www.jdfielder.com.

Interested in designing a classroom game, but have no idea where to start? Being a fan of classroom games, I developed this checklist to help me think through my own designs.  The only checklist items that I think are absolutely necessary are the objective and win conditions, as both are crucial for identifying the concepts you are measuring and providing students with clear and achievable goals. Other checklist items are dependent on your design. For example, if your game is not map-based, then a map and scale are not required, but a game with many pieces likely needs a detailed inventory. Game on!

  • Win Conditions: how the game ends.  Can be competitive (zero-sum) or cooperative (non-zero sum).  Games in which all teams can win are still challenging
  • Objective: what is the specific goal of your game?
  • Number of Players: helps the designer conceptualize the game size and boundaries. 
  • Level of Detail: abstract to elaborate setting.  Increased detail improves conceptual accuracy, but requires significantly more time to develop and play.  Not that abstract games are necessarily easier to design!
  • Inventory: all required pieces and parts to play the game. Be exhaustive, even down to number of spare rulebooks and pencils.  
  • Map or Board: visual display of the gameplay area. 
  • Scale: if the game requires length and volume measurement.  Example: each hex or square equals 1/6 of a mile. 
  • Course of Play: every step for running a game from start to finish. This will be the most detailed portion of the game. 
  • Combat Resolution: determining outcome of players cooperating or conflicting during the course of play.
  • Rewards and Punishments: mechanisms for players to advance or regress based on performance.
  • Measurement: scoring the game.  Can be qualitative (e.g. area of controlled space) or quantitative (number of points).
  • Arbitration: handling rule and player disputes.
  • Feedback: discussing game outcomes and recommended game improvements.
  • Glossary: define key terms.

Recommended Reading:

Asal, Victor. “Playing Games with International Relations,” International Studies Perspectives, Vol. 6, No. 3 (2006): 359-373. 

Dunnigan, James. Wargames Handbook, Third Edition: How to Play and Design Commercial and Professional Wargames. Lincoln, NE: iUniverse, 2000. 

Macklin, Colleen, and John Sharp. Games, Design and Play: A Detailed Approach to Iterative Game Design. Boston, MA: Addison-Wesley Professional, 2016. 

Sabin, Philip. Simulating War: Studying Conflict through Simulation Games. New York, Continuum, 2012.