Information Literacy Exercise

Today we have a guest post from Colin Brown, assistant teaching professor in the Department of Political Science at Northeastern University. He can be reached at colin [dot] brown [at] northeastern [dot] edu.

It seems safe to say that political scientists have some concerns these days about information literacy, and information literacy is likely an implicit learning outcome for many of us. This blog has provided a number of good exercises for bringing information literacy into research methods, reading academic research, and headline writing. Inspired by these examples, I attempted to include this skill in my introductory comparative politics class, where democratic (de)consolidation is a major topic. In theory, the class gives students enough background to start keeping up with events around the world—if they choose to do so.

The exercise I tried this year, now available on APSA Educate, forces them to update slightly-out-of-date readings on a country facing democratic backsliding (Poland) by finding out what’s happened there in the four or five years since they were published. Students were assigned to small groups, and each was given a different kind of source to examine during a class session. One group read newspaper articles, another examined democracy indexes, yet another searched Wikipedia, etc. Students then applied what they’d read to course concepts—has democracy gotten weaker or stronger in Poland since these were published? Students then discussed what they trusted or distrusted about each type of source, and the potential merits of each.

I had a few key goals for students:

  • Think about source material for future courses. In an intro course, students not only might be unfamiliar with how research articles work, but also may not have a lot of practice in thinking about online source credibility.
  • Understand that while sources vary in credibility, there are pros and cons to using even the most credible sources. For example, the students who looked at V-Dem, Freedom House, etc., got clear, direct answers to the exercise’s questions, but they also correctly pointed out that they had to accept these organizations’ conceptualizations of democracy. And less credible sources like Wikipedia still had things to offer if used carefully.
  • Bridge the gap between classroom learning and events in the broader world and show how what they’re learning might help them understand the news.

When I ran this exercise in class this year, I budgeted only about 25 minutes for it, when it turned out to need 40 minutes or more to give students enough time to look at multiple sources in their category. We ended up using another 25 minutes the next day but dividing the exercise into two sessions probably led to more shallow searching and a less systematic attempt to make sense of sources.

When running this exercise in the future, I will think more explicitly about the balance between handholding and allowing students to practice seeking things out on their own. Last time I provided a couple of search terms, told them to keep looking outward beyond these, and to keep a record of what they searched for (which as best I could tell no group did). Next time I will probably experiment with either giving students a fully curated list of search terms, so they can observe how this affects their search results, or, conversely, I might give them even more time to “flail” about on their own before offering suggestions.

What You Think Depends On Where You Stand

Our superb librarians survey students and faculty annually. Results from this year’s survey are in. Student responses to one of the questions:

Faculty responses:

Notice that the frequencies of responses from these two groups are essentially mirror images of each other. Students are extrinsically motivated by grades, so they think in instrumental terms: I need correctly formatted citations and the specified minimum number of sources. Otherwise my grade will be negatively affected. Knowing whether a source is reputable is far less important. Faculty think the reverse: the ability to locate scholarly source material and analyze information for bias matters most.

I have tried to solve this problem in the past, and could not find a satisfactory solution. Consequently, I have focused more on curating quality content for student to consume than on marking down students because of their reliance on websites that are top-listed in Google searches. In fact, it’s one of the reasons I decided to stop assigning traditional research papers.

Given the survey results though, the problem extends far beyond my small corner of the curriculum. I’m not going to solve it independently.

Readers might find these other posts on information literacy skills to be of interest:

The Methods Silo Effect and Fixing Poor Research Skills

Googling

Write Your Own Headlines Activity

Write Your Own Headlines Activity

This post comes from Chelsea Kaufman, assistant professor of political science at Wingate University. She can be contacted at c[dot]kaufman[at]wingate[dot]edu.

In teaching undergraduate research methods, I often find that the students are intimidated by the subject matter and don’t see its relevance to their lives. I have increasingly emphasized to students that it prepares them to be savvy consumers of political information wherever they might encounter it. This approach introduces an additional challenge, however: students often lack the information literacy skills to evaluate the sources that they access. If I want students to have the skills to evaluate the political information they encounter, I obviously need to teach them these skills. How exactly can this be accomplished? 

It is not enough to tell students which sources are acceptable, because people tend to trust information that aligns with their political predispositions. Simply lecturing to students about the dangers of misinformation can reinforce false beliefs and increase their distrust of reliable sources. 

To avoid this conundrum, I have students write their own headlines based on public opinion poll data. I first find a poll with results covered in several media outlets. I then send students a link to (or print out) the results of the poll, without providing them any context as to how it was covered in the media. After writing the headlines, students share them and compare theirs with those of their classmates and with published headlines about the data. Students learn to interpret data and evaluate whether it was given accurate coverage in the media. As the final part of the lesson, I then ask them to evaluate the polling methods used to obtain the data, by, for example, considering how a question’s wording might have impacted the responses. 

You can view detailed instructions for the activity on APSA Educate. You can also read more about this topic and find examples of additional activities in my article Civic Education in a Fake News Era: Lessons for the Methods Classroom or my chapter in The Palgrave Handbook of Political Research Pedagogy

Looking Backward and Forward

Expanding on my last post on failures from this semester:

From where I stand, information literacy skills are important, because they help one identify and demolish specious claims made by authority figures. An assignment that, for example, forces students to locate three peer-reviewed journal articles is practice in finding credible information. It also allows students to determine whether a topic is suitable for a semester-long research project.

To me, these outcomes are both beneficial and rather obvious. But from the students’ perspective, the assignment could simply be yet another meaningless hoop to jump through on the way to getting another A+ on a transcript. Given the sources many students cited in the different stages of their storymap projects, it looks like too many of them customarily take the latter approach to research.

Therefore, in future courses that involve research projects, I should create assignments that are limited to the task of locating scholarly sources and place those assignments at the beginning of the semester. I should demonstrate why this skill is useful outside of the classroom.

I’ve noticed a similar problem with student writing — really basic errors that indicate a lack of proofreading. I don’t expend more effort evaluating a student’s work than the student did creating it. But I do know that sloppy writing indicates sloppy thinking and that the former advertises one’s propensity for the latter to the rest of the world. Again, I should demonstrate early in the semester why it’s important to proofread one’s work before it reaches an audience. My favorite example? The missing Oxford comma that cost a dairy company US$5 million.

I’m also seeing, from the last few journal article worksheets students are submitting, that many still do not have a clear understanding of how evidence-based arguments are constructed in academic literature. An author typically poses a research hypothesis or question at the beginning of a journal article and concludes with the same hypothesis or question reworded as declarative statement. I.e., “Why is the sky blue?” in the introduction with “The sky is blue because . . . ” as the conclusion. Yet on worksheets some students are writing that the hypothesis is about one thing while the conclusion is about some other thing. So again, students need practice in understanding the components of a written argument in scholarly literature, and that practice needs to happen early in the semester.

In principle I’m talking about scaffolding. But many of my assignments are attempts at getting students to builds several different skills simultaneously. I think I need to disentangle my goals for these assignments so that they target only one skill at a time.

The Article Summary

(Photo credit: Joanne H. Lee, Santa Clara University)

Today we have a guest post about teaching the research process by Anne Baker, assistant professor of political science at Santa Clara University. She can be reached at aebaker [at] scu [dot] edu.

Getting students to use academic articles for research papers can be a challenge. In my experience, many students, even those in upper-level courses, are not familiar with search engines such as JSTOR, Lexus Nexus, or Political Science Complete. And if students do happen to use Google Scholar, they frequently rely on excerpts from sources instead of entire articles that they might not have access to. So, what can be done to replace these habits with better practices?

In my advanced writing course on the presidency, I have developed a class activity which provides students with skills they will need if they are going to successfully locate and utilize academic references for their research papers. First, I want them to be able to use the library’s website to access search engines. Second, I want them to understand that research is an iterative process. Sometimes you don’t find what you need for a variety of reasons and you should be able to determine what those reasons are—whether its human error, the need for a wider search net, or that no one has written on the topic (this last possibility always surprises the Google generation). Third, students need to become acquainted with the literature on the presidency, including the subfield’s primary journal, by discovering how research practices in political science have changed overtime, even in a subfield which remains largely qualitative.

I have students work in pairs and I provide them with two search terms related to the institution of the presidency (e.g. signing statements, executive orders, oath of office). I pick the search terms carefully knowing that some topics have no scholarship and represent dead ends and others have later but not earlier scholarship or vice versa. The first step of the activity provides instructions about how to first locate JSTOR on the library’s website and then how to access Presidential Studies Quarterly using JSTOR’s advanced search options. Helpfully, for the purposes of this activity, JSTOR only has copies of the journal until 2000. To access later copies, students have to use the Wiley database, which students have to figure out how to find.

For each search term, I have students locate one article published in the last few years and then another for 1995-2000—a total of four articles. Next, students identify the research question and method the authors used, noting whether it is qualitative or quantitative, the sources of data regardless of method, the type of analysis (e.g. text, interviews, statistical), and the date of publication. After they have their four articles and perform this analysis, I ask them to compare the results of both searches. Finally, we have a class discussion in which we explore road blocks and challenges encountered and review how the field has changed over time.

I have found that this activity makes students more likely to cite academic articles in their final research papers and use them more effectively to support their arguments. Students also exhibit a much better understanding of the subfield and are more likely to use the other search engines that they encountered while on the library’s website. And they learn that research takes time and requires shifting your strategies to find the information you need.

SIFTing for the Facts

The Winnowers, 1855, Gustave Courbet

Anyone interested in teaching students how fact check claims should visit the Check, Please! starter course. Check, Please! is a free training course in how to use the SIFT method to evaluate the accuracy of online information:

  • Stop.
  • Investigate the source.
  • Find trusted coverage.
  • Trace claims, quotes, and media back to the original context.

Check, Please! was created by the company Notion (which I have no financial interest in) but as stated on the website’s homepage, the content is intended for reuse and revision.

Scholarly Literature As a Conversation

Below is an example of a recent in-class exercise that I used in my comparative politics course, created in collaboration with our crack staff of librarians. The exercise is designed to teach students that the scholarly journal articles they are reading represent an ongoing conversation between experts. I have included the information students were supposed to locate in italics. Feel free to modify the activity to meet your own needs.

Warm Up

Which former president of the American Political Science Association was one of Dr. Raymond’s professors? (Lucian Pye) First person to submit the correct answer before time expires earns 10 points toward their final course grade.

Backtracking

Locate this article: Larry Diamond, “Why Are There No Arab Democracies?” Journal of Democracy 21, 1 (January 2010): 93-112.

In this article, find citations for academic journal articles about democracy in the Arab world. Locate these articles. Write their citations:

  • Alfred Stepan and Graeme B. Robertson, “An ‘Arab’ More Than a ‘Muslim’ Democracy Gap,” Journal of Democracy 14 (July 2003): 30–44.
  • Mark Tessler and Eleanor Gao, “Gauging Arab Support for Democracy,” Journal of Democracy 16 (July 2005): 82–97.
  • Amaney Jamal and Mark Tessler, “The Democracy Barometers: Attitudes in the Arab World,” Journal of Democracy 19 (January 2008): 97–110.

One of the above articles (Jamal and Tessler) contains a citation for an academic journal article written by another former president of the American Political Science Association. Locate this article. Write the citation:

  • Samuel Huntington, “Will More Countries Become Democratic?” Political Science Quarterly 99, 2 (Summer 1984): 193-218

In the above article, find citations for these two works: the original version of an academic journal article that you have read for this course, and a book that you have read about for this course. What are the citations for the two works?

  • Seymour Martin Lipset, “Some Social Requisites of Democracy: Economic Development and Political Legitimacy,” American Political Science Review 53 (1959): 75.
  • Barrington Moore, Jr., Social Origins of Dictatorship and Democracy (Boston: Beacon Press, 1966), 418.

Synthesis

How do the ideas contained in the scholarly works listed above relate to the ideas presented by Diamond in the initial article?

Your Own Research

Choose a research topic of interest (does not have to be related to this course). On the back of this paper, track citations of academic journal articles about the topic through time. What do the authors of these articles say about the topic? How do their ideas correspond or differ?

Information Literacy as Research Methods IV


R M Media Ltd, CC BY-SA license

As I mentioned in my first post in this series, my interdisciplinary methods course includes a research proposal assignment consisting of:

  • An introduction containing a research question, hypothesis, rationale, and context.
  • A one-paragraph abstract.
  • Two-page discussion of the design of the proposed research project, the types of data that will be collected, how the data will be analyzed, and how this process will test the hypothesis and provide an answer to the research question.
  • A bibliography of references.

The proposal is intended to prepare students for an actual research project that they will design, conduct, and report on before graduating. I’ve created three smaller practice assignments that scaffold different aspects of the final proposal. The first uses research on Bolivia; here are the instructions:

  1. Read the rubric.
  2. Read either a chapter from Jim Schultz and Melissa Crane Draper, eds, Dignity and Defiance: Stories from Bolivia’s Challenge to Globalization, UC Press, 2008, or Isabel M. Scarborough, “Two Generations of Bolivian Female Vendors,” Ethnology 49, 2 (Spring 2010): 87-104.
  3. Read:
    Writing a Good Research Question
    What Makes a Good Research Question
    Constructing Hypotheses in Quantitative Research
    Forming a Good Hypothesis for Scientific Research
    Annotated Bibliography Guidelines

Write and submit the following as a single document:

  • A research question about the local effect of globalization in Bolivia.
  • A hypothesis derived from your research question.
  • A one-paragraph annotated bibliography entry for the item you read about Bolivia. Include the source’s complete bibliographic information.
Continue reading “Information Literacy as Research Methods IV”

Information Literacy as Research Methods III


R M Media Ltd, CC BY-SA license

An important component of both statistical and information literacy is the ability to recognize the difference between correlation and causation. Teaching this skill is made even more difficult by cognitive biases that lead to errors in probabilistic thinking.* So I decided to hit my students over the head with Chapter 4 from Charles Wheelan’s Naked Statistics and, from Tyler Vigen’s Spurious Correlations website, an image of the 99.26% correlation between the divorce rate in Maine and margarine consumption.

The assignment asked students to submit a written response to this question:

Why are these two variables so highly correlated? Does divorce cause margarine consumption or does margarine consumption cause divorce? Why?

All the students who completed the assignment answered the question correctly: neither one causes the other. In class, students identified several possible intervening variables, including:

  • People eat margarine and margarine-laced products as an emotional comfort food when relationships end.
  • Divorce leads to a greater number of households, with each household purchasing its own tub of margarine.

Students’ ideas led in turn to a discussion of how to appropriately measure these variables and construct new hypotheses.

*An excellent overview of this topic is Jack A. Hope and Ivan W. Kelly, “Common Difficulties with Probabilistic Reasoning,” The Mathematics Teacher 76, 8 (November 1983): 565-570.

Links to all posts in this series about information literacy:

Information Literacy as Research Methods II


R M Media Ltd, CC BY-SA license

Another post about the methods course that I’m now teaching. Chapter 3 of Naked Statistics is about deceptive description. So here is the accompanying assignment . . .

Many high school seniors are interested in attending Southwest America State University for college. Before 2015, applicants to this university had to submit high school transcripts that include average GPA scores, SAT scores, and an essay. In 2015, the application process changed; applicants had to submit high school transcripts with average GPA scores and two essays, while submission of SAT scores became optional. In 2019, the university claimed that the academic quality of its students had increased since 2011 given this pattern in the average SAT score of each year’s incoming class:

  • 2011 – 990
  • 2012 – 1130
  • 2013 – 1090
  • 2014 – 1150
  • 2015 – 1160
  • 2016 – 1185
  • 2017 – 1170
  • 2018 – 1190

Is this claim deceptive? Why?

Continue reading “Information Literacy as Research Methods II”