Outsourcing Manuscript Reviews

Smile SecretaryAcademic journal editors regularly ask me to anonymously review manuscripts that have been submitted for publication. Given that this work is unpaid and has a negligible effect on my prospects for promotion, it usually ends up far from the top of my priority list.

I recently realized that this task could be outsourced to undergraduate students as a writing assignment. Students in many of my courses already analyze journal articles, so why not make the process experiential? They can use the same guidelines and rubrics they use now, but with a more authentic role, audience, and format.

I can see a series of scaffolded components to this exercise:

  • Evaluate the manuscript from a stylistic perspective. Is the writing free of  mechanical errors? Is it concise and easy to follow?
  • Locate a piece of literature in the manuscript’s bibliography. Analyze it using the criteria that I link to above. Explain whether the author of the manuscript under review is referencing this literature appropriately.
  • Analyze the argument in the manuscript itself using the same criteria.
  • Make a recommendation about the manuscript–for example, accept, revise, or reject–and justify one’s recommendation. This could be performed in teams as a collaborative activity, with the members of each team deciding upon a joint recommendation and then presenting this recommendation to the class.
  • I compile the students’ work into a single assessment of the manuscript and submit it to the journal’s editorial staff.

The downside to this idea is that I never know when I will be asked to review a manuscript, so I can’t schedule it as an assignment before the course starts. But I’ve found that both I and my students often enjoy a change in routine.

The Mechanics of Text Analysis

DetectiveIn the past I’ve used some very specific exercises to train students how to analyze journal articles and other texts. Here is one of them:

Select a representative text and either distribute copies or require that students print it out. Prepare a few simple questions about the text that relate to the argument it contains.

Each student then . . .

  1. Circles or otherwise identifies words and phrases in the article that provide clues to the structure of the author’s argument. These “clue words” are:
    • Main, primary, only
    • Not, cannot, no, never, seldom, rarely
    • None, neither, nor
    • All, any, entire, most, each
    • Must, always, generally, often, will
    • But (especially if combined with “only” or “must”)
    • However, although, in contrast, contrary, instead, unless, despite
    • False, incorrect, contradict, fail
    • True, correct
    • Should, ought, shall
    • Cause, effect, reason, depends, because, imply
    • Assumes, assumption
    • Claim, argument, argue, contend
    • Proof, prove, evidence, empirical
    • In fact, thus, therefore, of course, thereby, similarly
    • Tend, tendency
    • Conclude, conclusion, result
  2. Writes answers to the assigned questions.
  3. Demonstrates how the clue words in the text indicated answers to the questions.
  4. Reformulates one of the question-answer pairs into a hypothesis and provides one or more pieces of evidence from the text to support the hypothesis.

Students then discuss or do formal presentations of their results in class.

I’ll provide an example of how this works in my next post.

Predatory Publishing or Why Professors Are Not Kanye West

illuminated manuscriptUniversity professors exist in a semi-public world, which means we usually suffer the negative consequences of pseudo-celebrity status but rarely get to enjoy any of its benefits. An exception that perhaps proves the rule is the terrible interview of Reza Aslan by a Fox News program host — his deft handling of the host’s stupidity will undoubtedly make his book sell many more copies than it otherwise would have (the interview is analyzed in some detail here).

A more mundane example is the periodic email solicitations for manuscript submissions that we receive from predatory open-access journal publishers. These publishers are essentially vanity presses; authors have to pay fees for the publication of their work. Anonymous peer review frequently doesn’t happen and editing is haphazard. The problem? It can be difficult to identify whether a particular journal is bogus or not.

Jeffrey Beall, a librarian and associate professor at the University of Colorado–Denver, has made this task easier by maintaining a list of predatory open-access journals and their publishers. It’s an extensive list and regularly updated.

Reading a Journal Article

Another skill-building exercise related to information literacy that I will be using in my upcoming thesis writing course is “How to Read a Journal Article.” Students will need to locate several peer-reviewed journal articles on their thesis topics and complete worksheets that contain the following questions:

  • What is the complete bibliographic citation for the article?
  • What is the hypothesis or research question, and where is it located in the article?
  • How does the author use other sources to engage in a dialogue with scholars who have written on the same or a related topic?
  • Do any of the other sources referred to in the article look like they will be useful for your research? If so, which ones?
  • What is the dependent variable in the argument being presented?
  • How has the author operationalized the dependent variable?
  • What are the independent variables in the argument being presented?
  • How has the author operationalized the independent variables?
  • What kinds of data were collected and analyzed? What methods were used?
  • What are the conclusions in relation to the hypothesis?
  • What message is the author trying to get across about her or his work in relations to that of others?

I developed this worksheet assignment after realizing that many students simply do not know how arguments in scholarly literature are structured. It also gets to Amanda’s point about students defaulting to a Google search after they’ve been told to find and use only high-quality information. While Google Scholar might turn up some peer-review journal articles that relate to whatever topics students are researching, they need to be able to understand the arguments in those articles, which means understanding how the arguments are organized. Most students won’t practice these skills unless they are required to do so.