iron sharpens iron

Alan Jacobs

There’s been a lot of talk in my Twitter feed and elsewhere online about this NYT story on open-access online peer review of literary scholarship. For me, this model marks an obviously significant improvement over the usual peer-review model. Pretty much everyone else whose views I’ve seen feels the same way, but their emphasis — and for the most part that of the story itself — is chiefly on the value of making decisions more open and transparent and therefore more accountable.Those are good things, but I’m more interested in how this richer, denser, and yet also faster process could make scholarship just plain better. By aggregating responses, this model allows the author to see how people in general are responding to his or her argument — and then to reply in turn, either by agreeing with the criticisms or by re-emphasizing the original argument. In either case, the author then has the opportunity to revise the article in ways that can greatly strengthen its argument. After all, even if you don’t agree with your critics, the clearer understanding you have of their objections the more effectively you can address those objections. And if you do agree with your critics, at least to a point, you can revise your argument accordingly.Also, the more eyes that read your work — assuming, as is fair to assume in this case, that they are pretty well-informed eyes — the more likely it is that someone will come up with an apposite quotation that helps or challenges your thesis, or will alert you to some research that you hadn’t known about that affects the argument. This won’t always be an obviously good thing for you — it wouldn’t be pleasant to discover data that undermines your whole thesis, or to find that someone else has already made your argument — but for serious scholars this will almost never happen, and less serious ones will benefit from the lesson in the need to cover all your research bases.More important, this model will be better for the cause of knowledge itself. Stronger arguments are stronger because they take the legitimate available evidence more fully into account. If we can get interested parties to do more to share the evidence they have, we will have more of what the Bible calls iron sharpening iron. And that’s good for the cause of scholarship.

Text Patterns

August 26, 2010

6 Comments

  2. I also managed to erase my first comment. Sorry. The gist of it was to say 1) good post and 2) open review is not necessarily faster. I did a story for the Chronicle last month about the SQ experiment. (See link above.) Some of the writers and editors involved told me that they had to find extra time to monitor and respond to comments and then find more time to incorporate those exchanges into their revisions. In their experience, the open process had advantages but was not necessary faster than closed review.

    Cheers,

    Jennifer Howard
    Senior Reporter
    CHE

  3. Thanks very much for the link, Jennifer! Might we make a distinction between (a) speed of response from readers and (b) speed with which those responses are dealt with by the writers?

  4. Alan,

    Seems reasonable. Readers might well respond more quickly in an open setting, though if they get involved in a conversation with the writer, that could end up drawing out the process. (The results could be worth it, though.) The added time I heard about was mostly in comment monitoring, manuscript revising, and overall editorial work flow.

    Jen

  5. I saw the NYT article as well and hope the kind of review starting in the Shakespeare Quarterly spreads elsewhere: I'm chiefly interested in 20th C American novels, and so far as I'm aware, no journals in the field have adopted this kind of thing yet.

    Last year I wrote a post on Careers—and careerism—in academia and criticism that advocated a system similar but not identical to the one the Shakespeare Quarterly is doing:

    The best ideas I can come up resolve issues in academic publishing: right now, it can take years to publish an essay in a peer-reviewed journal, which then locks it behind pay walls on the Internet. The length raises the obvious and uncomfortable question: if it takes three years to publish a paper, is the paper really that important? That this process takes forever is hardly new; Lucky Jim mocked it in the 1950s.

    My solution: have peer-reviewed journals “publish” online, and have publication be a link to the author’s paper on the author’s website. The journal’s editor could also copy that paper to their own site after anonymous peer review. That way, the information is freely available, especially to people in countries where most universities can’t afford journal subscriptions under the present model; the theoretical “size” of a journal could be limitless, although the practicalities of reading would probably still limit that size; there would still be a recognized body of work that makes up, say “Modern Fiction Studies;” and the journal could still issue a print edition every n months or years for those who prefer it.

    Note also the long and insightful comment Jason Fisher left in my post.

    In addition, as with so many things, Tyler Cowen has already been here: see, for example, this post from 2006. I believe he also wrote about an economics journal that now requires or offers submitters the ability to have a straight, up-or-down decision from peer reviewers, as well as commenters, as to whether the paper in its current form will be accepted.

    As a result, if you submit a paper on "Trade Policy and the Financial Crisis," or whatever, the peer reviewer has to give a binary, "publish" or "don't publish" decision, along with their critical remarks. So if the peer reviewer says "publish, and here's what I think," the original writer doesn't have to take those criticisms if he or she doesn't want to. This is an efficient method of avoiding the (long) cycles of tweaking, which sometimes end up matching the peer reviewer's prejudices / predilections without necessarily improving the paper.

Comments are closed.