Please create an account to participate in the Slashdot moderation system

 



Forgot your password?
typodupeerror
×
The Media Science

What Happens When Nobody Proofreads an Academic Paper 170

An anonymous reader writes: Drafts are drafts for a reason. Not only do they tend to contain unpolished writing and unfinished thoughts, they're often filled with little notes we leave ourselves to fill in later. Slate reports on a paper recently published in the journal Ethology that contained an unfortunate self-note that made it into the final, published article, despite layers upon layers of editing, peer review, and proofreading. In the middle of a sentence about shoaling preferences, the note asks, "should we cite the crappy Gabor paper here?" When notified of the mistake, the publisher quickly took it down and said they would "investigate" how the line wasn't caught. One of the authors said it wasn't intentional and apologized for the impolite error.
This discussion has been archived. No new comments can be posted.

What Happens When Nobody Proofreads an Academic Paper

Comments Filter:
  • by Anonymous Coward on Wednesday November 12, 2014 @12:24AM (#48366107)

    If anyone cares to read the passage with the insert here's a twitter pic [twitter.com] of it in use.

  • by gweihir ( 88907 ) on Wednesday November 12, 2014 @12:37AM (#48366149)

    There are two types of reviewers: The valuable ones that actually read a paper and try to understand it, and the worthless ones that look at title, abstract and who wrote it (usually easy to find out even in anonymous review). The first type catches these things, the second does not and quite often lest bad papers in and keeps good papers out. The second type is much more common.

    Or to put it short: Peer review is broken, as there is no quality control in most cases.

    • by timeOday ( 582209 ) on Wednesday November 12, 2014 @01:56AM (#48366355)
      "Peer review is broken" is such a broad statement, it's like claiming "clothes today aren't well-made." Peer-review is as good or bad as the individual journal.

      Granted, the average quality of "journals" has probably plummeted in recent decades as there are far more PhDs, papers, and journals than in the past. But by the same token, the quality of the top 100 journals (or any fixed number) has probably increased. I say that because the ease of communications now helps, and because of all the progress and recent focus on repeatability and avoiding statistical pitfalls. (A lot of reporting on this implies it is somehow a new problem, but there is no reason to think that).

      • by dcw3 ( 649211 )

        http://www.the-scientist.com/?... [the-scientist.com]
        Despite a lack of evidence that peer review works, most scientists (by nature a skeptical lot) appear to believe in peer review. It's something that's held "absolutely sacred" in a field where people rarely accept anything with "blind faith," says Richard Smith, former editor of the BMJ and now CEO of UnitedHealth Europe and board member of PLoS. "It's very unscientific, really."

        http://www.ecnmag.com/blogs/20... [ecnmag.com]
        As soon as we receive a paper, we publish it," after a cursory q

        • This sort of thing is why reproducibility is the backbone of science, not peer review. Peer review is like some kind of low-pass filter at best. Reproducing the study is where the verification actually occurs.
      • by AthanasiusKircher ( 1333179 ) on Wednesday November 12, 2014 @08:36AM (#48367537)

        Peer-review is as good or bad as the individual journal.

        While this is probably true, I would go further and say that this particular issue (from TFS) has relatively little to do with peer-review.

        Most peer reviewers are not paid. When I've written reviews for articles, I'm assuming that I'm volunteering my time as an expert on the subject matter. So my primary purpose is to critique the argument, look at the design, see whether the conclusions are justified, etc.

        Things like fixing commas, rewording sentences, and proofreading for some sort of stupid error where the authors forgot to delete something -- that's not my primary purpose. If I have time and I see pervasive problems of style, I might say something in the review. If those stylistic things end up confusing the argument or making the thing hard to read, I might say something.

        But if I were reading this article, and there were a half-dozen comments or questions I had about methodology or argument on this page, would I bother saying, "Oh yeah, and don't forget to fix the stupid missed citation!" Maybe. But it wouldn't be my highest priority.

        I don't know what happens at this journal, but most high-quality journals have at least some copyediting done before publication. If the author didn't catch this error during revision, it should have been caught by the copyeditor. But the peer reviewer? Are we going to ask for expert volunteers in some academic discipline to fix commas next?

        Granted, the average quality of "journals" has probably plummeted in recent decades as there are far more PhDs, papers, and journals than in the past. But by the same token, the quality of the top 100 journals (or any fixed number) has probably increased.

        It depends on what you mean by "quality." If, by "quality," you mean the level and rigor of articles and research in major journals, maybe you have a point.

        But, if by "quality" of a publication, you mean the copyediting -- that has absolutely DECREASED in recent years. I can't tell you how many sets of proofs I've seen with all sorts of idiotic formatting errors, places where an editor tried to fix prose or move something in the layout and caused an absolute disaster to happen, etc. Heck, this isn't just articles -- I've seen recent books from major university presses that seem to have the same level of copyediting a cheap romance novel would have received 40 years ago. And heaven forbid that you have some complex set of figures or images that need to be laid out in a specific way -- the designers seem to go out of the way to screw things up by resizing or moving things about, even if you send them images designed to fit the page layout precisely.

        I haven't read the article referenced in TFA. But this all sounds like a proofreading and a copyediting problem. Peer reviewers? Yeah, I suppose they should have caught it if that citation would actually make a difference in the argument. Otherwise, I'm not sure what this has to do with peer review quality AT ALL.

    • Re: (Score:3, Informative)

      by Anonymous Coward

      There are two types of reviewers: The valuable ones that actually read a paper and try to understand it,...

      I actually had a look at the paper in question.

      I've probably got some details wrong but it was mainly an experimental study where they look at two closely related populations of fish - one in a toxic sulfur hot string and the other not. They find that females in the sulfur hot springs have a preference for (male) fish with spots while the other females don't have a preference.

      But then they launch into a pages of random speculation about what this observation might, or might not, mean (i.e. they had no idea)

      • by phantomfive ( 622387 ) on Wednesday November 12, 2014 @10:00AM (#48368117) Journal

        So what do you do if you're reviewing the paper? Do you try to take the random speculation seriously and spend days trying to make some sense of it and give it a meaningful review?

        If you're going to do peer review, then yes, you should try to make sense of it. How is that even a question?
        Secondly, if the paper has serious problems with most of the content, of course you should reject it and explain why. Then the author can fix it and resubmit. It's not like rejection is some kind of permanent, damaging thing.

    • by rmstar ( 114746 )

      There are two types of reviewers: The valuable ones that actually read a paper and try to understand it, and the worthless ones that look at title, abstract and who wrote it (usually easy to find out even in anonymous review).

      And then there's The Third Reviewer [youtube.com].

  • by NotSanguine ( 1917456 ) on Wednesday November 12, 2014 @12:38AM (#48366159) Journal

    and the articles they reference are wildly inaccurate. From TFA:

    Not sure how this made it through proofreading, peer review, and copyediting. Via http://t.co/sWaswaM2X4 [t.co] #addedvalue pic.twitter.com/8krLlvthAr — Dave Harris (@davidjayharris) November 10, 2014

    [Emphasis Added]

    So the paper was proofread, peer-reviewed and copyedited. Sigh.

    People make mistakes. Life is like that sometimes. The authors of the paper will face consequences for this. Hopefully, they'll learn from them.

    Nothing to see here, unless you wrote the paper or are the person referenced.. The post and the linked TFA are a waste of time.

    • by Anonymous Coward on Wednesday November 12, 2014 @01:17AM (#48366263)

      A more useful article than TFA is over at retractionwatch [retractionwatch.com].

    • Here's the paper: http://expirebox.com/download/... [expirebox.com]
    • To their credit it does mention 'despite layers upon layers of editing, peer review, and proofreading' right in the summery, so I assume it must in TFA (which I, as per tradition, have no intention of actually reading), but as we all know /. editors sure as shit don't have any room to criticize other people's mistakes in editing.

    • The point is, if that's the quality of proofreading, peer review, and copyediting, then none of those were very effective.

      That's kind of obvious. The fact that you're trying to defend it shows you might have some cognitive biases to fix.
      • The point is, if that's the quality of proofreading, peer review, and copyediting, then none of those were very effective. That's kind of obvious. The fact that you're trying to defend it shows you might have some cognitive biases to fix.

        Please tell me if anything I said was untrue. And I'm not exactly sure what cognitive bias you're ascribing to me. Please explain.

        • Untrue? Let me quote your sig, it's a good one: "No, no, you're not thinking; you're just being logical. --Niels Bohr"

          This is what happened to you man:

          --------point------------>

          O
          /|\ <--- you
          |
          / \

          The point is, the proofreading was so bad, it wasn't worth to be called proofreading. It's hilarious, and it shows what a lousy job everyone involved did. Furthermore, you say "there is nothing to see here," but you are wrong, there most certainly is something to see here. You can't se
          • Untrue? Let me quote your sig, it's a good one: "No, no, you're not thinking; you're just being logical. --Niels Bohr" This is what happened to you man: --------point------------> O /|\ <--- you | / \ The point is, the proofreading was so bad, it wasn't worth to be called proofreading. It's hilarious, and it shows what a lousy job everyone involved did. Furthermore, you say "there is nothing to see here," but you are wrong, there most certainly is something to see here. You can't see it because of your cognitive biases. So fix that. Relax and accept that sometimes scientific processes go hilariously wrong.

            Firstly. Proofreading isn't a scientific process. Secondly, as I and several others pointed out (as we *actually* read TFA) that the "comment" was added bery late in the process, after initial proofing, after peer review and, apparently, shortly before the article went to press.

            As for my "cognitive bias," I said:

            The authors of the paper will face consequences for this. Hopefully, they'll learn from them.

            What is more, a minor error in editing (albeit an embarrassing one) isn't a "failure of the scientific

    • But do people really expect Slashdot articles to be proofread? For that we'd need to employ editors to replace the scripts currently posting stories.

      • But do people really expect Slashdot articles to be proofread? For that we'd need to employ editors to replace the scripts currently posting stories.

        My point was more about the headline. Which was copied verbatim from the awful TFA. The titles of both completely misrepresent the situation. As for /.'s editors, I make not comment. Their work speaks for itself.

    • by geogob ( 569250 )

      Going through a proofread process doesn't necessarily mean it was actually proofread.

  • This could have been avoided if the authors had used LaTeX for writing their paper. It allows for comments in the text that don't become part of the formatted output.

    % Should we cite the crappy Gabor paper here?

    There are also various LaTeX packages for writing comments, adding annotations and tracking changes that could be useful when peer-reviewing a paper.

    • Re: (Score:2, Interesting)

      by Anonymous Coward

      Ten years ago I would've agreed with you, but word processors have caught up and they all have very good commenting systems now.

      This was technically weak authors (not using comments in the first place), a poor internal review process, and a terrible peer review process. Thinking about it more, it's not that surprising. Inventing some percentages, say 10% of authors don't use comments and 5% of peer reviews would miss it (1 internal, 2 external). You would have a roughly 1 in 10,000 chance of this slippin

      • Statistics on how well read the average article is are hard to come by.

        That is why I simply say "insert statistical method here" and continue on.

        Here is a fine example of usage. [link [nih.gov]]

    • by vux984 ( 928602 ) on Wednesday November 12, 2014 @04:05AM (#48366715)

      This could have been avoided if the authors had used LaTeX for writing their paper.

      Hardly. This would have been avoided if the authors had written:

      (************ SHOULD WE CITE THE CRAPPY XYZ PAPER HERE *************)

      And then it wouldn't have gotten missed even in Notepad. In anything more advanced than notepad I'd also format it bold, and in red too.

      Arguing for the commenting features of latex presume they would actually know about the feature, AND choose to use it. For all I know they did use latex, but didn't bother to mark it as a comment. (I mean, they probably used Word, and they didn't mark it as a comment with that either, which they could have done -- so why would switching to latex make them use the feature??)

      But using the commenting feature would also potentially be a detriment. They may well WANT their own review, and internal reviewers to see this stuff, so that they can render an opinion. Having it simply omitted from the PDF or printout they are looking at means they don't see it, and can't mark a note ... "Hey -- you should cite that paper" or "don't bother with that"... in their review notes.

      • Arguing for the commenting features of latex presume they would actually know about the feature, AND choose to use it.

        It would be surprising if anyone who has spent any time formatting with LateX didn't know about the commenting feature.

      • by godrik ( 1287354 )

        exactly, if my student writes the paper, I might not read the latex file myself. What I typically do is that all things that are not meant to be part of the article is either a \note{} or a \todo{} which resolve to write in bold, red, and change background color to yellow (or green). That way, it is impossible for me to miss it before it is sent to the reviewers.

    • by geogob ( 569250 )

      Inline comments are awkward in latex. It's one of the biggest flaw of tex IMO. A commenting method that comments out everything to the next line break will inherently break the text flow in the source file. This make production difficult and authors often fall back to non-commented notes in-line -- with the consequences seen here.

      This also the reason I will never to text iterations with co-authors (especially in the later production phases) on the tex files, but always and only with pdf files.

    • by tlhIngan ( 30335 )

      This could have been avoided if the authors had used LaTeX for writing their paper. It allows for comments in the text that don't become part of the formatted output.

      % Should we cite the crappy Gabor paper here?

      There are also various LaTeX packages for writing comments, adding annotations and tracking changes that could be useful when peer-reviewing a paper.

      Even if they use Microsoft Word, they could use the "comment" feature that puts up a comment in the margins with a arrow and highlight. And which can be

    • by fermion ( 181285 )
      Absolutely agree. I remember when I was helping put papers together how painful Word was to get to work, and how nice it was when I finally learned LaTex. I recently had to put a piece of research together, and now of course everyone uses MS Word and one has to use it. For collaboration.

      That said the error might not have been prevented with LaTex. If it was a conversion error from different versions of word, in which a comment was exposed, that might have been prevented. If it was a human error, a comme

  • by Moof123 ( 1292134 ) on Wednesday November 12, 2014 @01:03AM (#48366231)

    Happens all the time. We had a report that had one project member with a title of SRP Lap Dog. It was put there in jest about 6 months earlier, along with some swear words that actually did get caught int the final edits, but not the title. Professionals are human too, and stuff happens.

    • by gsslay ( 807818 )

      Yes these things happen all the time. Which is why anyone with a shred of professionalism and experience doesn't add crap like this to a paper that will be read by external people at some point. If you have a habit of doing this, it will catch you out eventually.

  • The more you specialise, the less you are understood. It is not without reason nature is biased against species who can only survive in specific environments.

  • by pesho ( 843750 ) on Wednesday November 12, 2014 @02:20AM (#48366405)
    Obviously Gabor did not review the manuscript.
  • by Mantrid42 ( 972953 ) on Wednesday November 12, 2014 @03:44AM (#48366631)
    Shit happens.
    • by hey! ( 33014 )

      On the plus side, think what this will do for the journal's impact factor!

  • Just kidding, but people will make a big deal out of this because they can twist it to whatever "everything is falling apart" worldview they hold. The statement got added post peer review accidentally and people that had read the paper a million times missed it. As they use to say at the height of the Roman empire, "Pol fit."

    Frankly, I bet that crappy Gabor paper gets a lot more interest now than it would have garnered with an appropriate reference.

  • Of course this happens. The world is going to direction where people are rushed through some watered-down education (where they get no chance to fail a couple of times first or think things through). They learn to solve problems quickly with some high-level tools. The attention to detail and mastering things down to core essentials is slipping. The guy with the coolest TED talk wins. Others are boring nerds wasting their time with abstract concepts. And hey, quality assurance, what's that? We need to ship t
  • According to the article, the comment was added in revisions after peer review. It should have been caught before publication, but it's not the reviewers' fault.

    One other thing: to catch this reliably, you need to have someone read through it who knows that it's the final version. Otherwise they may well assume that it's still an active question, waiting for views. And of course, you should always word your notes more politely!

  • by drinkypoo ( 153816 ) <drink@hyperlogos.org> on Wednesday November 12, 2014 @06:05AM (#48367077) Homepage Journal

    Is the Gabor paper crappy?

  • Clearly, they should have claimed this was merely an attempt at something new, a device to engage the reader. What do you think reader? Should we have cited the crappy Gabor paper here? Its a discussion point; not an error!

  • by rnturn ( 11092 ) on Wednesday November 12, 2014 @09:23AM (#48367823)
    In a previous life, I had put a humorous phrase -- a reference to ``Real Programming'' -- in a technical report that was support to be submitted to a government agency that we were working for under contract. None of the others who reviewed the report noticed it -- maybe they were too busy that day and didn't pay as much attention as they normally did. They'd typically spot any questionable grammar that I might have used and I was sure someone would catch it and send it back to me to change. Nobody did, though, and I was lucky enough to get it back and delete the phrase before the report went went out the door. Learned a valuable lessen about trusting proofreaders that day: Don't.
  • by Moof123 ( 1292134 ) on Wednesday November 12, 2014 @09:31AM (#48367889)

    In manufacturing there is a tendency to add extra inspectors after each slip up (well in defense related manufacturing anyway, from what I saw). Eventually every inspector comes to believe that what they are supposed to inspect ihas either already been inspected numerous times, or would get inspected by someone else later. Soon there are so many inspections that nobody actually does a real inspection, as they all believe their inspection is redundant. With multiple levels of proof reading I imagine a similar failure mode is going on here. Just one inspector should be tasked with QA signoff, not a crowd of them.

  • the NYT, MSNBC, and the Daily Show.

  • This story is exactly why I've encapsulated my self-notes and comments in c-code-style markings: /*this is a note to myself */

    It's trivial to skim a document for the existence of such markups. Yeah, it takes a little-self-discipline while writing, but it sure pays off.

  • Wait, slashdot is posting a story about lack of "editing, peer review, and proofreading"? That, good sirs, is irony.

  • Remind me again what china gave up here? no more increases of emissions by 2030?... yippy.

  • Sounds like a character from the newspaper comic strip, "Barney Google and Snuffy Smith".

An authority is a person who can tell you more about something than you really care to know.

Working...