Publishers Trial Paying Peer Reviewers - What Did They Find? (nature.com) 22
Two scientific journals that experimented with paying peer reviewers found the practice sped up the review process without compromising quality, according to findings published this month.
Critical Care Medicine offered $250 to half of 715 invited reviewers, with 53% accepting compared to 48% of unpaid reviewers. Paid reviews were completed one day faster on average. In a more dramatic result, Biology Open saw reviews completed in 4.6 business days when paying reviewers $284 per review, versus 38 days for unpaid reviews. "For the editors it has been extremely helpful because, prior to this, in some areas it was very difficult to secure reviewers," said Alejandra Clark, managing editor of Biology Open.
Critical Care Medicine offered $250 to half of 715 invited reviewers, with 53% accepting compared to 48% of unpaid reviewers. Paid reviews were completed one day faster on average. In a more dramatic result, Biology Open saw reviews completed in 4.6 business days when paying reviewers $284 per review, versus 38 days for unpaid reviews. "For the editors it has been extremely helpful because, prior to this, in some areas it was very difficult to secure reviewers," said Alejandra Clark, managing editor of Biology Open.
What's stopping them from... (Score:2)
Re: (Score:3)
Re: (Score:2)
That belies the real question though... Does paying the reviewers affect the quality of reviews you get?
Sure it speeds up the process. But it might cause your reviewers to give less care to their review and just make sure they get the Positive (or Negative) outcome issued quickly, since they want to be paid? Possibly reviewers' motives for offering to participate in the process will change completely: signing up for money - rather than signing up as a matter of altruism or professional leadership
Re: (Score:2)
That belies the real question though... Does paying the reviewers affect the quality of reviews you get?
According to the article (and first sentence of TFS) no. From TFS:
"Two scientific journals that experimented with paying peer reviewers found the practice sped up the review process without compromising quality, according to findings published this month."
From the article:
The results, published in the journal earlier this month1, found that paying for reviews moderately improved both the number of accepted invitations and the speed at which reviews were carried out. Some 53% of researchers accepted the invitation to review when offered payment, compared with 48% of those who received a standard, non-paid offer. On average, paid reviews came in one day earlier than unpaid ones. Journal editors assessed reviews from paid and unpaid reviewers and found no difference in quality.
Re: (Score:2)
I'm not sure how long a review takes, but $250 for an expert in the field isn't a whole of of money.
I suspect the results are more that by accepting payment the reviewers feel more of an obligation to get the work done quickly, since now it's transactional.
If I were worried about unintended consequences, it's not that reviewers would be motivated to to do poor reviews, but instead that there will be less long term motivation to do them at all.
It will move from a situation of one feeling duty bound to do the
Re: (Score:3)
It will move from a situation of one feeling duty bound to do them pro bono to a situation where it becomes underpaid work that people pass up
I suppose the publishers may end up having to pay more then. Still it seems only fair. Many publishers are generating massive profits off selling journal access apparently. So it does not really make sense that these journals would get the service of article reviews or submissions free of charge. Those who entities who contributed work to the journal deser
Scientific Societies (Score:3)
Many publishers are generating massive profits off selling journal access apparently.
It's more of a mixed model now. Some journals pay for access but others pay to publish and then make articles free to all. It's that last sort where paying reviewers is potentially going to be problematic because if you pay a reviewer and they say that the paper should not be published the publisher is now out of pocket since you only pay to publish when the paper is accepted. I suppose they could try to switch to a pay-to-submit model but that's unlikely to work since we only have limited grant funding an
Re: (Score:2)
People who review for the obligation of it (ie most everyone currently) will still review, they will just now get paid. People who did a solid job of reviewing have no reason to do a worse job. People who do a crappy job, about 1 in 3 in my experience, will have an incentive to do a better job. People who get asked, but rarely say yes
Re: (Score:2)
I suspect the results are more that by accepting payment the reviewers feel more of an obligation to get the work done quickly, since now it's transactional.
That's exactly it. It's also why many things charge you some token amount to do something, it creates a commitment to act. Conversely, a token nonzero charge creates friction to stop needless free-riding, where people will consume resources they don't need just because it's free.
Re: (Score:3)
I've not been paid for journal reviews. But it's not uncommon to get small honoraria for tenure reviews or external thesis reviews etc. Sometimes proposal reviewing. If I'm getting paid for something rather than doing it only as a service activity, it definitely goes up the "get it done promptly and get it done thoroughly" columns on my job lists. Altruism and professional service are still there. But I've also got a work ethic that kicks in if you're paying me. And, if I don't do it well, I won't get
Re: (Score:2)
Possibly reviewers' motives for offering to participate in the process will change completely: signing up for money - rather than signing up as a matter of altruism or professional leadership to advance your research field as a whole.
I suspect that over time the publishers would come to know which reviewers tend to just "mail it in" and which ones are conscientious. Also, a researcher gaining a bad rep as a peer reviewer might translate into more difficulty getting funding for their own research projects. So maybe the whole exercise is effectively self-policing.
I was paid to review papers (Score:3)
Who’d have thought? (Score:4, Insightful)
Show me the money (Score:2)
Re: Show me the money (Score:2)
Re: (Score:2)
I'm glad you were able to keep reviewing, its vital to science. I was at a national lab for many years, and over time the increasing pressure to account for all of our time, and to rush underfunded projects made it more and more difficult to do reviews. Doing it on "my own time" was great in concept, but "my own time" was already dedicated to my projects at the lab.
I think the idea of paying reviewers or having professional reviewers is a good one
Judge, jury, executioner (Score:2)
If you pay someone to do reviews then those reviews are worthless. By paying someone you create a contractual obligation, literally have them in your pocket. Therefore, their reviews fundamentally cannot be trusted, and you are in no position to make statements about quality.
Re: (Score:2)
Re: (Score:2)
If you pay someone to do reviews then those reviews are worthless. By paying someone you create a contractual obligation, literally have them in your pocket. Therefore, their reviews fundamentally cannot be trusted, and you are in no position to make statements about quality.
By that argument, any work produced for pay is worthless.
I've lost track of how many reviews I've done for free. I was recently paid a handsome sum to review a company for a potential investor. That review was the most careful and accurate one I've ever done, and the fastest, too. I was paid more than a month's salary for something that took a week's worth of effort, so until it was done, that review took precedence over everything else.
And now you're suggesting that I was somehow biased because I got pa
Lot of (non) reviewers here on slashdot (Score:2)
Gee, who would have thought that a journal could get peer reviews of articles by *paying* the reviewers, instead of only giving them credit for doing the review by mentioning their name... and they do it when they have time from doing their actual jobs, the ones that pay their bills?
I am a reviewer... (Score:2)
Thank yu for the belly laugh (Score:2)
The phrase "without compromising quality" in regards to scholarly papers sets a particularly low bar, Scraping under the bar in the social sciences may require a superfluid.
{^_-}