Independent Labs To Verify High-Profile Research Papers

ananyo writes "Scientific publishers are backing an initiative to encourage authors of high-profile research papers to get their results replicated by independent labs. Validation studies will earn authors a certificate and a second publication, and will save other researchers from basing their work on faulty results. The problem of irreproducible results has gained prominence in recent months. In March, a cancer researcher at Amgen pharmaceutical company reported that its scientists had repeated experiments in 53 'landmark' papers, but managed to confirm findings from only six of the studies. And last year, an internal survey at Bayer HealthCare found that inconsistencies between published findings and the company's own results caused delays or cancellations in about two-thirds of projects. Now, 'Reproducibility Initiative,' a commercial online portal is offering authors the chance of getting their results validated (albeit for a price). Once the validation studies are complete, the original authors will have the option of publishing the results in the open access journal PLoS ONE, linked to the original publication."

Dumb racket

[kencurry]

There is simply no way this would be effective for major research topics. They can't be experts across all fields, e.g., they would not have regulatory clearance to do medical studies. They would not have equipment or experience to do esoteric materials or particle physics etc. So yeah, call me extremely skeptical.

Re:Always been a problem

[Naffer]

It’s much worse than this. The burden of proof on people attempting to publish studies showing that work cannot be replicated is extremely high. Often many-fold more experiments and controls are required to show that it isn’t simply a failure on the part of the group attempting to repeat the experiment. Frequently these sorts of papers must offer an alternative hypothesis to explain both the original and new results as well. These sorts of studies are very difficult and time consuming, and can’t be given to junior graduate students who haven’t already proven themselves to be capable experimentalists. Thus to do something like this you need to assign one or more very capable senior students/postdoctoral workers, which costs money and time and takes away from original research.

Re:cool!

[donaggie03]

Negative results are sometimes just as interesting as positive ones. As you usually learn something.

You would think.

In the ideal world, that would be true, but in the real world, what you most often learn is that there are many different ways to screw up a delicate measurement in ways from which you learn little or nothing.

What are you talking about? Negative results doesn't mean someone screwed up a measurement. Negative results means the experiment ran correctly but the results went counter to the hypothesis. Negative results are the fruit of good science just as much as positive results are. Screwing up the measurements in an experiment is simply bad science, or not science at all.