Studies Suggest Massive Increase In Scientific Fraud 229
Titus Andronicus writes "Scientific fraud has always been with us. But as stated or suggested by some scientists, journal editors, and a few studies, the amount of scientific 'cheating' has far outpaced the expansion of science itself. According to some, the financial incentives to 'cut corners' have never been greater, resulting in record numbers of retractions from prestigious journals. From the article: 'For example, the journal Nature reported that published retractions had increased tenfold over the past decade, while the number of published papers had increased by just 44 percent.'"
Dystopic Reward System (Score:5, Interesting)
nope (Score:5, Interesting)
retractions is a bad measurement.
More and more data is open and available, so when 1 person committed fraud, it impacts many papers that come after it. The paper aren't committing fraud, there the victim of the first guy.
So I could commit frauds, and after 10 year it could impact 100 papers.
So retraction is a very poor way to determine this.
Re:nope (Score:4, Interesting)
Lot of pushback on the so-called "fraud epidemic" on the academic science blogs. The emerging concensus is that the campaign is part of a softening-up process for anti-climate science actions.
sPh
not necessarily fraud (Score:3, Interesting)
Perhaps the journals just don't do enough due-diligence anymore? The rush to publish in a world with 24-hour news and the internet...
Re:Plausible...a trend that is self amplifying (Score:5, Interesting)
Discover something marginal with real research, then use photoshop and obscure statistical methods to make it look like you have a real discovery. Make outlandish claims about the prospect of your discovery revolutionizing everything.
This is so true, particularly in small or relatively new fields, and particularly in the "softer" sciences. I took a course a few years back concerning a relatively small subfield of cognitive studies (an area which intersects with another obscure discipline), and the instructor assigned a half dozen papers to read each week, and class members would present a summary.
Basically, the instructor ended up using the primary literature of the field to show us how not to do good scientific research. About 90% of the time someone would point out a major "significant" correlation, the instructor would ask: but how many correlations did they try? Sometimes, there would be dozens and dozens of potential correlations checked in the article, and the one or two that actually worked would be touted as of "major significance."
Except when you try that many things, chances are something's going to correlate with something else. If you set your threshold at 95% confidence (common in soft science experiments where you don't have enough funding to get a lot of subjects), you'll get a correlation from random data about 1 out of 20 times. If you do dozens of correlations, you'll always find something.
But that wasn't the worst of it. The experiments were often poorly designed, because as an interdisciplinary subfield, most of the researchers didn't actually understand both areas that well. But the ambiguous manipulation of data then was generally used to justify the most absurd claims in the discussion section -- sweeping generalizations about how these findings might revolutionize our understanding of how the brain works or some other incredibly broad statement (usually false on its face, because the experiment was almost always so badly designed that it couldn't even say anything about the tiny subfield itself).
And then -- the worst part. Future articles would propagate the absurd sweeping conclusions from the discussions sections as if they were fact. A decade later, many of these claims had become "accepted knowledge" in the field.
I'd say about 75% of the articles we looked at -- and almost all of them were frequently cited and published in the central journals of the field -- were guilty of some sort of extreme bias in experiment design, data manipulation, or grossly exaggerated conclusions.
I know these things are far less frequent in the "hard" sciences, but the things I took away from this course were (1) how to read scientific articles carefully, and (2) there's a lot of crap being published out there that is barely "scientific."
Re:nope (Score:5, Interesting)
retractions is a bad measurement.
You do have to consider other explanations, e.g. maybe the internet makes it easier for scientists to get their hands on sufficient information to detect fraud, or maybe even journals have become more responsible about retracting articles one they're shown to be bad.
Orthogonal lines of evidence would indeed be useful for understanding what is going on.
information about information and leverage (Score:5, Interesting)
It might be interesting to think about the ways that increases in scientific fraud parallels the recent financial industry meltdown that resulted from the mortgage industry mess.
In the mortgage industry back in the old-old days, when you wanted to borrow money, you took your information (w2, bank account statements, etc,) down to the local bank which analyzed your finances and issued you a loan based on thier "gut" feeling on your credit worthiness. This was found to be a very non-scaleable, often discriminatory system, however the risk was localized therefore immediate feedback was available (banks that issued too many bad loans failed).
Then the industry evolved. Credit reporting agencies and credit scores were created to reduce discrimination, and automate decision processes and help quantify risk, and packaging was created to securitize loans which effectively aggregated and anonymized both borrowers and banks and attempted to present an abstract risk profile to folks investing in debt. The risk/return profile of this investment created a high demand for more securitized loans, creating a scarcity. What happens when demand exceeds supply? Either the price goes up (the yield of the debt investment goes down when the price goes up), or some risk takers will attempt to increase the supply by substituting marginal quality goods (loans that aren't well vetted). Then when others see their success with marginal quality goods, even the regular suppliers take the plunge and drop their quality to maintain their market share. Large coalitions enter the field and start to game the system. The lack of information available to the investors due to anonymization and aggregation amd increased leverage (firms started using derivatives and CDOs to invest in mortages) set us up for the financial industry fall. Then the cards all fell down.
Historically, scientific publishing when you wanted to get your paper published, you sent a pre-print to a journal and they attemped to referee the paper based on the "gut" feeling of their reviewers. This was fairly unscalable and often discriminatory system, but the risk of a poor quality paper was localized to the journal (basically journals that published too many bad papers would lose credibility).
We are in the midst of an evolution in scientific publishing. Now there are many mroe researchers and many more journals. Many journals don't have the staff to do a good job a vetting the papers, and the specialization, cost and expense of many research fields make peer-review "santity" checking across different research groups difficult. Ironically, as we have more information about science, we have less information about the quality of that information. Since published results attract scarce research dollars, the cost of doing good research that results in published papers go up (reducing the ROI on research dollars), or some risk takers will attempt to attract scarce research dollars with sub-quality work... and so on...
Let's hope that large coalitions don't enter to game the system, nor research grants are anonymized from author and institution as researchers move around and institutions do joint projects, nor that large research projects leverage questionable earlier research w/o information or verification or we may be building a similar house of cards with scientific research literature. Isn't scientific literature supposed to all be about leverage (standing on the shoulders of giants)? Aren't certain publication too-big-to-fail? Aren't large research coalitions monopolizing areas of grant money in certain fields and effectively owning the available peer-review resources? Maybe we've already set the table and just don't know it yet.
Some food for thought...
Re:Dystopic Reward System (Score:5, Interesting)
Cite, please. All references I can find are specifically to do with corporate sponsorship with journals and corporate sponsorship of reseach.
This must be why I mentioned corporations putting money into the central pot. If there's any ideology here, it's yours, since you have evidently taken a few things utterly out of the context in which they were placed and imposed your own idea of what I "must have" meant according to some fantastically inaccurate wall-chart of phrases-to-politics.
I said what I wanted to propose. In detail.
Plenty of private R&D in this framework. Private but decoupled.
Those who have marked me as "foe" on Slashdot would say so. Those, like you, who simply don't read what I write and prefer to imagine what you want me to have written - well, that used to make me mad. These days, it makes me wish I could emigrate to Mars on the basis that microbes and amoeba offer better conversation.
Re:nope (Score:3, Interesting)
Is that irony?
Re:Decent validation (Score:5, Interesting)
Gahh. Pet peeve of mine: I have come to the conclusion that people on the American side of the Atlantic don't understand what a model or a theory is for.
Science is not a collection of facts: it is a collection of theories supported by facts. When someone tries to publish something without model or explanation, it is your duty as a reviewer to reject the article with great prejudice. Because it it the theories that advance science.
This is because although the experiments will not get repeated (sure, they might if we scientists had job security and enough funding -- won't happen), the theories and models will get tested with new experiments. And this is really how science advances. Real science is the formulation of theories and not testing randomly new drugs: this also has marginal utility, but can never be as solid as, say, the theory of gravitation. Therefore, don't be surprised when people publish results that turn out to be a fluke, when they are pressed for time: this is because the reviewers accepted papers which were not framed within theories. Models and theories based on first principles are the only thing one can be reasonably certain of...
I suspect that the reason climate science and evolution are misunderstood in America more than elsewhere is that the education system here does not emphasize systematic knowledge and the power of models. Evolution cannot be "experimentally proven". but it can be used to formulate a great number of hypotheses which can then be verified experimentally.
Re:asshats vs stupes and crooks (Score:5, Interesting)
The rise in retractions seems to have almost become exponential over the past 6 years. That we knew and had predicted humans were causing climate change by increasing the warming of the climate was pretty well understood by the late 1980's. In fact many of the predictions have been born out since then. We aren't seeing retractions of papers from the 1980's, but the late 2000's.
Anyway, here is an excellent resource called The Discovery of Global Warming (See the timeline: http://www.aip.org/history/climate/timeline.htm [aip.org]). It's also a book. I hope that clarifies where I stand on the issue and why I called them asshats. Because only an asshat would try to make a connection. And I'm still waiting for that list of retracted papers about global warming.