Are Research Papers Less Accurate and Truthful Than in the Past? (economist.com) 119
An anonymous reader shares an Economist report: An essential of science is that experiments should yield similar results if repeated. In recent years, however, some people have raised concerns that too many irreproducible results are being published. This phenomenon, it is suggested, may be a result of more studies having poor methodology, of more actual misconduct, or of both. Or it may not exist at all, as Daniele Fanelli of the London School of Economics suggests in this week's Proceedings of the National Academy of Sciences. First, although the number of erroneous papers retracted by journals has increased, so has the number of journals carrying retractions. Allowing for this, the number of retractions per journal has not gone up. Second, scientific-misconduct investigations by the Office of Research Integrity (ORI) in America are no more frequent than 20 years ago, nor are they more likely to find wrongdoing.
My research says.. (Score:5, Funny)
Re: (Score:2)
So the quality and reliability of research papers over the past decades, has not improved one little bit, yeah I guess, so much for the efficiency of computers, research papers just as bad as always, ahh the presstitutes. Why does this story feel like it is just filling empty space. C'mon research people how about trying to do better ;D.
Re: (Score:3, Insightful)
Similarly, for information on other topics, not only is there a huge quantity available but with a very uneven level of quality. You don't want to rely on the ne
Re: (Score:1, Informative)
You fail to name a single example and just talk out of your ass, that's why you were downmodded.
Re: (Score:2)
Most of us have been exposed to clearly retarded studies and know what he's talking about.
Re: (Score:1)
That's a meaningless statement, since the total number of publications has also exploded. Of course, there are more bogus studies in total numbers. The number of total publications world-wide has increased manyfold. The question of the headline only makes sense as a question about the relative percentage of obviously bad studies. I see no indication of that anywhere, and yes, I am a scientist. In fact, the overall quality and demands have increased a lot in my discipline.
On a side note, at least in my field
Re: (Score:3)
The flaw isn’t in science, but in the education fields motto, publish or perish.
To be funded scientists are expected to show results. Most of the time these results are no conclusive evidence. Which doesn’t get those grants in the door. And will not get your name known.
Scientists are people too, so like all of us when under pressure, will sacrifice their ideals for a paycheck. Emblish a paper, write on a outlier action that was interesting.
Being the people who pay for these grants rarely rea
Re: (Score:2)
That suggestion is totally crazy. Almost all foundational research in all sciences is and has always been state-funded, and even in the rare places where research is done in companies (e.g. pharmaceutical industry) this research would be impossible without the more foundational state-funded research & education.
If you remove that funding in a highly industrialized country, the quality and quantity of scientific research in that country would fall down to ridiculous levels, those of 3rd world countries o
Re: (Score:3)
Re: (Score:2)
Yeah nothing but private funding. That'll work even better!!!!
Seriously there is no way you belong on this site.
Re: (Score:2, Interesting)
Despite your feelings to the contrary, the answer is no.
Re: (Score:2)
And 100% of random guys and Anonymous Cowards on Slashdot disagree.
I think I know who I'm betting on.
Re: (Score:2)
if you took a look at the whole, the answer might be different.
it's just that they don't count some journals as science anymore - or the papers themselves don't have lies but they don't have truth either.
seriously though, the amount of papers in the cs field.. is that most of them are just crap published for the sake of publishing, starting even from basic things like making a paper on how some api works with 3 pages of filler. not even kidding.
Re: (Score:2)
That's always been the case in a lot of fields. People have to publish to get PhDs and keep jobs, so they publish a lot of filler.
That's not the same thing as publishing false science or retracting research. Something can be uninteresting research without being fal
Re: (Score:2)
Haven't we seen articles about this before? Tons of papers with results that can't be reproduced. But hey nobody reads these papers unless they need to write one of their own anyhow.
Re:Simple answer (Score:4, Funny)
Re: (Score:2)
Well, if nobody reads those papers then nobody can have shown that their results are not reproducible either. If you want to be a trendy science critic, you should at least pick a consistent position.
Re: (Score:2)
Yes.
I suppose this is the time for the obligatory mention of Betteridge's Law of Headlines.
No, the transparency is just better today. (Score:4, Interesting)
Lots of old reports, research, etc are known to have been grossly biased, outright doctored, or using questionable results towards questionable confirmations or refutations of the hypothesis.
As a simple example, go read up on the Coca plant, and UN level attempts to eradicate wild plants from its ENTIRE HABITAT RANGE and the ecological damage that has been done as a result.
Hint: While refined cocaine in recreational quantities is dangerous and addictive, individual leaves contain 5 percent of coca extracts per leaf mass, and have non-recreational uses especially in their natural range, as well as modern commercial success as teas and other herbal supplements. Furthermore use as a topical anesthetic requires 1/100th of the dosage used by people to get high. The danger is that unlike marijuana, and more like opiates, cocaine can saturate receptors to the point of causing cardiac arrest or other serious medical issues if misused.
My point being: Any science currently considered controversial will have politically motivated research aimed to either prove or refute the stance that is most politically favorable to the people in power, whether those people are religious, governmental, or commercial in nature. Always has been, likely always will be.
So.... (Score:1)
Re: (Score:2)
Everything was always better (Score:5, Insightful)
Why are are the headlines now questions? (Score:5, Insightful)
Almost every slashdot article in the last couple days has been a question.
Fucking knock it off.
Re: (Score:2)
It's just the new "may," as in: "Research papers may be less accurate and truthful than in the past."
This ain't news, nerds. (Score:5, Informative)
In 2016, The Journal Nature published a story by Monya Baker, where more than 70% of 1,576 researchers tried and failed to reproduce other scientist's experiments [2].
Even worse, many did claim to have reproduced the Pons and Fleischmann Cold Fusion experiment shortly after their press release in 1989 [3]. So many in fact, Nathan Lewis of Cal Tech quipped "Cold fusion has been verified by no university without a good football team" [4].
The problem has been around for decades. I'm thinking there might be reasons, like patents, contracts, grants, money, and prestige. It could be that science, or at least a bunch of scientists, ain't what they're cracked up to be. Or maybe football appendages and their cozens just aren't that important.
[1]https://en.wikipedia.org/wiki/Replication_crisis [wikipedia.org]
[2]https://www.nature.com/news/1-500-scientists-lift-the-lid-on-reproducibility-1.19970 [nature.com]
[3]https://en.wikipedia.org/wiki/Fleischmann%E2%80%93Pons_experiment [wikipedia.org]
https://bwi.forums.rivals.com/threads/scientists-fleischmann-and-pons-cold-fusion-or-cold-illusion-25-years-later.10260/ [rivals.com]
Cart before horse? (Score:4, Insightful)
Re: (Score:2)
I read the doctorate thesis from a medical doctor. She is investigating an illness that is hard to measure. You can take a sample from a patient and look at it and say: yes he/she has the disease. If you guess wrong as to where to take that sample, you see nothing: normal tissue. But you still don't know if the patient has the disease. So there is a need for some "blood test" that says: "yes you have it" or "no you don't".
So she took a group of 200 patients and 200 healthy people, measured everything that c
Re: (Score:2)
Or more correctly not being able to [reproduce experimental results]. Kind of the smoking gun type of evidence that says junk science is embedded as core to modern scienceÃ(TM)s purpose.
"Modern science"'s purpose these days is to push political agendas that attract funding from politically-polarized groups, the largest being the US government. Anyone attempting to publish real scientific research that casts doubt upon the politically-correct conclusions embraced by the political/scientific 'establishment' are ostracized and personally destroyed in the media and shunned by academic and scientific peers.
The problem with "modern science" is that there's been too much politics injected into a
Grants May Have Agendas (Score:5, Interesting)
Re: (Score:3)
I doubt the political or ideological agenda of current scientists is anywhere near as obvious, over-arching, and narrowly-defined as that of the Royal Society or Oxford of a century ago.
Re: (Score:3, Informative)
You aren't a scientist, but those of us that are know 80% of our job is writing grant proposals and networking for opportunities to fund our research. If you do not provide results beneficial to a grant provider, there will not be a second grant.
More about product sales ... (Score:2)
Politics doesn't need stats to push an agenda.
Re:Grants May Have Agendas (Score:5, Insightful)
No, that's always been the case. Lead in gasoline, smoking, acid rain, pesticides, etc., they all had studies to prove that they caused no harm at all. In fact, there are people who basically "manage' this sort of publication, a playbook if you will. As in, the best way to counter something bad in your industry is to manufacture controversy, and the way you do that is by getting studies done in your favor. I cite those cases above because those were "managed" by the same group of people who basically do just that - manufacture controversy. (And yes, that same group is behind climate change opposition as well).
Some history of that can be found in Merchants of Doubt [wikipedia.org]. They came up with the playbook on how to manufacture controversy and thus push regulations out.
And let's not forget other cases like vaccines causing autism and plenty of food related papers all paid for by various aspects of industry.
What's happened is recently the Internet has made it much easier to find information, so hunting down who the sponsors of a paper out is much easier even when they hide through 10 layers of corporate shields, and people are able to seek out the original document much more easily and thus analyze the results. The fact that everyone is moving towards open data as well makes it much easier to spot frauds.
Article Self-contradicts (Score:5, Interesting)
It also seems very narrowly focussed on deliberate attempts to mislead since it concentrates on discipline procedures and investigations. However, the reproducibility problem is generally acknowledged to be mainly due to poor scientific practice, e.g. claiming that correlation implies causation or not understanding statistics, and not due to deliberate malfeasance.
The data also show that there does appear to be a slight increase in the number of corrections per journal - although this is only small and the plot fails to provide error bars so it is impossible to know whether or not this is statistically meaningful. It also cryptically mentions that this is for journals which issue corrections suggesting that there are journals which do not issue them.
The number of invitations I get to be an editor on new journals by predatory publishers has markedly increased over the past few years so, at least based on my experience, that there appear to be many more predatory journals than there used to be and I would be amazed if any cared enough to publish errata given that there is no money in it for them so, if the fraction of junk publications has increased this might entirely hide a large source of irreproducible papers from this study.
Re: (Score:2)
You make an unfounded assumption, namely that "each country's individual accuracy rate (as a fraction) is constant" - that's not supported by the article.
It is for the US, China and India all of which at various points the article states have constant retraction or investigation rates. Admittedly the statements are made without any data to support them but they are made. The data they do give suggests that the rate may actually show a small overall rise as well if you judge the statistical uncertainty by eye from the plot - providing error bars would have been better.
Economist joins the crowd (Score:1)
Papers are less truthful? This must have been written by climate change deniers!
Office of Research Integrity has more resources? (Score:3)
scientific-misconduct investigations by the Office of Research Integrity (ORI) in America are no more frequent than 20 years ago
Does the Office of Research Integrity have access to more resources than they did 20 years ago? If the number of misconduct investigations is limited by their capacity to investigate then "no more frequent" is meaningless. If ORI can expand their capacity as needed, as more questionable research is reported, then "no more frequent" may be meaningful.
Aren't they *more* truthful than in the past? (Score:1)
Hindsight is 20/20 (Score:1)
Research Papers in present are probably at same or maybe even at better level of accuracy and veracity compared to past ones, except that what remains in scientific use today has gone through years of debate and filtering and has established its merit. What was bad in past is mostly actively forgotten. So in a way, we see matured, weeded-out picture of the past.
We seem to come to expect faster results and instant quality, but that is probably not going to happen. Patience is needed. Scientific progress has
It's accessibility (Score:1)
Every shitty little study gets tweeted and blogged and newsed around, Elsevier is dead, everybody can download the original paper from sci-hub instead of relying on popsci articles by nonscientists. So less filtering, more critical exposure. Which is a good thing, but it also makes it look like the overall quality has decreased.
This applies mostly to medicine and social science (Score:3, Informative)
This report may eventually get retracted itself (Score:1)
Headline from the future:
After reviewing the findings that research papers are less accurate and truthful in the past, and suggesting that this may be the result of poor methodology or willful misconduct or both or neither, we respectfully withdraw and retract our findings.
Truthful?? Dunno. Accurate...WELL... (Score:2)
Let's just say that the confidence levels used today would have been laughed at 50 years ago.
Re: (Score:2)
Yep. Except for particle physics, which uses 6 sigma. Most fields use 2 sigma.
As a researcher, I would say yes. (Score:2)
I am constantly aggravated by the amount of articles published without enough information to successfully reproduce the experiment. Even more often I find lies by omission, where a very important aspect of the experiments' protocol is not mentioned.
I just found that an experimental protocol used by a dozen papers, has a glaring problem that absolutely should have been mentioned in all of them, but at least the ones that first introduced it. I am thinking to publish a short article discussing just this glari
And, the answer is..... (Score:2)
I would say no (Score:2)
STEM is a saturated field with people jockeying for position. If you publish anything there are people out to take you down. The competition can be fierce. So I would say better overall, but with the occasional poor paper getting accepted.
misconduct is the wrong way to look at this (Score:4, Informative)
This isn't a matter of misconduct, that's the wrong way to look at the current failure of science to... do science. (I am a scientist.)
Other metrics are more useful. My favorite is "research efficiency." This is a decidedly commercial metric, it's the amount of revenue or economic activity (in dollars) generated by $1 of scientific research investment. It's been going down since about 1980. Surprisingly, research areas pitched as "basic research" (i.e. math, astronomy) tend to do well with this metric. It's the research that's sold to the public as industrially focused (i.e. my field, nanotechnology) that tends to do the worst.
Another useful metric is the % of science PhDs who stay in science for at least 10 years after getting their degree. This measures how effective we are at training our scientific workforce. That's down significantly over the last 30 years as well. What we teach people now is not what they need to succeed in science after training (which is getting longer and longer).
The metric most scientists are looking for is reproducibility, or the percentage of papers which can be repeated by simply following the instructions in the paper. Papers have grown in length and complexity in the last 40 years. It's pretty hard to argue that reproducibility has actually gone down because older papers simply don't include details we now expect. Of course, this is very hard to measure in any case. That's the thesis of TFA. It doesn't change the very real feeling (and data) that science is somehow not delivering on our investment.
Misconduct is... you're going to have some when there are people involved. You're also going to have mistakes and papers which are disproven very quickly. I have a personal pet peeve for papers that promise extraordinarily cheap hardware by assuming labor is free, manufacturing can be done at large scale without investment in tooling, and working capital is free. Things like this are not actually misconduct, no matter how misleading they are.
IT is about grant money - not about finding truth (Score:2)
The signal to noise in the research field varies a bit - but over all it is so bad that most medical and nutritional work is best ignored until replicated.
The norm is there are a few teams doing good work - but most of the papers are grant money prostitution. This muddies the waters for people trying to learn about some topic - reinforcing the accepted narrative in order to keep the grant gravy-train flowing.
A couple of examples:
There is a long list of rodent papers with titles such as "High fat diet caus
How science works (Score:2)
Every scientific paper should be viewed with skepticism. I don't mean that in a bad way. Skepticism is part of how science works. Every paper you read, assume it may have errors and the conclusions may be wrong. Hopefully the reviewers caught the worst problems, but don't count on it. It takes a lot of other people reading a paper to spot all the problems. Experiments also need to be reproduced. It takes years for the community to reach a consensus about whether a paper's conclusions were right or no
I'm not sure that means what you think it means... (Score:2)
If the number of retractions has increased due to an increase in the amount of researched being performed and published scientific misconduct investigations would be expected to have increased proportionally all else being equal. Not having done so doesn't indicate there is less misconduct, it indicates less of the misconduct
TL;DR (Score:2)
First a scientific study found a surge in the number of non-reproducible scientific studies. Then a second study tried to reproduce the results of the first and failed, thereby simultaneously proving and disproving the original study's point.
Re: (Score:2)
You can, of course cite this research, right? You wouldn't just be creating strawmen because your dishonest, stupid and lazy.
Re: (Score:2)
Just because that's how you would do it doesn't mean that scientists in general are doing it that way. Scientists in the harder sciences like physics and chemistry and yes, climate science are pretty much required to conform with reality or other scientists will take them to task for their misinformation.
Re: (Score:2)
Sure thing AC. And yet surface temperatures keep increasing, sea level keeps rising, ice keeps melting just like the climate scientists predicted. You'd think in the nearly 200 years of climate science (Fourier discovered the greenhouse effect in 1824) if there was something seriously wrong with it that other scientists would have been able to point it out by now. Climate science has been under intense scrutiny for over 30 years now and no one has been able to shoot it down yet. Instead we have conspir