Science Is Littered With 'Zombie Studies' - Retracted Research Still Referenced By Others (thehill.com) 71
The Hill published this warning from an Information Sciences assistant professor:
Since 1980, more than 40,000 scientific publications have been retracted. They either contained errors, were based on outdated knowledge or were outright frauds... Yet these zombie publications continue to be cited and used, unwittingly, to support new arguments. Why? Almost always it's because nobody noticed they had been retracted...
Just by citing a zombie publication, new research becomes infected: A single unreliable citation can threaten the reliability of the research that cites it, and that infection can cascade, spreading across hundreds of papers. A 2019 paper on childhood cancer, for example, cites 51 different retracted papers, making its research likely impossible to salvage. For the scientific record to be a record of the best available knowledge, we need to take a knowledge maintenance perspective on the scholarly literature... And we need to build on that knowledge, not on the errors and fraud...
Slow science, living articles and reducing the pressure to publish are among the interventions that could help. We need a healthy, trustworthy ecosystem that rewards effort, not just results... Individuals and organizations that do the work of science must ensure that the work doesn't end at publication. Sometimes, it is just the beginning.
Just by citing a zombie publication, new research becomes infected: A single unreliable citation can threaten the reliability of the research that cites it, and that infection can cascade, spreading across hundreds of papers. A 2019 paper on childhood cancer, for example, cites 51 different retracted papers, making its research likely impossible to salvage. For the scientific record to be a record of the best available knowledge, we need to take a knowledge maintenance perspective on the scholarly literature... And we need to build on that knowledge, not on the errors and fraud...
Slow science, living articles and reducing the pressure to publish are among the interventions that could help. We need a healthy, trustworthy ecosystem that rewards effort, not just results... Individuals and organizations that do the work of science must ensure that the work doesn't end at publication. Sometimes, it is just the beginning.
Rewarding bullshit (Score:5, Interesting)
Or it rewards falsifying data until you've built a decades and billions of dollars of research on false claims only to find out after the fact it was all for nothing and you're forcefully ejected as head of your department at Stanford. Or it rewards generic paper mill cheats that just print falsified data that just agrees with already established dogma. Or it rewards skewering statistics so hard you'd believe a coin flip had a 99% chance of landing on heads every single time.
There's nothing at all about the current reward system for scientific research outside a corporation that rewards anything like results. The pioneers of MRNA vaccines barely made it through any grants at all before it was finally acknowledged such might be useful and they were given their Nobel Prizes, while there are tenured professors wander around claiming to have found aliens every other week. Yes I just said in comparison corporations are better, do you have any idea how low of a bar that is to somehow limbo under?
Re: (Score:3)
"It often happens that, if a lie be believed only for an hour, it has done its work, and there is no further occasion for it".
- Jonathan Swift, “The Examiner”, 1715.
Re:Rewarding bullshit (Score:5, Insightful)
Re: (Score:2)
Most egregious cases come from social sciences [nationalpost.com]. However, society has much lower expectations for that field, so outright fraud that happening in social sciences isn't as big of a scandal as amyloid research. It should be, as what was hallucinated by social scientists now spilling into broader society in form of devastatingly harmful policies and social movements (e.g., defund the police, reverse discrimination, DEI ).
Re: (Score:2)
Re:Rewarding bullshit (Score:4, Insightful)
Well stated, and I agree completely. I would point out that there are yet more facets to this problem.
I have been a reviewer for journals (medical). I have written detailed reviews explaining why some fluffy article should be rejected as repetitive or duplicative with nothing original, generally weak, poor language and spelling, faulty methods and analysis, unaware of prior art and knowledge, erroneous conclusions - or - why it has merit and should be published but only after some corrections. But, despite that, next month the article appears in print as is, unedited, uncorrected.
In my own experiences of this sort, I always felt that the issue was the old adage, "never attribute to malice that which can be attributed to stupidity". I never thought that there was any intent to defraud or deceive, nothing nefarious, just a blind obsession on the part of the publishers or editors-in-chief that they had to fill the pages, which I understand in the abstract, but they always had more material than they needed.
These experiences may differ based on the field of study or the journals involved, and undoubtedly "darker" motivations exist in some places, but I think that many bad publications are a consequence of mental and material laziness, not deliberate falsification.
Also -
The article and comments make me think that there should be some sort of international clearing house or listing service of retracted publications. Any editor or publisher receiving new papers for review should cross check the references against the blacklist, or require the authors to do so. That would be a simple line of defense.
Re:Rewarding bullshit (Score:5, Informative)
There is: Retraction Watch [retractiondatabase.org]. But just checking new papers wouldn't be enough: the review process is often slow enough that you need to recheck at each step in case of new retractions.
The other issue here is how important they are. TFS says that "A single unreliable citation can threaten the reliability of the research that cites it" and that's true if it relies on the cited paper, but I get the impression that increasingly a lot of citations are merely to establish a claim to know the state of the art and win over the the reviewers.
Re: (Score:2)
You've got to set the rewards mechanism to avoid reinforcing improper actions. Simply linking stuff together won't do that.
Re: (Score:2)
Unfortunately, that is very true. The whole system is broken with actually good research having trouble getting published and bulshit ruling supreme. The same is then true for the scientists: If you invest the time and effort to do actually valuable research, you have _less_ publications and usually no chance for a longer-term position or tenure. Hence bullshit is what rules everything as the bullsit-artists are the only ines that make it in the current system.
Perverted incentives all around.
Re: (Score:1)
My view is that this, like so many other problems today, would go away quite quickly if we just demanded one thing - accountability.
Re: (Score:3)
What does that even MEAN in this context? If I quote something that is later retracted, what accountability do you suggest?
Re: (Score:1)
Fair question. I suggest that the author of the retracted paper, the journal that reviewed and published the paper, and, yes, authors that reference the retracted paper should be negatively impacted – maybe by establishing come type of broadly agreed to guidelines as to how the reputation of the various actors would be effected. Otherwise, without adverse impacts, this will just continue to contribute to the ever increasing diminution of the public trust in science as a whole. Not saying this would
Blame or Fix? (Score:5, Insightful)
We can blame people for being "lazy" or we can fix the problem by automating retraction (or just plain bad reference) detection, the science version of "spidering for dead links."
Which of the two do you think people prefer?
Re: (Score:2)
Hard to Fix (Score:2)
We can blame people for being "lazy" or we can fix the problem by automating retraction
How do you automate retraction though? You can't just retract any paper that cites a retracted paper because the citation might be citing the paper as an example of something going wrong or the citation may have no impact on the paper's result.
The fact that we look up papers online helps: retracted papers can be immediately flagged as such so it is clear that nobody should use them for the scientific content. We could go a step further and add a warning to papers that cite a retracted paper that the res
Re: Hard to Fix (Score:2)
Cites and "Anticites"?
This would also help spread the fact that the Anticites were retracted.
Re: (Score:2)
GPP actually said "automating retraction detection", although I don't blame you for overlooking the second half of the noun phrase given the formatting thereof.
Re: (Score:2)
GPP actually said "automating retraction detection"
Yes, but that does not make any sense at all. Retraction is a deliberate process, it does not randomly happen without people making it happen so there is no need for "detection" unless you want to do something like automatically retract papers that cite the retracted paper. It's possible the OP meant notification but then you need figure out who read the paper and good luck with that.
Re: (Score:2)
Well, it could be useful when deciding whether or not to cite a reference, but it wouldn't tell you after you've already published with the cite in place.
Re: (Score:2)
Well, it could be useful when deciding whether or not to cite a reference
As long as you actually look up and read the reference seeing that it has been retracted is not a problem since, with everything being online, it is very clear when you access it that there has been a retraction. The problem is that some people just give references without ever having looked up the paper and that's a different problem.
Goodhart's Law again (Score:3)
When a measure becomes a target, it ceases to be a good measure.
Re: (Score:2)
Exactly so!
When has it been any different? (Score:3)
People were still writing about the ether years after Michelson and Morley, and if you go further back it gets a lot worse.
Not zombies; far worse (Score:2)
In science, it is common by those that understand it, to say that they stood on the shoulders of giants. The giants were not just the Einsteins, but all the ppl that did research and contributed to our knowledge. The problem becomes when you have not just lies, poorly written, but those with agendas. This is why soft science is really not to be trusted. Any science based on stats, is prone to manipulation and is hard to fight. Hard sciences will generally fin
Re: University (Score:2)
Re: University (Score:1)
This number is only an hypothesis, I need funding to publish my findings. And that's the number I am going to find, as "peer-review" is broken enough to pass almost anything. Then it will be science and you have to "trust the science". And, by the way, "numbers don't lie", so it will also be the Truth.
Or in other words... (Score:2)
Re: (Score:2)
Unfortunately it's very easy (or can be made to appear very easy by appropriate oversimplification) to measure quantity. Indeed, a large amount of what passes for !management science" consists precisely of learning how to oversimplify complicated systems in order to make them appear easily measurable and thus comparable.
Measuring quality is very difficult. In the first place it tends to be highly subjective and controversial. Also what looks good at first sight sometimes turns out to start smelling bad afte
Re: (Score:2)
The slight problem with that is that you also need to get rid of a lot of low-quality researchers and need to replace them with high-quality ones that mostly have gone to industry in frustration. Hence you face a steep uphill battle there.
Re: (Score:2)
Industry professionals publish research. It's not the sole domain of acedemia.
Re: (Score:2)
I am aware. I have one industrial journal publication and several conference ones. But most industrial people that would be qualified do not publish because their employers do not support it or the work needed to have something to publish.
Re: Or in other words... (Score:2)
Re: (Score:2)
There is no such thing as "poor-quality knowledge" and "useless" implies much more than "less valuable". Stick to quoting others for insights.
Re: (Score:2)
Serious question (Score:1)
With the state of science as it stands, with all the retracted studies, studies still linking to those... It seems foolish to just trust any new study without manually checking all those links personally every single time.
I'm gonna assume that most people are like me and don't have the energy or know-how to do this.
And yet I remember people here attacking "anti vaxxers" to a degree of profanity that is unbecoming to adults.
So even if "anti vaxxers"were scientifically wrong... in light of stories like this o
Re: (Score:2)
I'm gonna assume that most people are like me and don't have the energy or know-how to do this.
You'd be quite wrong if the population you're considering are the scientists in a field that make a difference. Proper research starts with deep knowledge of the state of the art, and for that you need to be up-to-date on the important stuff, which means the seminal papers, the important follow-ups, and basically everything new on a more or less regular basis. This puts you in a position to be able to weed out crap rather fast.
If you do not follow a field closely, it is quite unlikely that you'll be able to
Re: (Score:2)
Which is why school science fairs are such a failure.
Re: (Score:2)
If you can find it, I recommend you read an article by Isaac Asimov called "The Sound of Panting", which is about the problem of getting rid of published results that are known to be bad.
Re: (Score:2)
If I can find what?
Re: (Score:2)
To whomever downvoted the above, I will assume you really hate answering the question so you want to make it go away, huh?
Meanwhile, the question is very valid. Both the salt and cholesterol craze have shown nicely how much medical science suffers from fads and bad methods or outright manipulation.
I'm starting to understand "anti-vaxxers" more and more. The urge to shoot stupid people in the face who want to shout you down is enormous.
And just to be clear, it's not the differing opinion that deserves a good
Re: (Score:2)
Perhaps people vote it down, not because of the question, but because of all the other stupid shit in the post. I suspect you aren't just "starting to understand" "anti-vaxxers"...
Re: (Score:2)
Well, now we add ad hominem to the list.
Not a single argument. You find my points stupid but you don't even know whether I'm the one with the better information or you.
You might just happen to be an MD with extensive experience in research and impeccable ethics and in that case, I would absolutely defer to your better understanding of things... but I doubt it. Such people usually are above ad hominems.
Re: (Score:2)
Time to introduce dependency graphs?
Re: (Score:3)
And yet I remember people here attacking "anti vaxxers" to a degree of profanity that is unbecoming to adults.
As they deserve to be. Vaccines, all vaccines, have proven to be astonishingly safe. It's why we no longer have smallpox to worry about. Polio and measles are effectively non-existent EXCEPT in those who [cdc.gov] don't get vaccinated [umn.edu]. Pick any affliction you want. Without vaccines all would be running rampant today through the population. Tens of millions would be hospitalized, have lifelong medical iss
Re: (Score:2)
I think it's understandable that many people don't trust "science" these days, but not because of stories like this one. Stories like this and like the reproducibility problems in some fields don't get enough mainstream coverage to impinge heavily on the general public. The politicisation of climate science is a far more significant factor; and, particularly in the US context, the influe
Re: (Score:2)
"stories like this one" are intended to sew distrust in science, not the other way around. Science is imperfect, research is invalidated continuously. That's why peer review exists in the first place.
Re: (Score:2)
Peer review isn't a panacea. Peer review seems impotent and mostly powerless in the face of the onslaught of bullshit of late.
Peers we're talking about are humans, and they have natural and physical limitations on how much bullshit they can filter per day. Try to stop an avalanche with a shovel.
AI might help, but unfortunately, it might help the bullshitters even more.
Any "numbers game" favours idiocy over integrity (Score:3)
The maximum height of the ceiling on the usefulness of new knowledge *can* be limited by the minimum height of the floor of old disinformation on nonsense.
Journals are rather more like hypebeasts and tabloids in action than serious publishers of "we screwed up, we reported this as if it were sound, it isn't", and it isn't a new problem, and it is't one that has seen significant changes ever since Lancet's "Vaccines = autism, tell all your Facebook friends!" moment.
Considering the stakes for science funding and consequences for failures to separate plausibly real from self evident nonsense, journals really are a goddamn shitshow.
John Stossel did a great report on that (Score:1)
Re: (Score:2)
The further right-wing you go, the more real that stupid shit looks. Oh look, a "libertarian".
Re: (Score:3)
Re: (Score:2)
Both sides BS. You just cherry pick from both sides and claim they both do it; implying they are the same and equal. This is also used as an excuse to remain ignorant and keep things the same. Lazy simpleton; maybe cowardly. A "right wing" bias.
The "right" being the maintainers of the status quo of the ruling / owning class having to sit on the right side of the room because the left side of the room would kick their ass because it's revolutionary times and the monarchy is close to collapse (it's a French
Re: (Score:2)
Re: (Score:3)
People in government now use your money to promote left-wing advocacy
Really?? Show me the results from the most recent federally-funded gun or firearm injury study...
I'll wait...
The replication problem makes this nearly moot (Score:2)
Hell, the NON-retracted papers are often half bullshit as well.
https://en.m.wikipedia.org/wik... [wikipedia.org]
OMG! 40k scientific publications retracted!! (Score:2)
Since 1980, more than 40,000 scientific publications have been retracted
... and something like 64 million [wordsrated.com] academic papers have been published since the year 1996. Some 87.000 [ucsd.edu] papers were published on Corona virus alone from the beginning of the pandemic until 2022. According to Nature, 1.5–2% of all scientific papers published in 2022 likely originated in paper-mills. If you are in biology and medicine the rate is about 3%. Is this a problem? Yes. Should we launch into a fit of hysteria and panic and declare the death of science? No, this is a fixable problem. Persona
Re: (Score:3)
Exactly. This topic is a troll intended to sew distrust among the gullible. Clearly working too.
Re: (Score:2)
How often are those 40k retracted studies being cited elsewhere?
Citations != Research (Score:2)
Bad citations do not make research itself "unsalvageable". If there is nothing but citations, there is no research.
Re: (Score:2)
So you are dubious about the " meta study".
I can't say I blame you.
Disappointing (Score:4, Funny)
Came here expecting zombies. 0/10 not enough undead.
Study (Score:1)