Fake Scientific Papers Are Alarmingly Common 64
From a Science magazine report, shared by schwit1: When neuropsychologist Bernhard Sabel put his new fake-paper detector to work, he was "shocked" by what it found. After screening some 5000 papers, he estimates up to 34% of neuroscience papers published in 2020 were likely made up or plagiarized; in medicine, the figure was 24%. Both numbers, which he and colleagues report in a medRxiv preprint posted on 8 May, are well above levels they calculated for 2010 -- and far larger than the 2% baseline estimated in a 2022 publishers' group report. "It is just too hard to believe" at first, says Sabel of Otto von Guericke University Magdeburg and editor-in-chief of Restorative Neurology and Neuroscience. It's as if "somebody tells you 30% of what you eat is toxic." His findings underscore what was widely suspected: Journals are awash in a rising tide of scientific manuscripts from paper mills -- secretive businesses that allow researchers to pad their publication records by paying for fake papers or undeserved authorship.
"Paper mills have made a fortune by basically attacking a system that has had no idea how to cope with this stuff," says Dorothy Bishop, a University of Oxford psychologist who studies fraudulent publishing practices. A 2 May announcement from the publisher Hindawi underlined the threat: It shut down four of its journals it found were "heavily compromised" by articles from paper mills. Sabel's tool relies on just two indicators -- authors who use private, noninstitutional email addresses, and those who list an affiliation with a hospital. It isn't a perfect solution, because of a high false-positive rate. Other developers of fake-paper detectors, who often reveal little about how their tools work, contend with similar issues. Still, the detectors raise hopes for gaining the advantage over paper mills, which churn out bogus manuscripts containing text, data, and images partly or wholly plagiarized or fabricated, often massaged by ghost writers.
Some papers are endorsed by unrigorous reviewers solicited by the authors. Such manuscripts threaten to corrupt the scientific literature, misleading readers and potentially distorting systematic reviews. The recent advent of artificial intelligence tools such as ChatGPT has amplified the concern. To fight back, the International Association of Scientific, Technical, and Medical Publishers (STM), representing 120 publishers, is leading an effort called the Integrity Hub to develop new tools. STM is not revealing much about the detection methods, to avoid tipping off paper mills. "There is a bit of an arms race," says Joris van Rossum, the Integrity Hub's product director. He did say one reliable sign of a fake is referencing many retracted papers; another involves manuscripts and reviews emailed from internet addresses crafted to look like those of legitimate institutions. Twenty publishers -- including the largest, such as Elsevier, Springer Nature, and Wiley -- are helping develop the Integrity Hub tools, and 10 of the publishers are expected to use a paper mill detector the group unveiled in April.
"Paper mills have made a fortune by basically attacking a system that has had no idea how to cope with this stuff," says Dorothy Bishop, a University of Oxford psychologist who studies fraudulent publishing practices. A 2 May announcement from the publisher Hindawi underlined the threat: It shut down four of its journals it found were "heavily compromised" by articles from paper mills. Sabel's tool relies on just two indicators -- authors who use private, noninstitutional email addresses, and those who list an affiliation with a hospital. It isn't a perfect solution, because of a high false-positive rate. Other developers of fake-paper detectors, who often reveal little about how their tools work, contend with similar issues. Still, the detectors raise hopes for gaining the advantage over paper mills, which churn out bogus manuscripts containing text, data, and images partly or wholly plagiarized or fabricated, often massaged by ghost writers.
Some papers are endorsed by unrigorous reviewers solicited by the authors. Such manuscripts threaten to corrupt the scientific literature, misleading readers and potentially distorting systematic reviews. The recent advent of artificial intelligence tools such as ChatGPT has amplified the concern. To fight back, the International Association of Scientific, Technical, and Medical Publishers (STM), representing 120 publishers, is leading an effort called the Integrity Hub to develop new tools. STM is not revealing much about the detection methods, to avoid tipping off paper mills. "There is a bit of an arms race," says Joris van Rossum, the Integrity Hub's product director. He did say one reliable sign of a fake is referencing many retracted papers; another involves manuscripts and reviews emailed from internet addresses crafted to look like those of legitimate institutions. Twenty publishers -- including the largest, such as Elsevier, Springer Nature, and Wiley -- are helping develop the Integrity Hub tools, and 10 of the publishers are expected to use a paper mill detector the group unveiled in April.
It is only going to get worse (Score:1)
Re: (Score:1, Insightful)
yep. and probably make them more difficult to catch.
Also, incentive problem.
Your entire academic career depends on publishing, so you have an incentive to cheat.
Re:It is only going to get worse (Score:5, Insightful)
The real bitch in ferreting it out is also the circular reference problem.
You get a bogus paper referring to another bogus paper and another and another. And after enough layers, it becomes EXCEEDINGLY difficult to filter BS as it gets entrenched in the paper-sphere.
Re: (Score:2)
The real bitch in ferreting it out is also the circular reference problem.
You get a bogus paper referring to another bogus paper and another and another. And after enough layers, it becomes EXCEEDINGLY difficult to filter BS as it gets entrenched in the paper-sphere.
Until you try to implement something based on the papers' findings.
Re: (Score:3)
This is a good thing. The more the bullshit is exposed the sooner the problem gets fixed.
Re: (Score:1)
Only if all the bullshitters involved face consequences like getting kicked out of their field.
Re: (Score:3)
I've found that this all varies depending upon the field. Ie, Chemistry has lots of names on each paper (or did) with the senior member always listed first. For CS, the number of names were smaller, 2 to 4, with the student's name often going first. Though it probably depends upon the prof too. Junior Professors are indeed under a lot of pressure.
I also think it's not just now that times have changed, but there's been change for the last century possibly. I know when I was in grad school in early 90s t
Re: (Score:2)
I'm not sure of how this will fall out here, but long term research a few years ago found that half of research results are wrong. If some are faked, then that might explain why that number was so high.
Re: (Score:3)
Re:It is only going to get worse (Score:4, Insightful)
It doesn't need to be novel or coherent. It just needs to get past the 'filters'.
So it can take existing papers and re-summarize them, or combine 2 papers and summarize them, ad nauseum.
And It doesn't matter if its very coherent, as quite a few places have very poor rigour when reviewing submitted papers.
Re: (Score:2)
ChatGPT can't write a coherent paper.
True. Neither can most student, though.
Re: (Score:2)
Fraud (Score:4, Interesting)
Re: (Score:2)
Re: (Score:2)
Re:Fraud (Score:5, Interesting)
While true, that's a horribly oversimplified model. So oversimplified that acting on it is incorrect.
I think that the BASIC problem may be the way the patent system is broken. If two groups independently invent something, only one can get the patent. Independent replication should be what is *demanded*. And both (all?) should get equal compensation.
There's a problem here, of course, when the cost of the experiment is so large that it can't be duplicated, but such results should not be considered resolved. (What's the mass of a W-boson? ... We've to two independent answers, and they don't quite agree. So if we only had one, should be believe it?)
Paper Mills (Score:5, Informative)
Re: (Score:2)
When academics have to go by "publish or perish" then publish they will. Hence a whole lot of bullshit gets published.
So an academic paper mill is ... (Score:1)
... a paper-manufacturing plant run by an agricultural university [wikipedia.org]??? Who knew?
Re: (Score:3)
Re:Paper Mills (idiom) (Score:2)
I'd just like to point out that a 'paper mill' is actually a factory where they manufacture paper.
Yeah, American idioms are one of the hardest things for non-native speakers to understand. Recently I used the phrase "kick the can down the road" and was met with a blank look until I explained the meaning. Then I had to explain that "kick the can down the road", "kick the can", and "kick the bucket" were all totally different concepts.
There are idioms in other languages that don't translate into English very well either. I was recently told to "shake the fleas" in French and I had that same blank look
Medicine is not a Science (Score:1)
Legitimate science, though, has no defense against fake papers. And it should not. Finding have to taken at face value, with a grain of salt. Ongoing research will determine if the findings have validity or use.
It is the mainstream belief that science and research discover facts, rather than being a process to determine what might be real, that causes all these problems. We see this in education where the maj
Re:Medicine is not a Science (Score:5, Insightful)
Legitimate science, though, has no defense against fake papers.
The scientific defense against fake papers is experiment and reproducibility. Science is built on the idea that you can't trust people.
Re: (Score:2)
MAKE REPRODUCTION SEXY AGAIN
er wait
Seriously though, how do we get more attempts to reproduce results?
Re: (Score:2)
From a minimalist perspective, it will happen automatically (slowly and haphazardly) as people try to use those results.
Re: (Score:2)
Does this mean the authors were lying or we just couldnâ(TM)t do it. We had talent and money. But maybe not skill.
Re: (Score:2)
Does this mean the authors were lying or we just couldnâ(TM)t do it
Could be either one. That sucks though.
Re: (Score:2)
We had talent and money. But maybe not skill.
I'll give a woodworking example/Analogy For several years a very skilled young woodworker worked at a company and became known for his excellent work, so well that the customers started saying he should start out on his own so he decided he would.
He had talent and money so he bought all the best tools, mitre saws, and table saws, and set up shop and the customers came to him with the orders yet not a single piece he made was perfect as it was in the last company. After a few months he had to close his b
Re: (Score:2)
Science is built on the idea that you can't trust people.
Especially don't trust scientists until the results/theorems/models are independently verified
Re: (Score:3)
Re: (Score:2)
The funny thing was all our professors were saying donâ(
Re: (Score:2)
The authors of the paper are in the neuroscience field, so they had an interesting in splitting their own field out and lumping all others together. In the paper, "medicine" and "general medicine" is used to refer to all medical sciences with the exception of neuroscience.
Being pedantic does your argument no favors.
Re: (Score:2)
He was telling me of one truly fascinating case. He also told me that he was behind on his publications, so one one trip he wrote 5 papers on the case. Three were accepted. One data point, three papers. Not rigorous science.
Psychological bunk is DANGEROUS (Score:2)
Those charlatans spew way too much dangerous shit.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Trust the science! (Score:2, Funny)
Of course one quarter of it is fraud, but hey those are still good odds, unless this study is fraudulent then....
Re: (Score:2)
Of course one quarter of it is fraud, but hey those are still good odds, unless this study is fraudulent then....
Actually, they said fraud or plagiarized.
Unless the paper plagiarized was fraud, the plagiarized papers may still be good results.
Re: (Score:3)
Did you notice that they mentioned a high false positive finding? And they listed their criteria for deciding that a paper is fraudulent in some way ("Sabel's tool relies on just two indicators -- authors who use private, noninstitutional email addresses, and those who list an affiliation with a hospital."), so the high false positive rate isn't surprising.
This may be a decent tool for gathering statistics (with large error bars), but it would be reckless to use it to judge an individual paper.
Publish or perish (Score:5, Insightful)
Two problems.
First, thete are too many marginal people in research. Schools want grad students, and let people in who aren't really capable of contributing to their field. But they make for cheap labor to grade undergrad homework and to produce reams if crappy papers.
Which brings us to the second problem: "publish or perish". Universities can't evaluate research on its merits, so they reach for a simple yardstick: how many papers have you published this year?
Combine the two, and most published research is crap. Even good research is broken up into "MPIs" - minimum publishable increments - so that you get as many papers as possible out of each project.
Re: (Score:1)
>reams [of] crappy papers
I wonder if Einstein's paper theorizing mass-energy equivalence could even get published today. Its text can fit on a single page, and he didn't provide experimental data.
Re:Publish or perish (Score:4, Informative)
Schools want grad students, and let people in who aren't really capable of contributing to their field. But they make for cheap labor to grade undergrad homework and to produce reams if crappy papers.
Which brings us to the second problem: "publish or perish". Universities can't evaluate research on its merits, so they reach for a simple yardstick: how many papers have you published this year?
Here's the problem. Neither of those problems apply to this result. From The Fine Summary:
Sabel's tool relies on just two indicators -- authors who use private, noninstitutional email addresses, and those who list an affiliation with a hospital.
If this flood of trash papers was being done to counter the "publish or perish" mandate from universities, they would use their academic email addresses in their by-lines to make damn sure they got credit. That's not happening. They're using Yahoo addresses and such-like. Personally I think scientists should have switched to using addresses they personally control for their publications a long time ago because no academic affiliation is forever, even before tenure was gutted, and being able to have a constant email address for all of your publications would help establish your personal brand, but to make that stick you have to do really good work. Instead academic scientists have been leaning heavily on the cachet of their specific academic affiliations for many many years, and now the well has been poisoned.
In any case, because the habit of clinging to their affiliations is so strong, papers published without them are legitimately suspect. No one playing the academic game would hide their affiliation. That's quite the opposite of their need.
Re: (Score:2)
Not sure what was done here, but email address and affiliation do not need to correspond. I know many people who use their private email on papers if they expect to change institution in the future.
Quis custodiet ipsos custodes (Score:2)
We need to fund replication experiments (Score:5, Interesting)
We need to start putting significant money and resources toward replicating important published science. And they can pay for it off the torts earned when they find fake research.
We also need to end the "publish or perish" mindset [slashdot.org] in academia, and to acknowledge the value of neutral or negative results [slashdot.org] instead of expecting every paper to be an enormous breakthrough. Science needs to proceed like a tortoise, slow and steady. No fast but with decades-long setbacks due to falsified information.
AI to the rescue? (Score:2)
It seems like this would be a good place to deploy AI to do a preliminary evaluation of the mathematics and pointing out possible issues. It seems like plagiarism should be caught rather easily considering the technology to detect it has been used on college papers for a couple decades.
Manipulated data is probably the largest issue here as without the original, it can only be detected by evaluating the experiment.
Questionable methodology (Score:2)
So, this guy uses two criteria for detecting fake papers: (1) authors who use private, non-institutional email addresses and (2) authors who list an affiliation with a hospital. My field is computer architecture and not medicine, but it seems like the real problem is low quality conferences and journals. Obviously the guy's two criteria are useless for computer architecture conferences. I wonder if these two criteria are even slightly useful for top-level medical journals.
The other big problem is the de
Re: (Score:2)
Neither private email nor affiliation with a hospital is a indication for a fake publication. Also in medical journals review is blind (or double blind). In most scientific journals (not just medical) authors can suggest reviewers, but an editor would not select a reviewer which might be biased. There are low-quality journals (in all fields) that published essentially anything, but your reputation as a scientist does not improve if you publish in such journals.
I also do not believe that most low-quality or
Re: (Score:2)
Also looking at the original paper it is also not really what they are doing. They look at various indicators for fake papers and try to validate them. They then find that "private email" and "hospital affiliation" has a detection sensitivity was 0.86 and the false-alarm rate
0.44 when validated on a set of papers known as fakes and presumed not fakes and suggest that this criterion could be used as a red flag.
Chicken Chicken Chicken paper (Score:2)
It's not that hard to get published. [isotropic.org] The hard part is delivering it at a conference [youtu.be] with a straight face.
In the humor section.
There are fake papers... (Score:2)
End the publish or perish insanity (Score:2)
If there was no requirement to publish crap all the time to stay in a field, keep a teaching job, do some basic but not exciting research on the side, then there would be no incentive to cheat the system on massive scales. And 34% is massive. Even the 2% claimed is really bad for the field of "science" which is supposed to be a search for truth. Oh, the irony!
Without all the faked up and honest but shitty papers getting cranked out there would be lots more resources available to vet the papers that did g
China (Score:1)
Remember this article the next time "researchers" claim China dominates the world in cited scientific papers.
Publishers are culpable (Score:2)
Global warming? (Score:1)
Trends? (Score:2)
Re: (Score:2)
That would be my expectation, based on experience. When I found plagiarism in a paper I was reviewing, that's where it was from. The editors let them get away with it after it was brought to the editors' attention.
Re: (Score:1)
ChatGPT says "Hold my beer..." (Score:2)
Hopefully he'll have a sense of humor about this a year from now when his perception of "common" has been adjusted by a few orders of magnitude. :)
My opinion (Score:1)