The Papers That Most Heavily Cite Retracted Studies (nature.com) 23
Data from giant project show how withdrawn research propagates through the literature. Nature: In January, a review paper about ways to detect human illnesses by examining the eye appeared in a conference proceedings published by the Institute of Electrical and Electronics Engineers (IEEE) in New York City. But neither its authors nor its editors noticed that 60% of the papers it cited had already been retracted. The case is one of the most extreme spotted by a giant project to find papers whose results might be in question because they cite retracted or problematic research. The project's creator, computer scientist Guillaume Cabanac at the University of Toulouse in France, shared his data with Nature's news team, which analysed it to find the papers that most heavily cite retracted work yet haven't themselves been withdrawn.
"We are not accusing anybody of doing something wrong. We are just observing that in some bibliographies, the references have been retracted or withdrawn, meaning that the paper may be unreliable," Cabanac says. He calls his tool a Feet of Clay Detector, referring to an analogy, originally from the Bible, about statues or edifices that collapse because of their weak clay foundations. The IEEE paper is the second-highest on the list assembled by Nature, with 18 of the 30 studies it cites withdrawn. Its authors didn't respond to requests for comment, but IEEE integrity director Luigi Longobardi says that the publisher didn't know about the issue until Nature asked, and that it is investigating. Cabanac, a research-integrity sleuth, has already created software to flag thousands of problematic papers in the literature for issues such as computer-written text or disguised plagiarism. He hopes that his latest detector, which he has been developing over the past two years and describes this week in a Comment article in Nature, will provide another way to stop bad research propagating through the scientific literature -- some of it fake work created by 'papermill' firms.
"We are not accusing anybody of doing something wrong. We are just observing that in some bibliographies, the references have been retracted or withdrawn, meaning that the paper may be unreliable," Cabanac says. He calls his tool a Feet of Clay Detector, referring to an analogy, originally from the Bible, about statues or edifices that collapse because of their weak clay foundations. The IEEE paper is the second-highest on the list assembled by Nature, with 18 of the 30 studies it cites withdrawn. Its authors didn't respond to requests for comment, but IEEE integrity director Luigi Longobardi says that the publisher didn't know about the issue until Nature asked, and that it is investigating. Cabanac, a research-integrity sleuth, has already created software to flag thousands of problematic papers in the literature for issues such as computer-written text or disguised plagiarism. He hopes that his latest detector, which he has been developing over the past two years and describes this week in a Comment article in Nature, will provide another way to stop bad research propagating through the scientific literature -- some of it fake work created by 'papermill' firms.
Yet Trust The LLM (Score:1)
How poisoned are the models already? GIGO.
Re:Yet Trust The LLM (Score:4, Funny)
I know a gal who has 6 fingers on one hand, and LIKES the AI results. She feels less alone.
Re: (Score:1)
Roses are red, Violets are blue.
What has 6 fingers and makes a fool out of you?
We lack tools (Score:5, Interesting)
We actually do not have good tools to know that this is happening. I write papers all the time. And there is no way I can easily tell whether any of the papers that I commonly cite have been retracted.
I am glad this guy wrote a detection tool, because I may be citing retracted works without knowing myself.
Re: We lack tools (Score:1)
Re: (Score:3)
Yes, that is what you would have to do at this point. Follow the reference again to the publishers website and see before submitting the paper whether it has been retracted or not. And then again after revision. And then again at time of camera ready.
But that's a bit of work to detect a problem that is pretty rare and quite unlikely to impact what I wrote.
Re: (Score:2)
Ask a 2003 webmaster. Link/referral checking was part of the job. All the same identifier hassles, procedural issues, and mixed-standards complications.
Re:We lack tools (Score:5, Interesting)
there is no way I can easily tell whether any of the papers that I commonly cite have been retracted.
You shoud click the "Crossmark" icon in your pdf, which brings you to the website of the publisher, where any notice about the paper you are reading will be displayed. But I think this isn't the real problem. The problem is that you might be publishing a perfectly written review paper, only for the references you cite to be retracted within 6 months, and meanwhile your paper is published and you won't constantly check for your references to be still be current.
But publishers have tools. If you follow the link to the paper that Nature cites "in January, a review paper..." that is based on 60% retracted papers https://ieeexplore.ieee.org/do... [ieee.org]
In the "references", many of them have a red icon, which isn't explicited but if you follow the link it correlates exactly with papers that are indicated as retracted, and leads to 63% of retracted papers just as Nature says. I conclude that IEEE knows very well when papers are retracted, and displays an icon when it is the case.
IEEE could easily implement a metric on top of the paper "careful, this paper is based 60% of retracted research". They could even produce updated PDFs of the papers with a notice "careful the following references have been retracted since publication".
Re: (Score:2)
I agree this should be automatic.
Depending on style files of the venue you write for you have to deactivate references, remove crossref, URLs, and DOIs from your bibfile or the PDF get generated wrong. Now, the publishing format that still use no links in document is pretty stupid in my opinion but still quite pervasive.
Note that if there is a proper way to do this, we probably could check at the time latex runs and add a note in the pdf that can be silenced with an option.
Re: (Score:1)
There may be fee-based databases that can provide such info. A quick Googlebinging gives: Scopus, Web of Science, Eric, and PubMed.
Re:We lack tools (Score:4, Insightful)
We actually do not have good tools to know that this is happening. I write papers all the time. And there is no way I can easily tell whether any of the papers that I commonly cite have been retracted.
I am glad this guy wrote a detection tool, because I may be citing retracted works without knowing myself.
Even if detecting papers that were retracted at the time of submission or publication were practical, performing this detection on an ongoing basis after publication is not practical, at least not on a manual basis. If there were a tool to automated this detection, then that might be useful.
But we shouldn't ding a paper for citing a retracted paper without knowing what how the citation was used? Was it used to substantiate a claim, particularly an important claim? Or was it used as introduction/background/motivation filler? Or was it used as an example of an incorrect idea?
Re: (Score:2)
Let's say somebody write a paper that determines that those who play video games standing up live longer than those who play sitting down and it turn
Re: (Score:1)
Given the reproducibility problem many papers have, the fact that the current paper is adequately testing or building on the analysis of the cited study shouldn't be taken for granted.
Especially in some of the softer studies that revolve around surveys and the like.
Re: (Score:2)
There is the Retraction Watch Database [retractiondatabase.org], which is directly supported by reference managers Zotero and EndNote. Whenever a reference in your library is retracted and shows up in the database (it's not complete), the reference manager notifies you.
If you're a researcher and aren't using a reference manager, you're probably not very good at your job.
Re-cursing (Score:2)
It's fake turtles all the way down to the Fakeverse
Re: (Score:2)
Then the Fakeverse goes Pop! as AI exacts titanium controls...
Are they talking about slashdot? (Score:2)
There are a whole lot of comments that cite studies that are either bunk or retracted!
Just the tip of the iceberg (Score:5, Insightful)
Re: (Score:3)
yeah, there is a lot of junk out there.
Often the problem is that you can't tell whether what you are doing will be valuable before you are done doing it. At that point, might as well write it up and add epsilon to global knowledge.
Re: (Score:2)
Should do it to news articles next (Score:3)
So-called journalists that first put out a social media posts and then cite it in a "news" articles, and then take down the original post, while putting out more "news" articles that cites each other, is one common way to create fake news.
Re: (Score:3)
That describes most journalism with the exception that the first few steps are done by some PR guy who passes the story to an appropriate trusted journalist. That likely describes this story's origins.
The real problem the story identifies is there is a huge quantity of junk science out there, it is very difficult to weed it out and it becomes the basis of more junk science. And any attempt to actually weed it out would be extremely threatening to a lot of people. What if you were required to retract any pa
Re: (Score:2)
Why even bother with fake posts when you can just say "People say...". Weasel words.