Journals With High Rates of Suspicious Papers Flagged By Science-Integrity Startup (nature.com) 28
schwit1 shares a report from Nature: Which scientific publishers and journals are worst affected by fraudulent or dubious research papers -- and which have done least to clean up their portfolio? A technology start-up founded to help publishers spot potentially problematic papers says that it has some answers, and has shared its early findings with Nature. The science-integrity website Argos, which was launched in September by Scitility, a technology firm headquartered in Sparks, Nevada, gives papers a risk score on the basis of their authors' publication records, and on whether the paper heavily cites already-retracted research. A paper categorized as 'high risk' might have multiple authors whose other studies have been retracted for reasons related to misconduct, for example. Having a high score doesn't prove that a paper is low quality, but suggests that it is worth investigating.
Argos is one of a growing number of research-integrity tools that look for red flags in papers. These include the Papermill Alarm, made by Clear Skies, and Signals, by Research Signals, both London-based firms. Because creators of such software sell their manuscript-screening tools to publishers, they are generally reluctant to name affected journals. But Argos, which is offering free accounts to individuals and fuller access to science-integrity sleuths and journalists, is the first to show public insights. "We wanted to build a piece of technology that was able to see hidden patterns and bring transparency to the industry," says Scitility co-founder Erik de Boer, who is based in Roosendaal, the Netherlands. By early October, Argos had flagged more than 40,000 high-risk and 180,000 medium-risk papers. It has also indexed more than 50,000 retracted papers.
Argos is one of a growing number of research-integrity tools that look for red flags in papers. These include the Papermill Alarm, made by Clear Skies, and Signals, by Research Signals, both London-based firms. Because creators of such software sell their manuscript-screening tools to publishers, they are generally reluctant to name affected journals. But Argos, which is offering free accounts to individuals and fuller access to science-integrity sleuths and journalists, is the first to show public insights. "We wanted to build a piece of technology that was able to see hidden patterns and bring transparency to the industry," says Scitility co-founder Erik de Boer, who is based in Roosendaal, the Netherlands. By early October, Argos had flagged more than 40,000 high-risk and 180,000 medium-risk papers. It has also indexed more than 50,000 retracted papers.
Okay, I guess (Score:3, Informative)
But you don't need AI to tell you a journal called Evidence-based Complementary and Alternative Medicine is junk. It tells you it is right there in the title.
A 7% retraction rate only means it's failed to retract the other 93%.
Re: (Score:2)
I have no problems with the title, as I had better results with SOME alternative medicines than I had with now traditional pharmaceutical-based treatments.
As a former hardliner skeptic (of the James Randi variety), itâ(TM)s too easy to fall into our own version of Dogma.
Unfortunately, the alt scenes attracts more than its fair share of quacks by its fundamental nature, so it wouldnâ(TM)t surprise me if the journal is junk status.
Re: (Score:3)
I won't pretend to know you well enough to know what changed your mind.
But I do know enough about alt-med to say it wasn't "Robust, high-quality, scientific evidence, with both a clearly understanding of method of action and strong clinical results."
Re: (Score:3)
Re: Okay, I guess (Score:2)
75% of medications have a natural/biological basis, they are plant derived or a synthetic form of a compound originally found in a plant or animal. And most of them were identified by studying traditional medicine.
Should go after social science grievance journals (Score:2)
Academic watch should get on the flawed, unscientific 'research' reported for decades from the grievance study journals.
There are thousands of agenda based research papers which have been used to shape the news, political discourse, laws made and government programs funded.
Essentially, the academic watch should try to recreate as many of the grievance studies papers which are heavily cited using modern scientific methods with unbiased sample groups, exclude self reported data, exclude only asking 1/2 the po
Re: (Score:2)
Not in Europe and European-influenced cultures. Until about 450 years ago, "medicine" meant following the teachings of Galen, many of which relied on assertion. The defining feature of the Renaissance is that it became permissible to question the accuracy of ancient Greek and Roman "science", and it took time for research to replace inherited wisdom. Bloodletting was still recommended as a treatment in at least one textbook published a mere century ago.
Now, if you want to talk about folk medicine, that's a
Re: (Score:1, Flamebait)
Re:Okay, I guess (Score:4, Interesting)
The problem with junk science is at the same time very serious, and overblown. It's overblown because it's pretty obvious, to anyone working with science, that "Evidence-based Complementary and Alternative Medicine" from Hindawi is going to be junk not worth your time reading it, citing it, or consider publishing with them. Hindawi is junk (and thankfully defunct), it's the one that tanks the stats, and their choices of journal names invariably give them away.
Junk publishers keep making up these pompous journal names. If you publish in journals, you daily receive spam to invite to publish in such journals. They play with the ego, call you "respected professor", say they were really impressed by your recent work "[some title they got from a database]" and invite you to publish "with a special discount", or even to be part of their scientific board.
If you're an academic from first world, there's no reason to publish with junk publishers. The reason they exist is mostly because many academics from the third world cannot perform research with the same level of standards due to lack of funding, even if in good faith. Still they get evaluated on number of papers published and number of citations, and hence they need to game the system. Another reason is some poorly funded institutions have unacceptable criteria e.g. "each PhD student needs to publish 4 papers" and their ways are write (useless) literature reviews, or publish with junk publishers. Hence the "high risk" journals that is studied here (due to scientists who are otherwise respectable, but occasionally publish in junk journals).
If you're from first world, you certainly have an understanding of what publishers and journals are junk, just by reading their name and getting spam from them. I'm not sure who wastes their time reading these journals, or dirties their own publications by citing junk journals. In any case, serious (normal) universities from US/EU/the usual friends classify journals and publishing with less renown journals (e.g. classified Q3/Q4) doesn't give any career points.
"Everybody" (science professionals) knows which journals and publishers are junk, and avoid publishing/citing these references. The problem is when science results reach a larger public that takes everything at face value.
Re: (Score:1)
The problem with junk science is at the same time very serious, and overblown. It's overblown because it's pretty obvious, to anyone working with science, that "Evidence-based Complementary and Alternative Medicine" from Hindawi is going to be junk not worth your time reading it, citing it, or consider publishing with them. Hindawi is junk (and thankfully defunct), it's the one that tanks the stats, and their choices of journal names invariably give them away.
Junk publishers keep making up these pompous journal names. If you publish in journals, you daily receive spam to invite to publish in such journals. They play with the ego, call you "respected professor", say they were really impressed by your recent work "[some title they got from a database]" and invite you to publish "with a special discount", or even to be part of their scientific board.
If you're an academic from first world, there's no reason to publish with junk publishers. The reason they exist is mostly because many academics from the third world cannot perform research with the same level of standards due to lack of funding, even if in good faith. Still they get evaluated on number of papers published and number of citations, and hence they need to game the system. Another reason is some poorly funded institutions have unacceptable criteria e.g. "each PhD student needs to publish 4 papers" and their ways are write (useless) literature reviews, or publish with junk publishers. Hence the "high risk" journals that is studied here (due to scientists who are otherwise respectable, but occasionally publish in junk journals).
If you're from first world, you certainly have an understanding of what publishers and journals are junk, just by reading their name and getting spam from them. I'm not sure who wastes their time reading these journals, or dirties their own publications by citing junk journals. In any case, serious (normal) universities from US/EU/the usual friends classify journals and publishing with less renown journals (e.g. classified Q3/Q4) doesn't give any career points.
"Everybody" (science professionals) knows which journals and publishers are junk, and avoid publishing/citing these references. The problem is when science results reach a larger public that takes everything at face value.
It's not just the odd individual dodgy publication in journals that can bring disrepute to a university department. And it's not always a 3rd world problem.
A single dodgy department can taint an entire university and cast doubt over all research and scientific output from all academic faculties that it interacts with. A prime example is the Lithuanian University of Health Sciences.
Homeopathy
University's own pharmacy manufactures and actively promotes homeopathic medicines.
https://en.wikipedia.org/wiki/... [wikipedia.org]
I'
Re: (Score:3)
Until there are consequences nothing will change (Score:5, Insightful)
Until there are consequences to the reputations of the institutions paying salaries and giving out grants nothing will change. We will continue to reward fraud more than real research. What we need is real criminal fraud charges against people who sponsor and reward bad science to further their own goals. The PHD student creating a fake paper that proves what their advisor said was publishable is just doing what they were taught in university. The student who didn't learn in first year to fake their lab results never made it to a masters.
Re: (Score:1)
The PHD student creating a fake paper that proves what their advisor said was publishable is just doing what they were taught in university. The student who didn't learn in first year to fake their lab results never made it to a masters.
That PHD students degree literally culminated with the intense creation of a thesis paper, presented to a panel for a religious amount of scrutiny and approval.
Somehow, I do not feel THAT degree, teaches you how to “fake papers”. The profession is FAR more corrupt than the education, which has a FAR more limited profit motive by comparison.
Re: (Score:2)
Mod parent up.
Increasing integrity by sorting chains-of-authority increases value, cuts noise, rewards those who actually do the work, and cuts wheat from chaff.
So long as their methods are exposed and open to criticism, I'm behind their efforts 100%.
Hard to Separate Fraud vs Mistake (Score:4, Informative)
In fact they have every interest to look the other way so they can continue to say "look at the great research my grants paid for, or look at how qualified our faculty is".
That's not really true because the more you let this carry on the worse the reputational damage will be for you when it gets found out, and believe me it will get found out eventually even if that's after the researcher has retired. The problem is that the people most able to find out what is going on are resarchers at other institutes and often these things start out as academic disagreements with one or more researchers pointing out flaws or serious questions about results.
At this stage it is not always clear who is correct and even later when it becomes clearer the results may be wrong it is not always easy to show that this is due to fraud vs a genuine mistake: science is not easy. What makes it more challenging is that sometimes one fraudulent paper can lead to others fooling themselves into genuinely thinking they have produced compatible results - albeit not as clean and convincing as the original and this can really complicate the hunt for the truth.
If you have a look at the high-pressure, room-temperature superconductor scandal you'll see many aspects of the above: papers that have now been forcibly retracted over the authors' objections due to evidence of fabricated data; other groups who appear to genuinely think they have seen superconductivity but about which there are now serious doubts but no evidence of fraud etc. Tracking this down and figuring it all out takes a real expert in the field and the experts who did all that work in this case have not exactly been thanked for it despite the valuable work they did - they have been shouted at at conferences, had email campaigns organized against them and one was even banned from posting papers to the arXiv.
If we want to fix this then, rather than legal consequences, which would have to be decided in a court of law that has no scientific expertise, the more effective way would be to find ways to protect and reward those experts willing to investigate such cases to find out what is really going on because there are not many people who have the strength of character to weather the storm of abuse in order to fix the field they love.
Re: (Score:2)
And, of course, there are always honest researchers who are unable to replicate accurate results for various reasons. I've read of one case where a reputable physicist failed to reproduce the results of the Michelson-Morley experiment. These things happen, and if, by some fluke, somebody manages to repeat a bogus experiment and make it work, it may just mean that two researchers made the same mistake. Tish happens.
A few things (Score:5, Interesting)
A retraction can happen - sometimes there are mistakes.
However, institutions that have a large number of retractions probably have something in their culture that needs addressed - possibly by bans for a certain amount of time. Same with individual researchers.
If egregious enough, permanent bans would be in order. An example is Andrew Jeremy Wakefield, disgraced researcher who initiated the whole modern anti-vaxx movement, and had a direct conflict of interest to make money off his "findings" He's been struck off the registry in GB, and is not licensed in the USA. That dude deserved what he got. Having made some political connections, he is still able to work a bit of mischief.
His is an unusually egregious example, and for individual researchers, an anomaly.
There is a certain amount of money in research. Money often leads to fraud. The question however, is who is making the money? There is where you look for the cause, and there is where the fix lies.
Re: (Score:2)
Re: (Score:2)
No mod points today, so I am writing in instead.
I agree with everything you said - accurate, thoughtful, insightful.
But, I would make one small amendment. Your last paragraph repeatedly uses the word "money". While true, a better word would have been "incentive". As you so nicely point out in the first paragraph, the publish or perish culture is a system of incentives. It ultimately leads to employment or promotion, thus more money, but for some the motivation may be fame, prestige, society appointments
High trust social mechanisms (Score:2)
Re: (Score:3)
It's even worse when everyone involved is under personal financial pressure to be corrupt. "Publish or perish" pollutes the system with low quality, mostly irrelevant works from otherwise decent researchers. Reputation concerns motivate organizations to bury 'mistakes' rather than out low quality or fraudulent works.
There is no check mechanism with a sustainable motive to catch and punish bad actors in the system.
Re: (Score:2)
So does this start from the bottom up or top down?
Re: (Score:2)
No imagination required, of course. We had four years of American allies cluing in to how fickle the relationship is from the US side, and doing their best to diversify their economic dependencies away from the US while using delay tactics to wait things out hoping for a political flip after 4 years.
Re: High trust social mechanisms (Score:2)
publish or perish + to many locked into the ivory (Score:1)
publish or perish + to many locked into the ivory tower is leading to this.
The low pay and big loans does not help as well.
Now how much real job skills do some of people publishing this really have? vs ivory tower skills?
Startup??? (Score:1)
Is this a money-making scam^w scheme?
Re: (Score:3)
In theory... Build a reputation for integrity, and eventually people will pay for your certification.
In practice, I expect these things to evolve into extortion schemes where you can buy reputation.
You can only fight the financial motives for so long, and they're almost all on the 'corrupt' side of things.