Some of the World's Most-Cited Scientists Have a Secret That's Just Been Exposed (sciencealert.com) 88
Long-time Slashdot reader schwit1 quotes Science Alert:
Among the 100,000 most cited scientists between 1996 to 2017, there's a stealthy pocket of researchers who represent "extreme self-citations and 'citation farms' (relatively small clusters of authors massively citing each other's papers)," explain the authors of the new study, led by physician turned meta-researcher John Ioannidis from Stanford University.
Ioannidis helps to run Stanford's meta-research innovation centre, called Metrics, which looks at identifying and solving systemic problems in scientific research. One of those problems, Ioannidis says, is how self-citations compromise the reliability of citation metrics as a whole, especially at the hands of extreme self-citers and their associated clusters. "I think that self-citation farms are far more common than we believe," Ioannidis told Nature. "Those with greater than 25 percent self-citation are not necessarily engaging in unethical behaviour, but closer scrutiny may be needed."
Ioannidis helps to run Stanford's meta-research innovation centre, called Metrics, which looks at identifying and solving systemic problems in scientific research. One of those problems, Ioannidis says, is how self-citations compromise the reliability of citation metrics as a whole, especially at the hands of extreme self-citers and their associated clusters. "I think that self-citation farms are far more common than we believe," Ioannidis told Nature. "Those with greater than 25 percent self-citation are not necessarily engaging in unethical behaviour, but closer scrutiny may be needed."
Goodhart's law (Score:5, Insightful)
Just another example of Goodhart's law in action:
"Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes."
In other words, when a measure becomes a target, it ceases to be a good measure. In the publish or perish world of academia, where the pressure to publish & get good metrics is intense, the scientists were just doing what the metrics system asked of them. Yes, there are extreme cases like the ones mentioned int the article, but there's an "understanding" between many researchers that citing each others' papers is a good thing to do. It's reciprocal altruism. Rather than pointing the finger at individuals, why not change the system so that we can have better quality science?
Re: (Score:3)
Citation needed. ;-P
Re:Goodhart's law (Score:4, Informative)
There you go:
Goodhart, C. (1981). Problems of Monetary Management: The U.K. Experience. In A. S. Courakis (Ed.), Inflation, Depression, and Economic Policy in the West (pp. 111–146). Rowman & Littlefield.
Re: (Score:2)
I was being ironic, but thanks.
Re: (Score:2)
I was being ironic, but thanks.
Strange thing is that I have known this for at least 30 years. Amazing that scientists have "kept this a secret" for so long.
There have always been citation sluts and those who implant their co authorship on every paper they can get away with. We always laughed at them.
In the end, it doesn't make much difference. It's the science version of people who collect friends on FaceBook.
Re: (Score:2)
I do medical research. During my PhD I was told to include someone as a co-author because 'we might want to collaborate with him in the future.'
A few of the journals are requiring that authors now provide a statement of their contribution to the paper. Generally they go something like this:
1st author: conceived and designed study, collected data, performed statistical analysis, wrote and revised paper.
2nd author: collected data, revised paper
subsequent half dozen plus authors: "contributed to study design a
Re: (Score:1)
3rd author: sucked my c***
Re: (Score:2)
It's one of the things that woke me up the fact that academia has its share of scam artists and the rest are completely aware and fine with it.
Re: (Score:2)
This. I thought everyone in academia knew about this for years. It's one of the things that woke me up the fact that academia has its share of scam artists and the rest are completely aware and fine with it.
Well - academia is made up of humans, so suffers from all of the veracity issues that other humans suffer from.
In a peer group, the sluts are more ridiculed than praised. Vanity means nothing. And it isn't that the rest are fine with it. Its just not a crime, so snicker, shake your head, and get on with the work at hand.
When it backfires is when the citation slut puts their name on a paper that gets retracted. That often cures them. Oh yeah.
Re: (Score:2)
The really funny thing is that you would post something like this while employing the karmawhoring trick of replying to a random posting in the first thread so that your comment shows up on top.
Have you considered Haldol?
Re: Goodhart's law (Score:2)
Re: (Score:2)
Citation provided [jalopnik.com].
Re:Goodhart's law (Score:5, Insightful)
Also of Campbell's Law [wikipedia.org]:
Campbell's Law is why structuring education reform around high stakes testing doesn't work as well as one might reasonably anticipate. Testing is an integral part of controlling any process, and in itself is a good thing. But as soon as you try to use it to oversimplify what is an inherently complex problem, things go wrong. People start gaming the system because the system is structured as a game.
Using citation counts to rank researchers and journals is more or less the same mistake. It's something worth looking at, but you simply can't boil down a researcher's value to a single number, and if you try to you effectively gamify the system.
Re: (Score:2)
I remember when MCSE courses first came out. after the first 2 years or so, an MCSE certificate was worthless, as people were gaming the system by memorizing the books and not doing any of the actual work. so they aced the tests, but couldn't do the actual work because they had never worked on the system before.
Re: (Score:2)
"Marketing".
Re: Goodhart's law (Score:2)
Re: (Score:2)
Re: (Score:2)
That is very well put. I didn't know Goodhart either.
The citation metrics as a reputation measure leads to gaming the system not only just in how citations are collected and traded but also in what is considered worth researching and how it is published.
focussing on the excesses is a bad approach. Tightening the citations system is a bad approach. But who is going to reduce the usage of reputation measures?
I'm reminded of David Bohm who pointed out that in an experiment they were letting monkeys make painti
Re:Goodhart's law (Score:4, Interesting)
The citation metrics as a reputation measure leads to gaming the system not only just in how citations are collected and traded but also in what is considered worth researching and how it is published.
And virtually everyone involved knows exactly what the citation sluts are doing, and ridicule them - discretely of course. It means virtually nothing other than vanity.
Re: (Score:2)
In some sectors it will mean little. But when reputation metrics become a measure on which your funding depends then whether you like it or not, you're going to use them. Apart from a principled minority.
Re: (Score:1)
What are you saying!! Don't you know my invaluable scientific contributions will save the civilization in the next 200 years!
It all comes down to what kind of human being you are - honest or dishonest. Good luck!
Re: (Score:1)
"Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes."
This sounds like something from quantum mechanics. Once you observe the system closely it changes its behavior.
Re: (Score:1)
Re:I'm shocked. Not. (Score:4, Insightful)
A quote taken out of context.
The full context shows there were legitimate reasons for the papers in question to be excluded from the IPCC. [skepticalscience.com]
Re: (Score:2)
Re: (Score:3)
MBH98 was probably the first attempt at a northern hemisphere reconstruction. Improvements to the methodology have been made, but it turns out that it doesn't really matter how you combine the date. You end up with a hockey stick [blogspot.com] because it's in the data. This is one of sciences most replicated results.
Here's another bunch [realclimate.org] of reconstructions. And another [blogspot.com]. There are literally dozens of attempts to reconstruct historic temperatures using different data and different methods and they all find substantiall
Re: I'm shocked. Not. (Score:2)
Re: (Score:3)
Please, do not bring this conspiracy theory to this website. The data has been reanalyzed by many people.
Fo example, it can be downloaded here:
http://www.meteo.psu.edu/holoc... [psu.edu]
Please, do not simple believe random stuff you read on the internet
Re: (Score:2)
Re: (Score:1)
There seems to be some confusion about what all the fuss was about, w/r/t Phil Jones' use of "Mike's Nature Trick" to "hide the decline."
The problem wasn't with their analysis of the data, Uecker, and it wasn't that they got the wrong results, Layzej. The problem was that they misrepresented the results which they got, to hide the weakness of their methodology, and create the hockey stick shape of their graph. Their own data, when graphed honestly, looked nothing like a "hockey stick."
Here's the background
20 year old conspiracy theories. (Score:2)
Re: (Score:2)
You live in your google bubble believing random stuff people (posted on the internet. This exactly similar to how anti-vaxxers are tapped in their conspiracy theory that all scientists conspired to hide the truth. I feel pity for you.
Re: (Score:2)
Your comment linked to exactly two scientific papers, the ones by Mann you attack, and a lot of random links on the internet. Think a bit about this.
Re: (Score:2)
The WMO is not supporting your argument, you just mentioned that they used Mann's graph. So I do not see any argument here in your favor. The "Center for the Study of Carbon Dioxide and Global Change" is a well known climate misinformation site founded by coal industry:
https://www.theguardian.com/en... [theguardian.com]
I don't know why you cite the Pages 2K Network, the findings are in agreement with Mann: "Recent warming reversed the long-term cooling; during the period AD 1971–2000, the area-weighted average reconstr
Re: (Score:2)
If you really want to learn something, maybe you read the report by the National Research Council:
National Research Council. 2006. Surface Temperature Reconstructions for the Last 2,000 Years. Washington, DC: The National Academies Press. https://doi.org/10.17226/11676 [doi.org].
It can be download for free:
https://www.nap.edu/catalog/11... [nap.edu]
(BTW: Robert Muller was one of the reviewers).
Re: I'm shocked. Not. (Score:4, Informative)
Re: (Score:1)
Mann lost a lawsuit because he refused to cough up his data and methods
So he published his results without revealing the data he used? You can do that in 'scientific community' ?
"Intredasting" Sounds more like 'old boy' network to me than 'scientific community'.
Re: (Score:1)
These are not facts. This is some stuff you read on the internet and decided you believe for reasons that are unclear to me.
Re: I'm shocked. Not. (Score:2)
Re: (Score:2)
You know this how?
Re: I'm shocked. Not. (Score:2)
Re: (Score:2)
I gave a link to his data above. Here is again: http://www.meteo.psu.edu/holoc... [psu.edu]
So do you think it is plausible that he lost this case because he "did not cough up his data"? I know this is a claim made now by every denier website. But do you have a reliable source?
Re: I'm shocked. Not. (Score:2)
Re: (Score:3)
This is nonsense.For example, the National Academy of Sciences looked into it. Here is nature reporting about it https://www.nature.com/article... [nature.com]
Quote:
"The academy essentially upholds Mann's findings, although the panel concluded that systematic
uncertainties in climate records from before 1600 were not communicated as clearly as they could
have been. The NAS also confirmed some problems with the statistics. But the mistakes had a
relatively minor impact on the overall finding, says Peter Bloomfield, a statis
Re: (Score:2)
I don't think it's nonsense at all. Your reply is too polarized, and Nature is trying to save Mann. In other cases the result would be : article is faulty, start over. I think the situation is fairly simple: the mere fact that we have to go back in time makes climate data intrinsically problematic , contradictory and frustrating. Conclusions which are intuitively straightforward are difficult to reach with rigid methodology because you never can work with that set of 'really clean data'. So people loosen th
Re: (Score:2)
Nature is only reporting about it. This was the a report by the National Research Council made at the request of the House of Representatives Committee on Science. Many people looked into this. The science is sound. Of course, the data was not clean. How could it? Mann et al. fully acknowledged these limitations in their paper which is still considered a milestone. The claim that it had be debunked is complete and utter bullshit.
Re: (Score:2)
he'd be working at Home Depot now
Dick's Sporting Goods.
In the hockey department.
Re: (Score:2)
Your link simply says "they weren't included" - not that there were legitimate reasons to exclude them. Actual, published papers were excluded because - why? No reason is listed anywhere. It seems that because the peer-reviewed, published papers ran counter to the beliefs of the IPCC authors, they chose to ignore it. No refutation was provided, just "we don't want it".
Part of science is explaining why your model/theory is correct, provide data that backs up your conclusions, and actually address (rather
Re: (Score:2)
Part of science is explaining why your model/theory is correct, provide data that backs up your conclusions, and actually address (rather than ignore) other theories and data that run counter to yours. Simply ignoring it isn't science - that's religion.
Exactly.
"We have 25 or so years invested in the work. Why should I make the data available to you, when your aim is to try and find something wrong with it." - Phil Jones (again, showing he has no idea how science is supposed to work)
Re: (Score:2)
It seems the contrarians they were mocking are largely ignored in the literature as well. They don't make it on the list of 100,000 most cited scientists. Jones and Mann are listed and score below average for self citation.
Those contrarians that do make it on this list, (eg Tol, "Richard S.J." or "Pielke Sr., Roger A.") are on the high end of average for self citation.
And here I was thinking their secret was that (Score:2)
in most press conferences, while announcing their findings, that they weren't wearing pants.
Trump encourages attacking Science!!! (Score:1, Flamebait)
Such unprecedented attack on the credibility of Scientists and Science weren't possible before the Trump era. Why do you hate Science?!
Lrnu, n shaal gebyy, vf abg vg?
Re: (Score:1)
Um hmm... ;-)
Seriously, though, I know you realize this, but many people don't: criticizing scientific corruption is not attacking science, it is defending science.
Trump has little to do with it, anyhow. Dr. Ioannidis is most famous for this seminal paper, published eleven years before President Trump was elected:
Why Most Published Research Findings Are False [plosmedicine.org], by John P. A. Ioannidis, 2005. PLoS Med 2(8): e124. doi:10.1371/journal.pmed.0020124.
Yes, you read that right: the peer-reviewed literature s
Re: (Score:2)
That Loannidis article appears to claim we have a 'Millikan epidemic' , measurements suffering from strong historic bias.
One should add that the more you're able to verify yourself (because you have the time and the competence) the less you have to rely on trust. For non scientists trust in science is a reasonable approach. For non -climate scientists trust in climate scientists is a reasonable approach. It's not guaranteed to work but it is reasonable. The polarized response to switch all trust to opponent
Re: (Score:2)
WHAT! We can't blame Trump! I thought we blamed him for everything?
Self-Cites, Self-Certs (including airworthiness) (Score:1)
Lost trust...
Good luck!
Science is often wrong... (Score:1, Troll)
The reason why Science is often wrong is because of the fallible, greedy, agenda driven, political, socially and economically compromised humans that are part of the community.
When we attach the word professional to someone... we should only be saying this persons knowledge is greater, but too often people use it for gatekeeping where people without a diploma or "professional" experience are expected to only listen and not be able to challenge the professional.
I have met several professionals in multiple fi
Re:Science is often wrong... (Score:4, Insightful)
While scientists may be wrong very often, Scientific knowledge is actually extremely reliable. The problem is that it may be difficult for non-scientists to know what is established science and what is not. If you get your information about science only from the headlines in the news who prematurely report about every questionable study if they think it might interest their reader (so there is a selection bias for unlikely result), and then often misunderstand a lot things, then it is easy to get the impression that science is a complete mess. But is often very obvious to scientists what is reliable knowledge and what is not. Good sources for information are consensus statements published by big scientific societies.
Citation Scores Were Always Worthless (Score:2, Insightful)
Citation scores are the equivalent of of saying "trust me" - the exact opposite of science. They were introduced for petty administrators to use for determining pay offers to staff and naturally that means people who want paid manipulate them.
Re: (Score:1)
"People who want paid"? That's not even English.
This is the world wide web. How self-centered, narcissistic, and narrow-minded are you to expect English to be everybody's first language?
I guess when you are incapable of countering a position with logic and facts, personal attacks based on grammar is one of the few ways someone as fucked up as you can reply.
Re: (Score:2)
That's not even English.
Apparently you have never been to Pittsburgh - that sentence is perfectly correct there.
Re: (Score:2)
"People who want paid"? That's not even English.
Yes it is. In England, anyway. Your local dialect may vary.
Re: (Score:2)
Citation scores are not the equivalent of saying "trust me". They are the equivalent of others (!) saying "there is something interesting". This is why you might want to consider removal of the self-citations for the purpose of rankings. Although I do not think it matters too much.
Re: (Score:2)
You get cited even when the citer is saying you're a fucking idiot.
Re: (Score:2)
Yes, this is one way to get famous.
Possibly exaggerated in terms of severity/meaning (Score:4, Insightful)
I'm not sure that this level of self-citation is a cause for concern by itself, and I'm especially unconvinced given that they are including papers of people who are co-authors. In very narrow fields, or in areas where a small number of people are doing a lot of the work, this doesn't strike me as at all strange. Another factor might be that some people are more willing to list tangentially related works in general which also includes some of their own works; there's a lot of variation in how much people think how closely connected something should be to be cited.
At the risk of being a little egotistical, I'm going to look at some of my own papers. I had a paper which improved a certain technical result about odd perfect numbers http://math.colgate.edu/~integers/vol18.html [colgate.edu] . A followup paper to that one is currently under review, where I improve on the bounds even further. That's a self-citation, but it would be extremely odd not to cite that. Similarly, I had a paper which was just recently published about the 2nd largest prime factor of an odd perfect number. I'm working with two other authors now on a paper which does something similar with the third largest prime factor. Not citing that would be very odd.
I'm particularly concerned about the discussion of groups of people engaging in citations. How does one distinguish between groups of people citing each other because they are engaging in mutual promotion and how that they are all working in the same, possibly narrow area? Similarly, the article acknowledges that the inclusion of coauthors in the self-citation index has potential issues.
I'm more comfortable with Ioannidis acknowledging in the article that this isn't a perfect metric and that it may be a reason to look closer more than anything else, and not a specific cause for concern.
One thing the article does not discuss that would be worth discussing is whether the higher percentage of self-citation in some countries is due in part to language and cultural barriers. The authors may be writing very good works that aren't getting noticed outside their own countries, so the percentage of self-citations ends up looking higher.
Re:Possibly exaggerated in terms of severity/meani (Score:4, Insightful)
Self-citing is completely normal and acceptable (and as you point out often required) and the paper by Ioannidis does not state anything else. Tthe slashdot headline and summary is again bit misleading.
Re: (Score:1)
Self-citation is a bad thing to include when using citations to measure the importance or impact of a piece of research.
As the GP pointed out, it would be very odd for him not NOT cite his own earlier work when improving upon it. But if even if he writes 100 more papers, each improving on the previous, it means nothing if no one else ever cites any of his papers.
Of course, citation counts is a terrible way to measure important or impact of research papers in the first place, but it is the metric that is us
The problem went beyond circular citations when (Score:3)
Re: (Score:2)
Organized crime basically. Scientists are no different than other groups of populace so don't be surprised to get same behavioral patterns.
Scientists are NO DIFFERENT from other groups of people? Then why hang out on Slashdot when you could be finding like minds on 4Chan?
Re: (Score:2)
The program stated that about 48% of all scientific papers involved some sort of cheating.
I knew that when it started talking about whistle blowers and scientists living high on the hog with government contracts it was going to be about Global Warming. It couldn't be ANYTHING else, nope, every scientist that wanted to cheat just for some reason said; "I want to study clouds mom and dad."
Re: (Score:2)
This issue was investigated by several committees which looked it this in detail. They all came to the conclusion that there was no misconduct. Are you saying they are part of a huge conspiracy?
Bad metric for quality of papers (Score:3)
Citations are fine. Self-citations are fine - nothing wrong with citing your own work if it is the best, or at least most convenient reference. The problem comes from using citations to somehow measure the quality / importance of a publication.
Some survey publications may be extremely valuable as a reference but have nothing really new in them. Other times the very first paper in a field may represent a truly new discovery, but later papers that fill in details are better references for future work.
I don't think there is an objective way to measure the performance of a scientist without having a deep knowledge of the field. Imagine trying to rank art, music or literature based on the number of viewers.