Research Findings That Are Probably Wrong Cited Far More Than Robust Ones, Study Finds (theguardian.com) 35
Scientific research findings that are probably wrong gain far more attention than robust results, according to academics who suspect that the bar for publication may be lower for papers with grabbier conclusions. From a report: Studies in top science, psychology and economics journals that fail to hold up when others repeat them are cited, on average, more than 100 times as often in follow-up papers than work that stands the test of time. The finding -- which is itself not exempt from the need for scrutiny -- has led the authors to suspect that more interesting papers are waved through more easily by reviewers and journal editors and, once published, attract more attention.
[...] The study in Science Advances is the latest to highlight the "replication crisis" where results, mostly in social science and medicine, fail to hold up when other researchers try to repeat experiments. Following an influential paper in 2005 titled Why most published research findings are false, three major projects have found replication rates as low as 39% in psychology journals, 61% in economics journals, and 62% in social science studies published in the Nature and Science, two of the most prestigious journals in the world.
[...] The study in Science Advances is the latest to highlight the "replication crisis" where results, mostly in social science and medicine, fail to hold up when other researchers try to repeat experiments. Following an influential paper in 2005 titled Why most published research findings are false, three major projects have found replication rates as low as 39% in psychology journals, 61% in economics journals, and 62% in social science studies published in the Nature and Science, two of the most prestigious journals in the world.
Exciting news! (Score:3)
Re:Exciting news! (Score:4, Insightful)
Another possibility is that more sensational results are more likely to be tested by replication.
Sometimes that replication will fail.
Mundane results may not be more reliable, just less likely to worth replicating.
Re:Exciting news! (Score:5, Interesting)
Incorrect papers are cited by correct papers to prove they are incorrect,
The vast majority of correct, but uncontroversial papers tend not to be cited because they are ground-breaking enough to need to be.
Citations have always been a terrible metric for measuring academic output for many, many reasons. However, they are very easy to measure, so university administrators and the type of public servants who run research funds love to use them because they don't require much thought.
Re:Exciting news! (Score:4, Informative)
Incorrect papers are also cited because research programs inspired by an exciting paper can go on (and keep producing false positives in small-scale studies) for years after they've been definitively disproven.
A couple of examples I know of are intranasal oxytocin research and research on the 5-HTTLPR serotonin transporter polymorphism. There are still researchers spraying oxytocin up the noses of autistic kids, and psychology papers testing for the serotonin polymorphism.
Part of the reason for this seems to be that the initial exciting result "escapes" from its primary field and into other fields, while the large-scale, well-conducted studies which find null results are talked about within the primary field, but that conversation doesn't cross disciplinary boundaries nearly as easily. Psychology researchers and GWAS researchers aren't going to conferences together, aren't reviewing each other's papers, aren't reading the same journals, aren't on the same rumour circuits. So there are occasional random injections from one discipline into another, but not a steady flow of information.
Re: (Score:1)
Re: (Score:2)
Re: (Score:2)
Incorrect papers are cited by correct papers to prove they are incorrect,
Disclaimer: I can only speak to computer science-ish areas.
Not they are not. I have never seen a research paper that directly addressed incorrectness of prior work. You always work around prior work by positioning yourself as a better/different solution.
Re: (Score:2)
Another possibility is that more sensational results are more likely to be tested by replication.
Sometimes that replication will fail.
Mundane results may not be more reliable, just less likely to worth replicating.
I wish I could believe you, but a stupid, gullible society that thrives on clickbait bullshit, won't even bother with replication. Or even validation.
Why do you think THIS study, was justified.
Re: (Score:1)
I wish I could believe you, but a stupid, gullible society that thrives on clickbait bullshit, won't even bother with replication. Or even validation.
Or emancipation or anticipation or procrastination either. In fact all they really care about is constipation and procreation, although not at the same time. That would be gross.
Re: (Score:2)
Sensationalism gets more attention than reliability!
[robust citation needed]
Re: (Score:2)
If you look into specific egregious examples, like Brian Wansink's genuine misconduct at Cornell's Brand and Food Lab, "sensationalism" doesn't begin to describe the phenomenon. Wansink was remarkably creative at ginning up provocative ideas, and came up with astonishingly clever lab apparatus, like "bottomless bowls" that surreptitiously refilled with soup without subjects realizing it.
He was unquestionably wildly creative, the problem was he was not someone to let a lack of a result let a clever experim
Re: (Score:1)
If you look into specific egregious examples, like Brian Wansink's genuine misconduct at Cornell's Brand and Food Lab, "sensationalism" doesn't begin to describe the phenomenon.
Hi There - thanks for the comment, it was a very interesting read. If you don’t mind, though, I do have one quick question. What the hairy fuck are you talking about?
Re: (Score:1)
Damn climate-change-denying mods.
Recursive article is recursive (Score:2)
Reminds me of the famous XKCD on correlation
Re: Well, duuh! (Score:2)
And your basis for concluding that they are crap is what?
50 years of dark matter mathematics (Score:2)
And not one reproducible result, except "we haven't measured all the matter". Not a single verifiable indication that the matter is non-baryoni. Note especially that black holes do *not* have to be non-baryonic, a large enough black hole can be made from enough quite ordinary matter.
Re: (Score:3, Informative)
Re: (Score:2)
That is precisely my point. Unfortunately, there are entire groups of physicists who proclaim that the undetected dark matter with its measured gravitational effects is proof of extraordinary and otherwise undetectable forms of matter, sources of gravitational mass which are neither baryonic nor electro-magnetic. I'm afraid it is the "luminiferous aether" of our generation, and it's unnecessary since there are several other candidates, such as interstellar and intergalactic bodies. With planetary densities
Re: (Score:2)
Two comments:
1) Your preferred candidate model is at odds with the _data_. Observational data is what you have to test a model against, and your preferred model fails spectacularly: it can't explain the baryon acoustic oscillation spectrum, or the large scale intergalactic structure, or the polarization spectrum of the microwave background, or the accelerating cosmic expansion rate. That's _why_ the current concordance model became accepted as most probable: it explains the data ... even data that were n
Re: (Score:3)
The "Lambda-CDM" model starts out with an assumption of Lambda, of "dark energy". As a proof of the existence of exotic so-called "dark matter", it's circular reasoning. As best I can tell, it so far has no predictive power and is not falsifiable. I dislike being this rude, but it's outright invention of equations with constants invented to match the data. It has too many invented constants and so far lacks falsifiability.
The luminiferous aether broke down as a model pretty quickly, especially when the spee
The sweet irony would be... (Score:1)
Re: (Score:2)
Not as ironic as all that "quality" science ending up on Sci-hub. Doubly ironic because P2P (Popularity Protocol) would reinforce that popularity instead of the quality.
Research "confirming" what is accepted... (Score:1)
Research "confirming" what is accepted is often a form of confirmation bias. Thus the questionable sources, or methodology are at worst overlooked or at best passed over as correct. Or as automated gobbledygook scientific research paper generators demonstrate, if it sounds "scientific" enough then it is accepted and published. My favorite source is when referencing a "future" research paper that is "in preparation."
The "Simpsons" had an episode where Troy McClure was doing a high school film, but then they
Scientists too... (Score:2)
Now we wait... (Score:2)
The Science (Score:2)
But it's "the science." It's always right. Y'all are a bunch of conspiracy theorists.