Study Shows One Third of All Studies Are Nonsense 391
SydShamino writes "CNN has a report on new research to confirm claims made in initial, well-publicized studies. According to the new study, about a third of all major studies from the last 15 years were subsequently shown to be inaccurate or overblown. The study abstract is available."
The topic is stated toooo broadly! (Score:3, Interesting)
it isn't, really
It is about the effectiveness of interventions... if you skipped over it, its worth a perusal to a skim... at the very least... but it would seem to me that the whole thing has lead to almost no positive conclusion itself... with 44% of the experiments being replications and 24% unchallenged... the 66% really don't seem to have much value...
Ahhh, academic research... only there can you get paid well to tell us absolutely nothing...
Re:more like (Score:3, Interesting)
Re:nice (Score:5, Interesting)
"This is not a very good time for something like this to happen."
So my question is this: when is a good time for an airplane full of people to crash into a residential neighborhood? This guy should designate a day for us so we can make sure all the airlines and pilots know when the good day for crashing is. Morons.
Re:Irony meter! (Score:4, Interesting)
Re:Obviously flawed (Score:5, Interesting)
Think about it: they are measuring highly cited studies that get a "stronger" result than other subsequent studies. However, they simply say "stronger"(at least in the abstract). Whenever you measure something other than by a census, you take a sample. Therefore, as anyone familiar with elementary statistics should know, you have sampling variation. Researchers then usually appeal to the 'central limit theorem' to assume that the mean comes from a normal distribution, and then, knowing the distribution of the mean, make a statement like "We are 95% confident that x is between a and b". The later experiment will make a similar such claim.
Assume the experiment to measure x was performed correctly and identically both times, and there is no change in the effect with respect to time. Each time the experiment is performed, a different mean value of x will be obtained due to sampling error. Since both measurements of x come from an identically distributed population, by symmetry 50% of the time the earlier one will be "stronger" than the later one shows. However, not all highly cited studies are repeated after publication, and not all "me too" studies are published, hence the figure less than 50%.
Clearly, what they should be doing is comparing the confidence intervals, and looking for a statistically percentage of studies which do not overlap. This then would be relevant, as it would show us about non-sampling errors, rather than sampling errors, such as experiments designed or performed or interpreted incorrectly.
AP Statistics (Score:2, Interesting)
I am willing to bet that the CNN study is correct in it's assumption that most studies are incorrect.
This study is actually alarming (Score:5, Interesting)
This is about the accuracy of clinical trial research. This is not about market research studies in the latest clothes fashions. Medicine is an extremely lucrative and risky field -- being associated with the group that pushes through the next Viagra can ensure that your family becomes the next Rockefellers. Your only opposition is the FDA (and the politicians that influence it, which are always hungry for money, which you have lots of).
There is a tremendous amount of pressure on pharmaceutical researchers to produce favorable results. Let's say that you're a new, idealistic researcher who runs some tests on a new drug that your employer wants to market. Your tests show that our drug produces an increased rate of cancer? Well, been nice having you work here...bye. Bob down the hall has consistently gotten us much better results to feed to the FDA for approval. We really don't know how or why he gets better results, but he's definitely the man we want on the job. Sure, maybe twenty years down the road there will be some complaining, but *we didn't know*...*we did all our due dilligence and somehow our results just wound up showing that our drug was okay*.
And even the more innocent "conclusive results" become suspect. A pharmaceutical doesn't want "inconclusive results", where "further tests are recommended". They have a bloody lifetime on the product ticking away, and a competition breathing down their neck. They want some scientist to sign off on this thing with a nice firm "Okay" or "Not Okay", or else what are they paying the guy for? He's not here to do ivory tower work -- he's here to serve the company, which is in the business of extracting savings from aging and achy baby boomers and subsidies paid for by their tax-paying children.
What is being said is that a full third of examined clinical trials were essentially horseshit. This is really not a laughing matter.
this excellent piece (of liberal propaganda) (Score:1, Interesting)
The "Blinded by Science" article, on the other hand, starts off with an interesting anecdote about what is presented as unreasonable balance, then goes into a rant on global warming that constitutes the bulk of the piece. Furthermore, it focuses exclusively on conservative positions as ridiculous.
It's a standard piece of stealth propaganda. It takes some of the worst excesses of conservative pseudoscience and fringe science and quietly slips opposition to the extremely political and not-especially-scientific Kyoto Protocol into the same category, and for good measure implicitly equates the scientific consensus that human activity is causing some measure of global warming with a consensus that it is causing a global warming emergency, and with mainstream support for extreme claims such as a quarter of all species being wiped out in the next 50 years.
But the initial bit, on breast cancer... are we to simply assume that those wacky conservative legislators have done it again, as he suggests? You don't have to search hard on the internet to find an opposing piece. Here's the first thing I found on google: http://www.leaderu.com/ftissues/ft9705/opinion/br
Looking around more, there seems to be a genuine, legitimate controversy on this point. Studies continue to be published in reputable journals suggesting a link. This is a terrible example of false balance.
Just because the law was written trying to scare girls away from abortions (as I'm willing to assume) rather than from a sincere concern for their health is no excuse for inaccurate reporting about research into the health concerns.
Politics and money drives the FUD. (Score:4, Interesting)
Surprised? (Score:2, Interesting)
Why do scientists jeopardize there careers by falsifying evidence and studies? Because they are human. Some do this unintentionally by subconsciously ignoring evidence that is contrary to their preconceived hypothesis. Others do this intentionally. They do this because if their hypothesis is found wanting; they will lose prestige and/or funding. Or sometimes the evidence points to concepts that they are unable or unwilling to understand or acknowledge as true. Every scientist has preconceived beliefs on what they will discover in a given study; the good ones do not let these beliefs get in the way of the evidence.
History is littered with studies and discoveries that were later shown to be hoaxes. The Piltdown Man is a very famous one; the supposed early human fossil was created to promote a certain view of human evolution. Another recent hoax was the Chinese "feathered dinosaur" fossils that were heavily promoted by the National Geographic to be ancestral to birds (circa 1996). In fact, National Geographic created an evolutionary timeline based on these fossils and presented it as fact. Not only were these supposed fossils later exposed as elaborate hoaxes, it was revealed that National Geographic had a major lapses in scientific and journalistic ethics. First off, the fossils were stolen and smuggled out of China; secondly they were not verified as being authentic. It was shown that National Geographic knew this and still published the story. Why? Because National Geographic is a big proponent of the theropods to birds view of bird evolution, so any evidence that fit this preconceived belief was accepted and not questioned. If on the other hand the evidence had supported the tree-reptiles to birds view, the fossils would have been very critically examined and rejected.
Of course this is not limited just to archeology; many other branches of science suffer from the same problem. But I can somewhat sympathize with these researchers. Imagine arguing your whole life that your hypothesis is correct, all the evidence for the past 50 years points to your conclusion, and then suddenly new evidence pops up that invalidates your life's work. Would you not be tempted to suppress the evidence? Like I said, scientists are human just like us, and while I do not condone suppressing contrary evidence, I can understand why someone would do just that. On the upside, history shows that evidence cannot be suppressed indefinitely, it will always surface. It may take several decades, as was the case with the plate tectonics theory, but evidence cannot be silenced forever.