Majority of Landmark Cancer Studies Cannot Be Replicated 233
New submitter Beeftopia writes with perhaps distressing news about cancer research. From the article: "During a decade as head of global cancer research at Amgen, C. Glenn Begley identified 53 'landmark' publications — papers in top journals, from reputable labs — for his team to reproduce. Begley sought to double-check the findings before trying to build on them for drug development. Result: 47 of the 53 could not be replicated. He described his findings in a commentary piece published on Wednesday in the journal Nature (paywalled) . ... But they and others fear the phenomenon is the product of a skewed system of incentives that has academics cutting corners to further their careers."
As is the fashion at Nature, you can only read the actual article if you are a subscriber or want to fork over $32. Anyone with access care to provide more insight?
Update: 04/06 14:00 GMT by U L : Naffer pointed us toward informative commentary in Pipeline. Thanks!
Grants-whores and publicists in academia?!?!? (Score:5, Insightful)
But they and others fear the phenomenon is the product of a skewed system of incentives that has academics cutting corners to further their careers.
As I've said before [slashdot.org], back when I was in academia, there were always grant-whores and academics more interested in their own interests than science around. Too many people have come to treat science with a reverence more appropriate to a religion than a system of knowledge and discovery, however. And so when I point out that there are scientists out there willing to cook the numbers, exaggerate, play to politics and/or public opinion, etc. I inevitably run into those who say "Science wouldn't allow that" (like my friend who's still in the field). But science is only as good as the people practicing it. And, in any field, there are always those willing to put their own personal interests ahead of the greater good.
I just hope this doesn't cast a shadow over those out there who *are* doing good work and *are* trying to do honest work. Sadly, some of the best researchers out there are the ones who make the least noise, get the least attention, get the least grants, and are least likely to get tenure.
Re:Grants-whores and publicists in academia?!?!? (Score:5, Insightful)
Those of us in other fields of science tend to hold up biomed as an example of how not to run a science. They tend to have a shoddy idea of experiment design and statistics. Same way when I was a student it was always the premeds who did all the cheating.
Absolute power corrupts absolutely. (Score:5, Insightful)
There is no greater "absolute power" than knowing that if you say or write something that others will like, they will pay you lots of money and make you famous.
It's not that money corrupts. Money is not the root of all evil; the full quote is "the love of money is the root of all evil." When our society decided that money was more important than truth, we surrendered truth to the void.
A research scientist thinks about his day. He can slightly fudge his cancer study, make big headlines, get a ton of grant money and get appointed chair at the university. Or he can go down the long hall to his boss and say, "Nope, this one didn't work either, and while I'd like to start a religion based on false hope, this isn't the false hope you're looking for." If he does that, he can then watch one of his subordinates fudge the cancer study, make big headlines, and be his boss at the same time next year.
Which choice would you make?
Re:Grants-whores and publicists in academia?!?!? (Score:5, Insightful)
But they and others fear the phenomenon is the product of a skewed system of incentives that has academics cutting corners to further their careers.
A few years back one of the USA's leading medical journals changed their rules to allow doctors who are receiving money from pharmaceuticals to publish reviews of the drugs sold by those same pharmaceuticals. We may have a problem that runs deeper than "cutting corners".
Re:Grants-whores and publicists in academia?!?!? (Score:4, Insightful)
Re:No Surprise Here (Score:5, Insightful)
My second thought is "Hmmm, academics/scientists skewing results for the sake of their own careers. Global warming, anyone?"
Your second thought is completely off because every single time someone actually tried to replicate global warming research, they DID get the same results. Unlike in the case of those medical papers TFA is about.
Re:Grants-whores and publicists in academia?!?!? (Score:4, Insightful)
Failure to replicate an experiment is not certain indication the the original experiment was flawed or manipulated. But it does wink suggestively in that direction.
Re:Grants-whores and publicists in academia?!?!? (Score:5, Insightful)
I've actually been on a ranting spree the past couple of days due to terrible journal manuscript submissions I've had to review recently. I can't tell you the number of times I've read a submission that was outright published in another periodical. Many foreign submitters don't understand the concept of plagiarism, let along self plagiarism. These "scientists" are ranked and compensated by the number of publications they produce, so they publish one piece of research and try to pass it off in as many periodicals as possible, essentially representing old research as brand new. This compensation system has obscured the true purpose of publication: what was once a means to disseminate your work to the general population is now a means to get you and your lab more money.
I take my responsibility as a reviewer very seriously; the job of a scientist is not only to create new research, but to critique and evaluate the research of others. But many academics who have been in the field longer than I approach reviewing as a chore, and only focus on half of the interesting part of their job, the research. I don't know how many of these terrible publications slip through the cracks due to lazy reviewers, but I'm sure it's more than I'm comfortable to admit to myself.
Re:Grants-whores and publicists in academia?!?!? (Score:3, Insightful)
The private industry has a strong incentive to downplay risks with their product. Big tobacco, big oil are two examples.
Re:Absolute power corrupts absolutely. (Score:5, Insightful)
No, its a little worse than that. The honest guy doesn't get tenure, and is eventually fired. The dishonest guy remains a "scientist" for life. So in the steady state, there will be many, many more dishonest "scientists" than honest ones.
Re:Grants-whores and publicists in academia?!?!? (Score:4, Insightful)
Yes, but unless that system is made as efficient as possible, it can take a very long time to correct itself. Eugenics [wikipedia.org] is the classic example. Sure, it was eventually shown to be so much junk science, but not before it contributed to millions being killed/lobotomized/institutionalized. Even though there were skeptics of it almost from the beginning, the biology and medical fields did a piss-poor job at self-correcting, and people suffered for decades after this should have been laughed away as humbug.
Simply saying "Well, it will eventually sort itself out" is not an excuse to avoid reform.
Re:Grants-whores and publicists in academia?!?!? (Score:5, Insightful)
These "scientists" are ranked and compensated by the number of publications they produce, so they publish one piece of research and try to pass it off in as many periodicals as possible, essentially representing old research as brand new.
This problem has also created backlash that affects genuine researchers. My adviser had been working on some new research and was invited to present at a conference. So he wrote up his work in-progress (limited to 4-pages), and presented there. When the work was completed he tried to submit it to a journal, and one of the reviewers rejected it because it was "just a copy of this prior work". This is despite the fact that the 12-page journal paper went into far more detail, provided proof for what were conjectures at the time of the conference, and corrected significant errors in that preliminary work. So now the only scientific record of this work is an incomplete incorrect account.
Re:Absolute power corrupts absolutely. (Score:5, Insightful)
Or he can go down the long hall to his boss and say, "Nope, this one didn't work either, and while I'd like to start a religion based on false hope, this isn't the false hope you're looking for."
And that's what *really* pissed me off about academia. Guys like that never get tenure, never get thanked. With so many of the people I worked with, "hypothesis" was synonymous with "foregone conclusion." The standard practice was to come up with your hypothesis, cook up a bunch of data to support it (dismissing any evidence that contradicted it with a little intellectual sleight-of-hand), publish, and then get your promotions and tenure. The guys who treated their hypotheses as ACTUAL hypotheses (that they might actually find to be wrong) were treated like bad researchers, when in fact, they were the *good* researchers. With so many people cooking the numbers, it began to be assumed that if your hypotheses weren't always proven right, it meant you were somehow flawed.
Re:Grants-whores and publicists in academia?!?!? (Score:4, Insightful)
We've long since reached the point where "breeding" has a lot less to do with anything than environment. The consciousness and mind are what makes humans different from any other animal. A dog, in the absence of training, will behave similarly to a trained dog purely by instinct. A human without education of any sort raised by animals in the wild is a completely different creature to one educated by the best minds on earth. The advantages, if they even exist, of genetic selection by breeding are demonstrably too insignificant to justify any cost, let alone the monstrous cost that eugenics brings.