Disappointing Cancer Study Results Go Unreported 77
An anonymous reader writes "Science News reports on a new study showing that most cancer drug trial results are never published, probably leaving patients vulnerable to cocktails that have already been shown to be dangerous or useless."
First Post ... sadly (Score:1, Insightful)
Where's the surprise in this? No news here.
Re:First Post ... sadly (Score:5, Insightful)
Second, this is new because someone's done the actual study and shown the degree to which studies don't go reported. Even if only half of the unreported studies were because of poor results, that's enough to skew things very, very badly.
Anyone doing this should get put in jail for a long, long time. It may not be fraud in the sense that they're publishing fraudulent results, but by not publishing results they're creating fraudulent overall data, with possibly deadly results. This needs to stop.
Re: (Score:2)
Re: (Score:2)
Re:First Post ... sadly (Score:4, Informative)
Anyone doing this should get put in jail for a long, long time. It may not be fraud in the sense that they're publishing fraudulent results, but by not publishing results they're creating fraudulent overall data, with possibly deadly results. This needs to stop.
That's a bit simplistic - often nobody is directly responsible for not publishing results, it comes about because major medical journals are not interested in pubishing negative findings. There are of course exceptions, when the negative findings come as a big surprise or there is a lot of anticipation about them, like the reports of the Alzheimer's disease vaccination not working earlier this year.
The medical journals themselves know this is a problem, but they aren't sure what to do about it. One suggestion has been the journals should make a 'commitment to publish' for a trial based on its design and importance, and then publish the findings whatever they are, but that would of course mean they'd be publishing a lot less interesting stuff.
Also scientists can't be bothered to write up negative findings, because there are proportionally so many more of them than positive ones.
There is a problem of pharma companies suppressing bad findings or writing misleading journal articles, but that's a wholly independant issue and should be dealt with by the schemes in develpment for enforcing results to be left in public databases after the trials are finishsed. The problem with that is that not many GPs will be searching through these databases
The real solution is probably for doctors not to rely on scientific journals for information (which they often don't really understand), but to wait for advice from bodies like the National Institute for Clinical Excellence [nice.org.uk] or the Cochrane Collaboration [cochrane.org] who do very thorough searches and synthesis.
Plus. (Score:4, Interesting)
There's a wide assumption that the researchers themselves really want to publish all results.
Unfortunately, as in almost the entire field of "science" nowadays, it's not the case.
Researchers themselves have a tendency to hide failures - given that most experiments result in failure, they tend to focus on reporting the ones reporting success.
This use of time simply makes most sense - they don't have the time to report all the failures, and reports of failures not as valuable as reports of success only makes it worse - think about what kind of views your peers will have towards you if most of your publications are negative results.
Sadly, this thinking is parasitic and is very prevalent across all research fields.
Journals are very selective given the limited number of pages they have. If I were a journal, I'd pick reports of success first. It's the evil of centrally-controlled publication, and the mindset that, "if a research is of any good value, it must appear on some journal".
Granted, peer review is a good thing, but there must be a way to give researches credibility without getting published on some journal.
Compounded with big-pharma-sourced funding with very fine strings attached...we have a really screwed up system.
Re: (Score:1)
you're dangerously wrong (Score:2, Interesting)
Even if only half of the unreported studies were because of poor results, that's enough to skew things very, very badly.
The basic idea is that you should only use drugs or drug combinations for which there is evidence that they work and are not harmful. If there's nothing published, don't use it!
but by not publishing results they're creating fraudulent overall data, with possibly deadly results. This needs to stop.
There's nothing "fraudulent" about it. Studies often fail for many reasons completely unrelat
Re: (Score:2)
Scientific experiments are usually one-sided: a positive result tells you something, a negative result tells you nothing.
That is soooooo wrong it hurts.
Re: (Score:1, Offtopic)
Funny. On a "quiet day", a FP attempt gets fourth.
Negative results (Score:3, Funny)
This sounds reasonable. From now on I'll be emailing my pals every time I don't get laid.
Re:Negative results (Score:5, Funny)
It would be a lot more efficient if you just had a cron job that sent out an email every night.
Re:Negative results (Score:5, Informative)
Whoever modded that down as overrated clearly didn't know enough to judge. You don't publish negative results because of the time and effort it takes to write it up.
The reasons vary for different fields. A lot of times, researchers don't bother doing proper controls until after the experiment works or doesn't. If the trial run works, you do the controls afterward to verify your result was real, if you don't get the result you expect you might try it again, doing some troubleshooting, but at some point you have to make a choice between a control that would be particularly onerous or expensive, or giving up on the experiment entirely. If you get a negative result that you're not interested in, you generally don't do the controls to prove to others it was a valid result because you don't care and have better things to do. But that's what you would have to do to publish it.
A negative but true result can also be even more difficult to prove than a positive result would be. If you are expecting one protein to interact with another one, and you get no result, it could be that they are and your test just isn't working. If you do the experiment a different way and still show no interaction that makes it a little more convincing, but doesn't prove that both systems are working. You can't say for sure they don't interact in real cells.
In clinical trials you could think of additional reasons why someone would not care to publish the negative results. The most obvious is that the drug company doesn't want to make it known that they're working on drug X. Not sure how that works, but you could imagine that they might have to patent it to keep others from using it, and then the clock on the patent starts before they actually get it working. They could spend 5 years refining it before it actually works, then more years before it gets to market, and they only have a few years before the patent runs out. If they don't patent it and aren't sure it's a complete dead-end, another company might take the results and make a working drug from it, effectively stealing the expensive work to get up to that point.
Not a lawyer or an expert on the pharmecutical industry obviously, but publishing means making it known, and they're only going to do that if they're sure they're done with it, if then.
It makes sense that they're not going to be published, and while it's less than ideal, I think it would be worse to force the pharmecuticals to publish negative results of trials. If you make the clinical trial phase that risky, companies would be more reluctant to develop new drugs that haven't already proven effective, and advances in cancer treatment would slow.
Re:Negative results (Score:5, Informative)
The most obvious is that the drug company doesn't want to make it known that they're working on drug X. Not sure how that works, but you could imagine that they might have to patent it to keep others from using it, and then the clock on the patent starts before they actually get it working.
Except clinical trials are not required to attain patent protection. Any drug company with good patent practitioners is going to have a patent application filed *before* clinical trials. You can always abandon it later if the clinical trials don't work out. It's much cheaper to file and then abandon an application on a drug than to miss the opportunity to get a patent on a successful drug.
Re: (Score:2)
Ah. Well then, nevermind. Maybe it's just the innate "Keep secret all that you possibly can," attitude.
Re:Negative results (Score:5, Interesting)
I have been told by someone who would know (psychiatrist) that drug trials (maybe just here in Australia) have to be announced before the trial begins otherwise the outcomes can't be published. The idea is that you have to announce a trial before you begin, and so if the outcomes are bad and you don't publish, the bad outcome can be inferred from the lack of publication, even if the specifics remain unknown.
A one sentence (Score:4, Funny)
Summary? You're practically forcing me to read TFA
More complaining and second-guessing (Score:2, Insightful)
Cancer patients are already vulnerable to cancer.
Not sure what the point of this story is. Sometimes things don't work out the way everyone wishes they would. Apparently every decision to say something or not say something always has to be second-guessed by third parties who have no responsibility or accountability -- but they get to demand things anyway.
I'm sure a lot more of these failed trials would be published if there was a financial incentive. The complainers should start a foundation and start pa
Re: (Score:2)
Do you realize how hilarious it is for you to complain about a study for complaining about studies? If you think it's wrong for third parties to second-guess studies and demand things with no responsibility or accountability, maybe you should stop doing it yourself.
Re: (Score:2)
I didn't demand anything. I suggested that the people demanding things start paying for the work they are demanding others do. Or they could just start minding their own business and stop demanding something for nothing.
If they're going to complain, why shouldn't they should expect complaints in return?
Re: (Score:2, Flamebait)
There is no difference between a suggestion and a demand when the person making it has no power over the target.
It's not about expecting complaints. It's about you saying that these complaints are bad, while doing the exact same thing yourself. What makes their complaints bad but yours legitimate?
Re: (Score:2)
There is no difference between a suggestion and a demand when the person making it has no power over the target.
Yes there is. There may be no difference in the results, but there is a difference in the method.
As for your second point (and your sig), quit trolling with weak pseudo-logic. He wasn't complaining that they were complaining: he was complaining at where their complaints were directed. You're allowed to complain about complainers when the original complaints are moronic!
Re: (Score:2)
He's complaint was about studies complaining about other studies. He was also complaining about a study. I don't see any difference between the two other than that he has a whole lot less funding and, I'm guessing here, expertise. Otherwise they look the same to me: both of them are people telling other people that their efforts would be better applied in a different way.
Re: (Score:1)
Now we see the fallacy of your argument. Go back and read what he said again. He wasn't complaining about the study. He was complaining about the story and the author's complaining based upon their study. Possibly might be also referring to people commenting here complaining, but then he never said that specifically.
Oh - and - a heartfelt "fuck you" back at ya.
Re:More complaining and second-guessing (Score:5, Insightful)
Re:More complaining and second-guessing (Score:5, Insightful)
It's not just cherry picking - which wouldn't matter so much in itself. Drugs are very hit and miss you expect lots of things to just not work.
The problem is they study drugs X, Y, and Z in combination and find that not only does it not help it makes the patients worse. They don't bother publishing since they didn't get anything useful out of it and no one is going to cite them...
A year later another group decides to study X, Y, and Z in combination. So a pointless study that harms patients is done because this second group never found out about the previous study in their literature search.
Re: (Score:1, Interesting)
From the article... there is a registry of all of these trials (from the US, at least). Any group that decides to run a trial is going to check this registry first. If they find that a trial was registered, but no paper was ever released they would probably: a) assume the trial was not a success or b) contact those involved with the trial and ask for more details.
Medical journals are as much for practicing MDs as they are for researchers. Flooding them with failed trials would only dilute the valuable mater
Re:More complaining and second-guessing (Score:4, Interesting)
As somebody who was previously in the academic medical field, this is not just a problem with cancer drugs. "Positive Outcome Bias" or "publish bias" is a huge problem.
http://www.ama-assn.org/public/peer/7_15_98/jpv71042.htm [ama-assn.org]
A negative study should be just as important as a positive study. If done well... obviously.
Published negative studies dissuade doctors from using certain offlabel treatments. Published negative studies prevent other docs from wasting time and money to discover the same results.
More importantly, many clinical changes are based on meta-studies... which as basically studies which combine all the available data. If negative studies are not published, it throws off these metastudies... and thus bad care occurs.
Re: (Score:1)
More importantly, many clinical changes are based on meta-studies... which as basically studies which combine all the available data. If negative studies are not published, it throws off these metastudies... and thus bad care occurs.
How does the positive publication bias "throw off" metastudies? Please, feel free to use statistical terms. I do not see how this is possible off-hand unless the metastudies use seriously flawed methodology.
Presumably, they aggregate "like-typed" (in the sense that they have "
Re: (Score:1)
This is my gut reaction to reading TFA. I understand that in a sense, this isn't new news. Some of the commenters have talked about the time and cost associated with organizing and publishing negative/unintended results. However, from a scientific point of view, this seems like the single most important reason to publis
Re: (Score:1)
The point of the story is that companies publish the successful trials on a drug, but don't publish failed trials on that same drug - i.e. they cherry-pick the results.
This (almost certainly) has nothing to do with drug companies cherry-picking results. It's extremely difficult to get negative results through peer review, and as a result there are very few studies of any sort that report "X doesn't have an effect" or "Y is not involved in Z."
Part of this is because of the statistical modeling associated with these studies. It's relatively easy to be certain, and to prove, that the observations from two groups are different: this is the type I error [wikipedia.org] everyone loves. We'
Re: (Score:2, Informative)
Maybe try reading the article?
FTFA:
"Of particular concern," they argue, is the especially poor showing by industry-sponsored trials, since they tended to probe the value of patented drugs -- "many of which are in clinical use."
I would say that's newsworthy if not all that surprising. Just another case of big drug companies screwing over the general population to make a buck. They run studies to test their drugs and if the results are unfavorable they just bury the study while keeping the drug in circulation.
Re: (Score:2)
Re: (Score:1)
So it's better for them to be vulnerable to cancer plus ineffective, expensive treatments, often with nasty side effects, that substitute for more effective ones?
These failed trials would be published if the researchers were actually doing science. If they're not, let's call it what it is: product development. And let's remove the fiction that what they are do
Cancer treatment is a product (Score:1, Troll)
The ideal product will lessen the symptoms, but not cure the disease, at least until a 10-year-course of expensive treatment is over.
This is how you fellow humans view you: as a potential walking dollar sign, in exchange for their goods and services.
I hear horse urine cures cancer. I have fresh horse urine for only $9 per ounce. Do you want to be cured, or not?
Re: (Score:1, Funny)
Many of the older anti-drinking crowd in these parts, and others, commonly refer to beer as horse's piss so I guess I will just stick with beer. Though its price has risen absurdly in my lifetime, it isn't quite to $9/oz yet.
Re:Cancer treatment is a product (Score:5, Funny)
I have fresh horse urine for only $9 per ounce. Do you want to be cured, or not?
Nay.
Re: (Score:3, Insightful)
No, they'd much rather have those 10 years worth of payments up front, so they can invest or re-invest it.
You seriously don't think people would pay more for a week-long cure than for a week's worth of "treatment-in-perpetuity?"
Re: (Score:2)
.
The "cure for cancer" is just the beginning - at that point you can begin to think seriously about directing research towards the anti-aging drugs, the rejuvenation techniques, that have been stuff of epic myth, science fiction and fantasy since the beginnings of human history.
Re: (Score:1)
Re: (Score:3, Interesting)
Horse urine can't cure cancer, although it is used in hormone replacement therapy. I suppose some cancer patients driven to early menopause due to radiation and chemotherapy do use it, though...
http://en.wikipedia.org/wiki/Premarin [wikipedia.org]
Is Greed built into our genes? (Score:2)
Too bad that particularly nasty trait (greed) is built right into our genes. I don't ever see any well-meaning social movement eradicating it from our species.
One reason I oppose the doctrine that we're all equal is that I've observed the more intelligent, confident and physically attractive people are less greedy.
There are people out there who, unlike Hollywood stars, look great without makeup, went to good schools and have good careers doing things that (unlike Hollywood) make life better for people.
Then there are the ugly fat guys in Porsches who seem to think that having their own business as a patent sniper makes them "equal" to someone who is smarter, more
Financial incentive (Score:4, Interesting)
The obvious reason is that it takes time and money to publish study results, neither of which are recouped currently if the study showed negative results.
The obvious fix is to reward pharmaceutical companies financially for publishing all results. Form a subentity within the NIH with the power to purchase study data and results that can be published by the government or a peer-reviewed journal.
Re: (Score:2)
The obvious fix is to reward pharmaceutical companies financially for publishing all results.
Or simply compel them by law to publish all study results for any drugs under consideration for active use at any later stage. Their excuse for the enormous advantages they gain through patent protection and the like, and the borderline ethics of a lot of their business practices, is that R&D in a regulated industry is expensive. The cost of publishing failed tests as well would be absolutely negligible if the R&D costs as much overall as they keep telling us.
Re: (Score:2)
You can only publish if you can find a publisher. The most obvious solution is to expand the role of the (inter)national trial register to be a place to go to find out results as well.
Re: (Score:1)
I know - anonymous coward and all..
But it's not cheap to develop effective drugs. The cost to produce them (effort vs gains) and the limited timespan they have to recoup said costs is a large reason they are so expensive.
What might work is a repository of failed experiments that isn't too stringent on the peer reviewing or reproduction. This way you get some history as to what was done (and didn't work) without the cost associated with PROVING that it did work.
If at any time in the future these results woul
Re: (Score:2)
> reward pharmaceutical companies financially for publishing all results
What a brilliant idea. If that actually occurs, I'll immediately buy large amounts of stock in these companies. And then read their reports:
"Report on whale pee on cancer: unsuccessful"
"Report on heavy wishing on cancer: unsuccessful"
"Report on digital watches on cancer: unsuccessful"
The three reports above will already pull in a couple of ten thousand bucks...
Cargo Cult Science (Score:5, Insightful)
Re: (Score:2)
Nothing new (Score:4, Insightful)
Re:Nothing new (Score:5, Informative)
The pharmaceutical industry HAS been caught doing some sinister things with their studies. They do ten studies and publish the three that can be interpreted positively. The seven that can't never see the light of day. Unless of course they accidentally get published on the company web page. Then the sinister part: the drug is approved and sold to treat the condition. The clinical trials registry was formed to try to clean up that particular mess.
This particular story doesn't seem to describe anything sinister. The trials were all in the registry (that's where the authors got the data for the study). Sure, the journals tended not to publish the negative results, but they were all available in the registry.
As far as I can tell from the article they didn't even look at the actual worrisome situation where there are a few published positive studies and many more unpublished negative studies, for the same treatment.
Re: (Score:1)
My impression from the article was that nobody checks this registry (aka file drawer). I didn't mean to give pharma a free pass -- the article suggests they have a greater rate of "file drawer" than non-industry research.
The article highlighted the problem perfectly -- only 20% of the studies in the registry are reaching publication. Since publication is where many people (doctors) learn about what's new, 80% of the data is never seen.
They are trying to do something about it, such as require results to be
Re: (Score:2)
If that's happening (which the study didn't seem to address in any way), then it's the physician's fault, not the researchers. All the data were published in the appropriate places. You can't ask the mainstream journals to publish everything, or they will become "The Journal of Studies That Found Nothing New." I would certainly hope that my physician would check for published negative studies before using some off-label experimental treatment on me.
Realistically, most physicians hear about treatments the
Re: (Score:1)
I thought it was pretty clear they don't consider the registry to be effectively disseminating results: "Data from fewer than one in five research trials are ever published. Findings from the vast majority of human trials become buried for reasons that may never come to light". If blame is to be placed, I agree it's always on the reader -- researchers don't bother publishing failed studies because nobody wants to publish them. They don't sell subscriptions. But that hardly means we can expect doctors to
Re:Nothing new (Score:4, Informative)
"Published" in a major journal. ALL of the studies were published in the registry. As I said, you can't ask the major journals to publish every failed study.
If you want to try and standardize trials, go for it. I wish you luck. Speaking as someone who actually does them, it won't happen. There are all sorts of reasons why trials are difficult to compare. Different centres, different technology, different standards of care. Ethics committees in different places demand different things. Not to mention you rarely have a situation where many trials are testing exactly the same thing. Most meta-analyses look at trials that are designed to look at slightly different things and then try to compensate for the differences in study design.
Re: (Score:2)
"If that's happening (which the study didn't seem to address in any way), then it's the physician's fault"
Oh reeeeeeally?
"I would certainly hope that my physician would check for published negative studies before using some off-label experimental treatment on me."
Have you spent any time in a doctors office? I mean from the backend? Examine how the physicians time (let alone that of their staff) is allocated.
60% or more is tied up in administrative tasks that:
C.T.
From TFA (Score:2)
It's called publication bias and it exists in most fields. Publisher want to break significant results, so even if your research was extremely well designed, and the fact that you found negative results could be extremely important, they
Re: (Score:2)
Medical Profession and Institutions medieval (Score:2)
I'm not at all surprised that findings don't get published. The only way to force them to be published would be to make it a condition of doing the study or a legal requirement. The pharm companies couldn't care less that patients die - they prefer the competition wastes time repeating the same tests rather than taking advantage of the work they've already done.
The entire way that the medical system works really needs a shake up. Other industries with less at stake have more stringent quality control proces
This is a general problem with science (Score:1)
This is "dark data" (Score:1, Interesting)
Some scientists are interested on getting all data published, even if it didn't result in big news. They call this the "dark data".
http://www.wired.com/science/discoveries/magazine/15-10/st_essay
I'm publishing the results of a failed cancer tria (Score:2)
My mother went through a 6 week series of trials of epsom salts against colo-rectal cancer.
The mixture was ineffective.
My, how very useful this information is for cancer patients!
Would my mother have received the quality of the care she eventually did get if her doctors had missed relevant articles in medical journals thanks to the massive signal to noise ratio?
If every failure were published, the cancer research community would suffer the same "eternal september" the usenet community did.
Re:I'm publishing the results of a failed cancer t (Score:4, Insightful)
Yes, because that is exactly the same thing. *sigh*
What if the researchers developing new drugs and treatments had access to the failures of others so that they knew what *not* to try. Outside of your pathetically childish and facetious example about Epsom salts, this information could be invaluable. Would you have wanted your mother to die because scientists working for Pfizer didn't tell the community about a failed treatment that they had already tried which GlaxoSmithkline then spent 2 years replicating, at the expense of another possibly more fruitful avenue of research?
an argument for open source research (Score:2)
I wouldn't have been able to see so far had I not been standing on the shoulders of giants applies to stuff that doesn't work as well. Some things, Like medicine, are too important not to have companies and researchers work together on. This is just an example of how legislature is created by money. The Golden rule: those who have all the gold, make all the rules!