Medical Journals Fight Burying of Inconvenient Research 32
A dozen leading medical journals have announced that they plan to refuse publication of clinical studies unless those studies were publicly registered ahead of time. Part of the intention is to prevent researchers from privately doing multiple studies and then selectively releasing for publication only those which yield favorable results. There are many other journals which have not signed on to this plan, however, and it remains to be seen what will happen. Personally, I'm surprised it's taken this long; as Karl Popper wrote, "what distinguishes the scientific approach and method from the prescientific approach is the method of attempted falsification."
Great idea! (Score:4, Insightful)
On second thought, they're all funding their own studies, so it's probably a non-issue.
Finally (Score:2, Insightful)
Re:Finally (Score:3, Interesting)
If they don't know why things are failing, how can they improve?
Re:Finally (Score:5, Insightful)
Publishing null results would help people avoid repeating things that have been done already, as well as help refine research to see if there is a positive result hidden in the null.
In the case of big medical studies it could provide a great source for data mining, where metastudies of a bunch of null results might suggest something that would be hard to see without a lot of overlapping data.
Re:Finally (Score:2)
I got fed up with this after a while, and when other people got positive results, I started telling them, "Don't worry, a positive result is just as good as a null result." It got me a lot of funny looks.
Later, working as an applied physicist, where null results are far less common, I tried to include a s
Meanwhile (Score:5, Informative)
Please note that despite Jamie's spin (and the Times') all studies have to be reported to the FDA. I'm not quite sure of all the differences between this new registry and the existing registration, but the idea that drug companies can perform trials in complete secrecy is as wrong as the idea that NIH-funded research does all (or even much) of the work of drug development -- a point Lowe addresses elsewhere.
Re:Meanwhile (Score:5, Interesting)
No they just do the basic research that results in the drug leads. The companies then do the expensive but scientifically easy trials and rake in all the money (and now it seems, the credit as well).
And since when is an industry spokesman considered a reliable source of information..?
Re:Meanwhile (Score:5, Interesting)
This nicely illustrates Lowe's point: what you're saying is widely believed, but is absolutely, utterly, entirely, absurdly false. (Except for the non-sequitur at the end about "credit" which I don't understand at all.)
And since when is an industry spokesman considered a reliable source of information..?
First, he's a chemist, not a "spokesman". Second, he (and I) do precisely the work you claim doesn't exist and might be thought to have something to say on the matter. But, if you want to limit your "reliable sources of information" to people who don't know what they're talking about, that's certainly your right.
Sigh... (Score:2)
My point was that to claim that no drug development is done with public funding is misleading and mendacious - because while it
Re:Meanwhile (Score:1)
Re:Meanwhile (Score:1)
Partial solution (Score:4, Insightful)
And nothing stops a drug company from funding a bunch of studies that aren't registered, and then registering duplicate studies that they then expect to be most favorable. Of course, the registration process would then add expense and delay in getting out this sort of slanted results.
At least even with the current system, I expect the peer reviewed journals are much better than the sort of "studies" that get published regarding the computer industry (e.g., TCO of Windows vs. Linux).
Re:Partial solution (Score:4, Informative)
The results of the studies are random, or contain randomness. If you fund a bunch of studies (say, 20 studies) and one of the shows positive results, you can't just duplicate the results. You'd have to duplicate the study, again asking a hospital or two to recruit 300 patients to test the drug, and this time the results might not be positive. You can't even recruit the same patients over, because the standard is to recruit the first consecutive 300 patients to walk in the door who fit the criteria (and consent to the study, of course) to prevent selection bias. Any other way of recruiting would raise red flags that any medical student could spot a mile away.
(For the record --yes, IAAD.)
Re:Partial solution (Score:2, Informative)
Re:Partial solution (Score:3, Interesting)
Methodology can't be substituted that easily (Score:4, Informative)
It's much harder to set your own study methodology (and still get the same respect). The family doc who makes an effort to do evidence-based medicine (as opposed to "this is so because my doddering old professor back in medical school said that his teacher said it was so") looks for several key points in the abstract of the journal articles. (What, you think s/he has time to read the whole thing?) You want a randomized, double-blind placebo-controlled multi-centre study with a large sample of the general population . If these keywords are there, the doc jumps to the "conclusion" heading and generally acknowledges it to be valid. The main text of the article is just for medical students who have to do their presentations. :)
There are other study methodologies with such names as "cohort studies", "case-controlled", "retrospective" which get some respect but are viewed generally as a steppingstone to getting funding for the ultimate gold standard, the randomized clinical trial. So if a company's drug trial works with case-controlled, that might not be enough to get FDA approval for whatever labeled use they're after.
Only 20 studies needed to prove placebo works (Score:5, Informative)
What this means is that, if left to chance, the experiment/trial/study would be positive in only 1 in 20 times. Example: you give a bunch of sick people an experimental drug, and you give another bunch of sick people a placebo (or a known standard of treatment). The people on the experimental drug get better. Was it really because of the drug? Maybe it was just random chance --but if that chance is calculated to be only 1 in 20 or less, then we say, "Yeah, it's probably because of the drug, and not just chance."
The overwhelming majority of drug trials are corporate-funded. A company that's desperate enough to get its drug to market could easily fund 20 studies, and even if the drug were just placebo, chances are good that 1 in 20 of those studies would turn out positive. (Yes, yes, I'm just approximating.) Without the "negative results registry", you'd think that the drug was working.
Would companies be desperate enough to do this? You bet. I'm not saying that any particular company did this, but consider what happens to get a drug to market. Someone invents a molecule (typically a lab with 12 employess or something), gets bought by a biotech firm with 4 employees (they subcontract everything out), some other lab tests it on animals, some other firm develops a formulation (tablets, capsules, makes sure it doesn't melt inside the bottle or degrade, etc.), a big drug company buys the formulation (or the entire firm) and starts gearing up for clinical trials while submitting for FDA approval. This all takes about five years. What if it doesn't work out? As a backup, the Big Pharma Company also invests in about five backup compounds, and each of those compounds has five backup compounds. We're talking about, after ten years and researching thirty compounds, you might get ONE drug out to the market. (Btw, my wife is the project manager for a bunch of these drug research pipelines at one such Big Pharma company.) But, boy, will that drug make it big! What if the drug didn't really work? Well, let's make it look like it did! (I can see Big Pharma CEO's rationalizing this as "let's put it in the best light possible.")
Example of drug research being biased? Ever heard of celecoxib (Celebrex)? Wow, anti-inflammatory pain reliever that, unlike ASA (Aspirin) or ibuprofen (Advil, Motrin), does NOT irritate your stomach! No stomach bleeding (uncommon but serious side effect of ASA/ibuprofen)! They did the research and showed that people actually did (statistically) significantly better than ASA after six months. JAMA (Journal of the American Medical Association) published the study and even sang its praises in the editorial. They get it out and market it to all the physicians all over the place.
And then we find out that the study went on for more than 6 months. We find out that beyond 6 months, the people using Celebrex got WORSE, and deteriorated until at 12 months, they were no better than ASA users. Boy was the JAMA editor mad! (If I recall, he even publicly lambasted Searle for this in the New York Times.)
But you know what? It didn't matter. Celebrex was everywhere, on American TV ads, and people asked for it. Docs who don't really have time to delve into the medical literature already had established in their mind that "Celebrex is better". (My colleagues certainly continued to use it even when ASA was sufficient.) And the drug reps, who ooze snake oil from their skin pores, keep pushing it. One drug rep even questioned my choice of medication when I was getting it from our drug sample closet. I lit into her like you wouldn't believe. Is she the doctor or am I?? (whew, catharsis, feel better now)
So, yes, I think the companies are perfectly capable of doing this (stacking the studies). The benefits are just too great. I welcome the use of the clinical trial pre-registry.
Re:Only 20 studies needed to prove placebo works (Score:3, Interesting)
Re:Only 20 studies needed to prove placebo works (Score:2, Insightful)
Would companies be desperate enough to do this? You bet.
I bet not. These studies can cost millions. Sure, BigPharma has BigMoney, but you're suggesting that they can easily soak a 2000% increase in their phase 4 clinical trial budget!? N
Re:Only 20 studies needed to prove placebo works (Score:1)
1: drug works and you wasted a few mill but your about to make 100's of mill so not a big deal.
2: drug works and one study would have failed to detect that... HUGE WIN!
3: drug fails and random chance favor's you. Make back most / all your reasearch money.
4: drug fails on all study's... O well you lost all your money anyway.
Now I can easly see this being worth i
Wouldn't u spend billions to prove placebo works? (Score:3, Insightful)
I bet not. These studies can cost millions. Sure, BigPharma has BigMoney, but you're suggesting that they can easily soak a 2000% increase in their phase 4 clinical trial budget!? No way.
If you could take a placebo and fund 20 studies to show that it worked, wouldn't you? That's like saying, I can sink millions and billions into this research to prove that potato chips cure cancer. And I hold the patent on the potato chip. I gain the abili
Re:Only 20 studies needed to prove placebo works (Score:1)
One word: Celebrex
(As I mentioned in my previous post, Searle obviously did not give everything to the FDA, who fast-tracked the approval.)
You are right that, realistically, you can't get people to use a drug based on a single trial; that's just an extreme example to show that even placebo can do well if you study it hard enough and allow publication bias. In real
Re:Only 20 studies needed to prove placebo works (Score:1, Interesting)
I agree that selective publication has the potential to influence use of an approved drug. My point was that there's no way to get a new drug approved based on selective data submission, unless the company's officers are willing to take a substantial risk of going to jail.
Second, Searle did give the FDA all that data on Celebrex, but the whole mess took place after the drug was already approved.
Celebrex was originally approved in Dec 1998. At that time, FDA told Se
Everybody knows (Score:1)