Scientist Says Most Scientific Papers Are Wrong 656
An anonymous reader writes "According to epidemiologist John Ioannidis, the majority of published scientific papers are wrong. If Ioannidis's own paper is right, a randomly chosen scientific paper has less than a 50% chance of being true. He also says that many papers may only be accurate measures of the prevailing bias among scientists. However, a senior editor of a scientific journal says that scientists are already aware of this: 'When I read the literature, I'm not reading it to find proof like a textbook. I'm reading to get ideas. So even if something is wrong with the paper, if they have the kernel of a novel idea, that's something to think about.'"
groan (Score:3, Funny)
Great... watch the Creationist/Intelligent Design kooks run with this.
Re:groan (Score:4, Insightful)
Yes, but unlike religious dogma, scientific theories are meant to be falsifiable.
Unless someone in the ID camp is willing to admit that God is falsifiable, their theory will not be considered science.
Re:groan (Score:2, Insightful)
In ~10 years of trying to find scientific evidence of just a single god the ID people haven't published a single paper. So now they attack the school system when their original mission (evidence) has failed.
Re:groan (Score:5, Insightful)
Based on all of the anti-creationalist comments I thought maybe I had misread the summary, so I looked at the article itself.
Not once did I see mention of the universe's creation in the summary or in the linked artical, in fact the example stated was "such as whether a particular gene influences a particular disease."
It seems to me that lately a lot of comments on slashdot have been trying to start a witch hunt for advocates of ID. Can we please knock it off and stop screaming wolf every time some thing that is related to science is mentioned on slashdot.
Re:groan (Score:5, Funny)
You realize you're talking to zealots, right? I don't think "please" is going to cover it.
Sadly, they did publish (Score:5, Interesting)
Skeptic had their take on it in the last issue. In a nutshell
Re:Sadly, they did publish (Score:3, Insightful)
Actually, they were probably a few random /. mods who said the article was "insightful".
Re:groan (Score:3, Funny)
Re: groan (Score:3, Interesting)
Sure, the vast majority of stuff in Biblical Archaeology Review (if it's still publ
Re: groan (Score:3, Interesting)
Oh yes there frikken would. It might be called something like Ancient Roman Archeological Review and it would be a bunch of archaeologists talking about archaeological evidence being used to describe events taking place in ancient Rome, just as Biblical Archaeological Review would discuss archaeological evidence for events taking place during the Bible.
Re: groan (Score:3, Interesting)
> Yes, I agree with your first statement, though, it would be hard to believe in purpose without something to create the purpose, wouldn't it?
Why so? Is purpose necessarily dictated by some external agency? Can't purpose-oriented creatures decide on their own purposes? Do people who believe in a false religion have any difficulty finding a purpose in life?
> By that, I mean I understand someone could rationally conclude the idea that the complex system we live in didn't just...happen out of raw chance
Re: groan (Score:3, Funny)
Re:But it did spark a tasty open letter (Score:3, Funny)
But what our scientist does not realize is that every time he makes a measurement, the Flying Spaghetti Monster is there changing the results with His Noodly Appendage.
Which explains why most scientific papers are wrong.
Re:groan (Score:4, Informative)
God's irrelevant to whether ID is true or not --- it's whether ID is falsifiable or not that's important.
Which, AFAICT, it isn't, so it's still not science. But let's at least be precise when slagging them off...
You miss the point (Score:3, Insightful)
Here's the attitude.
"you want someone to believe human origins from a set of people that told me I would die if I smoked and ate a cheeseburger and I'm still living."
"well now basically you are just making up evolution to fit your story together. Well I can do that too. Can't test it either way, can we..."
I wonder if Global Warming isn't approaching (Score:5, Insightful)
The only question is, who decides which science is wrong? I doubt very seriously any big money areas will have a published high rate of error. After the high money science the next protected type would be whatever is en vogue for the time.
Scientific integrity took a big dive in the late 80s as special interest groups suddenly realized that marketing, confusion, and intimidation were far better at advancing agenda than honest science.
Re:I wonder if Global Warming isn't approaching (Score:3, Interesting)
First things first; We need to figure out the facts of the situation, without regard to the consequences of those facts.
"Global Climate Change" is a scientific theory with good support.
Some of the reactions to those changes and ant
Re:groan (Score:3, Funny)
Ah, but they don't actually need to admit such a thing, because the Babel fish is a dead giveaway, isn't it? It could not have evolved by chance. It proves God exists, so therefore, by ID/Creationists' own arguements, God doesn't. QED.
Re:groan (Score:5, Insightful)
e.g. Flying Spaghetti Monsterism
http://en.wikipedia.org/wiki/Spaghetti_%26_Pulsar
Re:groan (Score:3, Funny)
http://www.newscientist.com/channel/life/gm-food/d n7729 [newscientist.com]
Researchers at the Centre for Ecology and Hydrology in Dorset, UK, tested the herbicide glufosinate ammonium
Re:groan (Score:4, Insightful)
I do dispute the scientific validity of the claim that ONLY intelligence can create new forms of life.
Your straw man is now on fire.
Re: groan (Score:4, Insightful)
[snip irrelevant bullshit]
> Why would anyone want to close their eyes and cover their ears and say "I can't hear you - there is only evolution - there is no intelligent design - I'm not listening to you"? When actual real scientists are creating organisms which other scientists cannot distinguish from similar species found in nature?
I don't see anyone applying ID methodologies to determine whether this plant is the result of intelligent design or not. Any idea why?
Re:groan (Score:3, Insightful)
generally, apart from America, the rest of the developed world has no question about the validity of evolution. there is so much evidence it doesn't even bear consideration except in the USA where extremists try to claim the bible is fact when historically no other Christians have had this opinion because religion isn't supposed to be about facts anyway.
the problem is that science has been so sucessful that now everyone looks at things
We got 'em on the run (Score:3, Insightful)
Re:groan (Score:3, Insightful)
Realistically, I have been through grad school with a REAL science degree. I now work as a researcher at Penn State University. Even though I am not promoting wierdo pseudo-science, I am not to impressed with the scientific community as well. They pull some of the sam
Re:groan (Score:4, Insightful)
The thing is, there are different degrees of wrongness. It's one thing to write a scientific paper that may be "wrong" in the sense that the evidence for a particular hypothesis may not be quite as strong as the author would like; if that hypothesis is of any particular importance, and it is in fact false, sooner or later someone will come along and show it to be so. It's quite another to attack and try to tear down science itself, which is what the creationist/ID folks do. In the world they'd like to create, there would be no such thing as actual science -- and what passed for science in that world would be wrong, not just marginally wrong but drastically so, all the time.
Lamarck and Darwin were wrong too (Score:5, Informative)
That is how the system works.
But just because these two scientists were wrong about the precise mechanics of evolution doesn't mean that they were wrong about how the data should be interpreted. The data shows that life has progressed to meet the demands of its environment. Survival of the fittest is correct, but there is no straight-line progression of lifeforms leading one from another as was supposed when these authors first penned their ideas.
Scientific ideas may come and go, but the data set just gets larger. That is why this guy can claim the others are wrong: he has a better data set.
Re:Lamarck and Darwin were wrong too (Score:5, Insightful)
But if the creationists/intelligent design advocates/Christian fundamentalists want to use this to say that they're right, they're relying on a logical fallacy. Just because a few papers are wrong doesn't mean that their view is correct. Their view of creationism is not the only alternative to the view of evolution present in a few possibly flawed papers. Evolution may work in a way that we aren't sure about, but this doesn't prove that intelligent design is correct.
Re:Lamarck and Darwin were wrong too (Score:5, Interesting)
Actually, strictly speaking, Einstein (and friends) didn't prove Galileo or anyone else wrong. That had already been done by others. Thus, precise measurements of the orbit of Mercury and turned up discrepancies with Newton's and others' laws of orbital mechanics. The Michaelson-Morley experiments produced the apparently-absurd result that light moved at the same speed relative to all observers, even if those observers were moving relative to each other or the light source. Etc.
What Einstein did was develop a new theoretic approach that could explain a number of these anomalies. It was then up to the scientific community to viciously attack Einstein's theories, and attempt to prove him wrong. They've been at this for a century now, and all of their tests so far have end up with results consistent with Einstein's theories, to within the error bounds of the measurements. In scientific circles, this constitutes "proof" that Einstein's theories are either correct, or are very close to correct.
Even then, the earlier theories hadn't really been proven wrong. Rather, they were shown to be merely good approximations. After all, if your instruments can measure something to 12 places, but Einstein's and Newton's equations predict a difference in the 20th place, you can't show either set of equations to be wrong. This is why those earlier "disproved" theories are still taught in science and engineering schools. Newton's equations are a lot simpler than Einstein's, and in situations where you can't measure the difference, you might as well use the simplest equations. You just have to be careful not to apply the simpler equations in situations where they aren't good enough.
But note that Einstein himself didn't disprove those earlier theories; that had been done by the others that found the anomalies. And Einstein didn't prove his own theories; that has been done by a century of tests by the entire scientific community. He did the really hard job: He came up with his wild new theories of a universe that behaved rather differently than anyone thought. But his theories were consistent with those strange observations. And his theories included equations that could be tested against the real universe. And his theories keep passing every test that anyone comes up with.
Now if we could get some other would-be scientists to present us with versions of their theories that can be tested against the real universe
Re:Lamarck and Darwin were wrong too (Score:5, Funny)
If fish aren't landmines, how can you be sure we're really elk?
But, more to the point, should you be really drawing any conclusions?
No. Not until the smoke clears, and I can once again tell the difference between billiard balls, atoms, monkeys, landmines, and elk. And neither should you.
Re:Lamarck and Darwin were wrong too (Score:3, Insightful)
Cite the evidence to support your statement. You have posted more than once how evolution is lacking in scientific merit without one shred of evidence.
I'm calling your bluff. Post the evidence.
Face it, there's a new orthodoxy.
As the Who sang: "Meet the new boss/Same as the old boss." Reactionary religious people bashing scientific evidence is nothing new.
Post your evidence that evolution is lacking scientific merit.
Re:Lamarck and Darwin were wrong too (Score:3, Insightful)
I never made that claim.
You said:
Similarly, the things which evolution appears to imply could be mis-conclusions.
and:
Yeah but I would think you would look at the fundamental flaws in Newtonian physics as very, very instructive about how not to put too much faith in Evolution.
and finally:
If atoms aren't billard balls, how can you be sure we're monkeys?
You seem to run around in circles taking sideways shots at evolution, but when called on your
Re:Lamarck and Darwin were wrong too (Score:3, Interesting)
Re:Lamarck and Darwin were wrong too (Score:5, Insightful)
Engineer says: Most papers are never read! (Score:5, Insightful)
Engineers often read papers to solve problems. When they know about them! (Google Scholar might fix this)
A worse problem than them often being wrong is that:
a) there is frequently no way to determine if a given paper is accurate, has mistakes, is partially accurate, is laughable, was accurate at one point but is outdated, etc etc. At least from an outsider's perspective.
b) there is no good way to stay abreast of current interesting developments - hell, there's no way to see interesting things from 20 years ago easily! Again, this is from an interested outsider's perspective.
Once or twice a year I have the luxury of spending a week or two in an engineering library for the express purpose of finding out new and interesting things in my field. I'm SHOCKED at the amount of material that is being duplicated (often badly) in industry, material that is inaccurate or poor quality, and VERY GOOD material that never sees the light of day again.
Microbiologist says: No kidding! (Score:3, Interesting)
Wrong in a non-scientific sense (Score:5, Insightful)
But the article at hand, isn't talking about that kind of "wrong". He is talking about conclusions that can not be supported by the data presented. Either the reasoning is faulty or the data collection methods are so faulty that no meaningful conclusions can be drawn.
When a theory is proven wrong in the scientific sense, it is a good thing. We learn something new and that be the basis for further developments. But if a theory is proven "wrong" in the mechanical sense, we have no new insights, just a relief from further time wasting.
Re:Lamarck and Darwin were wrong too (Score:4, Interesting)
Huh? What is then the driver of evolution? Gerbils? I guess getting a PhD in 2003 counts as a long time ago...
Re:Lamarck and Darwin were wrong too (Score:4, Informative)
Darwin's original theory of evolution was based on two very obvious things; variations among organisms and natural selection. Natural selection was not something that Darwin "discovered", the idea had been around for a bit and is quite intuitive - those that are better adapted to their environment will survive; rather he proposed a mechanism through which beneficial changes in a population are preserved in the offspring. But he did not do a good job of how these changes are brought about. We all know that the offspring of an organism will not be identical to the parent (unless it is a clone); so that children are taller, stronger, lighter/darker (you get the idea) than their parents. What Darwin proposed was that those variations were unfavorable to an organism were weeded out by natural selection (i.e. the organism was killed by predators, could not feed, etc); while the advantageous were preserved. Darwin assumed that the variation that he saw in a species (or between closely related species) would continue indefinitely; that if an organism that can run 40 mph has offspring that can run 1 mph faster and the offspring's offspring continue the trend, this will continue until in several generations these organisms will be able to reach speeds of 100 mph.
This is where Darwin went wrong, although we must not be too hard on him because he had no knowledge of genetics. The genetic code of an organism puts an upper limit on the variation that is possible. Mendel demonstrated this in his famous experiments with peas; which incidentally took place a little after Darwin wrote "Origin". He crossed peas with red and white flowers and was able to get red, white, and pink flowers; but he never got orange, blue, or yellow flowers. Mendel could only select for those traits that were encoded in the genes. So if the population as a whole did not already have the genes for 100 mph speed (recessive or otherwise) there is no way that natural selection can select those genes. A good rule of thumb to remember is that natural selection cannot select what does not exist. So natural selection cannot drive evolution by itself as it cannot produce new traits, and the traits of an organism are determined by its' genes. So there needs to be a mechanism or process that can create new genes; and this is where mutations come in. Mutations by definition are copying mistakes that change the genes. So if there are new genes, natural selection can do its' job of weeding out the bad traits and leaving the good ones. This is known as Neo-Darwinism as the primary idea of primitive organisms evolving into complex ones remains but the process driving it is not natural selection but mutations. In Neo-Darwinism, natural selection is seen as just an obvious footnote.
Re:Lamarck and Darwin were wrong too (Score:4, Informative)
Oops, how embarrassing for you, because a REALLY top-flight education would have taught you to be more precise. Natural selection IS the driving force that propogates desirable characteristics (and therefore evolution). You forgot that natural selection operates on mutations just as it does on intra-species genetic variation. Perhaps you meant "Darwinism" instead of natural selection, but even then some clear thinking would show that Darwin's theories are very compatable with evolution via mutational variation even though he did not recognize it. After all, where do you think all those intra-species variations came from except mutation, interbreeding and natural selection?
Imagine a species that has only genetic mutation and no intra-species variability (a population of clones, perhaps). Any genetic mutation will be either good for offspring or bad. Natural selection "decides" which is which. Without natural selection mutations are, by definition, neither good nor bad, nor can they result in any evolutionary advancement. If there is no disadvantage to change then neither is there any advantage. Without natural selection random mutational variation would eventually populate the universe with a near-infinite number of completely different individuals.
Far from being relegated to the scientific scrap heap, natural selection is still the prime mover (in fact the only mover) in evolution.
Re:Lamarck and Darwin were wrong too (Score:4, Insightful)
Someday these gaps in our knowledge will be explained rather than merely explained away, but the worst and most dangerous kind of pseudoscience comes from those who claim that science is already complete and attack those who point out evidence which isn't explained by current theories. Evolution is a fact, but it is not one single theory, but rather a changing assortment of related hypotheses, many of which will never be scientifically verified in the most rigorous sense of the term, and even if they were, would not categorically disprove hypotheses such as that intelligence is inherent in the universe and started the ball of life rolling or that life originally evolved on another planet and was seeded here by accident or on purpose, or any of hundreds of speculative variations on our current orthodox conjectures.
Re:Why listen to them if they are always wrong? (Score:5, Insightful)
It is the only objective process for assessing facts from fiction.
In other words, if the best a scientist can tell you today is that, he might be wrong tomorrow, why even bother listening to him?
No one is forcing you to listen. You ignore the information provided by science at your peril.
So you can use science for real things, like physics and design of military weapons and consumer goods, but the rest of it is so much speculative nonsense.
Quantum physics is speculative, but you don't seem to be throwing your computer out the window.
The consequences of guessing wrong about the origin of humanity are completely immaterial to most people's lives.
Dead wrong.
Stalin believed that Darwinian evolution was just a bouguoise concept. He believed in Lamarckian evolution and directed his agricultural ministry to ignore studies that supported Darwinain evolution. Their agricultural industry suffered and people went hungry in the process.
You can't show people evolving any more than someone else can show God making something.
I can show a progression of hominid fossils leading to homo sapien sapien. The Bible is silent about these fossils.
It's immaterial, unprovable, and so why fight over it?
It may be immaterial to you, but the theory is consistent with the evidence we possess. You may not choose to believe it, but that the only thing immaterial about this discussion.
Yeah you can roll out the eliptical argument that evolution is somehow necessary for medicine but most doctors are concerned with the human species, here and now, and now plants and people are related.
Why bother? You obviously believe that the scientific method works differently for investigations related to the origin of humanity than it does when applied to chemistry.
To wit, you can get a Chem E degree and still get into Med School.
You are correct.
Just don't whine to me when you have difficulty making sense of the data you gather without using evolutionary theory.
You will never amount to anything more than a glorified technician.
I can live with that.
Re:Why listen to them if they are always wrong? (Score:3, Insightful)
Except the grandparent didn't say that. He said that that Stalin couldn't feed his people because he directed his ministry to ignore studies that supported Darwinain evolution, which meant his country used inferior methods. The grandparent did not in any way conclude that a nation has to believe in evolution to feed its people. He merely pointe
Re:Why listen to them if they are always wrong? (Score:4, Insightful)
And to think you posted that with a device that is arguably high technology. Gee. It's a good thing those practical thinkers at Signetics and Intel didn't listen to those shifty eyed physicists.....
So you can use science for real things, like physics and design of military weapons and consumer goods, but the rest of it is so much speculative nonsense.
Newsflash. The same people that don't like evolution only like physics when it can be used to attack evolution. The rest of the time it gives rise to uncomfortable facts like the Earth being round and the Universe being billions of years old. They'll get around to rest of the so-called "useful" sciences once those pesky life sciences have been properly re-aligned. I'm also glad people like you don't decide what is "useful" in science. After a demonstration of electrical phenomena, the Queen of England asked Micheal Faraday what of what possible use was all this nattering about electricity. He replied, "Of what use is a newborn babe." Sheesh.
Science is all about being wrong. 99% of it is long painful slogging through mucky fields of sheer wrong and trivialities to find the occaisional nugget of right. I'm mildly amazed that Scientists Can Be Wrong is a subject of discussion. This is only a problem when people who don't have any idea how science works expect scientists to be some sort of infallible priesthood. It also doesn't help when the press seizes on new research that hasn't endured years of attacks and splashes it all over the place. It is as though Firebird 0.3 is headlined as the New Killer App. The press is the worst offender in this regard.
You can't show people evolving any more than someone else can show God making something. It's immaterial, unprovable, and so why fight over it?
You can show things that reproduce really fast evolving. It is quite easy with microorganisms and it isn't too awful bad with insects like fruit flies. It's a bit harder with some fast reproducing plants and an absolute pisser with anything that takes more than a week or two to reproduce. One can still do things like genome tracing and compare and contrast with currently living things that haven't changed in a long time. It is hard to show people evolving. It isn't all that hard to show the effects of evolution on people. Unless of course you live in the US.......
Re:Lamarck and Darwin were wrong too (Score:3, Interesting)
These people know little about the workings of science, and merely took up ID as another weapon for their own agenda. In doing so they discredited both ID and the agenda they were trying to support.
As both a Christian and a scientifically minded person that makes me rather sad and a bit angry.
Re:Lamarck and Darwin were wrong too (Score:4, Insightful)
How so? How do you avoid an endless loop of "who created who"?
To make the Intelligent Design theory work, you need to arbitrarily stop at some point and declare that "this is where the intelligent designer ends, and this is where his/her/its works begin".
In other words, if God created the universe, who created God? And who created that God? After all, at each point one could argue that the preceding step in the creation heirarchy is too complex to have created itself through random chance and so necessitates the existence of a higher power.
Well (Score:4, Funny)
Re:Well (Score:3, Insightful)
Do I believe it? (Score:3, Funny)
I can't believe it!
mod parent funny (Score:2)
Blinded by Science (Score:5, Insightful)
Re:Blinded by Science (Score:5, Interesting)
Let's say a scientist comes up with a new idea, does the research, and publishes a result. OK, assuming we buy the article's premise, there's a 0.5 chance s/he's made a mistake. Now another scientist and then a third duplicate the experiment and get the same result. The odds that the original proposition is in error drops to 0.25, then to 0.125. The odds are now 8:1 the result is valid.
See cold fusion for an example of the converse.
Well, what would you expect. (Score:3, Insightful)
I wish people would be a little more weary of automatically believing everything they read in a scientific paper, or worse a crap article from a journalist who doesn't even understand the paper in the first place.
Studies, Papers, Research (Score:5, Funny)
I patiently await the next article: "Research Shows Three-Quarters of All Researches Are Bullshit".
Re:Studies, Papers, Research (Score:3, Funny)
Most wrong? (Score:3, Informative)
Please be more specific next time. Thank you.
Re:Most wrong? (Score:2)
If you think papers in those journals are mostly correct,
Re:Most wrong? (Score:4, Interesting)
Even when some of these papers could be wrong in their conclusions, or maybe one or two algorithm flaws, but it was papers like these (image processing, etc) that contributed to technology used today, like MPEG4 video.
My point is, unlike these which are done with scientific methodology, in *medical* "research papers" there's oh so much money at stake. I'm sure the article could have said "most medicine research papers are wrong", and I would have believed that.
But science is much more than medicine, and as a scientist, I find it an insult to stain the name of Science because of commercial vias in medical research.
Curiously, I googled for "bias in medical research" (with quotes) and here's the top result, of 426 search results:
Bias in Medical Research [suite101.com] by Maria Spicer.
In contrast, googling for "bias in image processing research" (with quotes) yielded no results.
Of course, google is only a very statistical method for finding out whether something exists or not, but I think you get my point.
Possibly (Score:2)
Peer Review (Score:2)
Does it also mean that unpublished scientific papers are right 50% of the time?
Re: Peer Review (Score:4, Interesting)
> Does this mean that peer review fails as a method to filter out time-wasting, tree-killing dreck?
Peer review isn't a certification of correctness. It's just supposed to filter out the papers where the authors didn't do their homework. It can spot bad logic, use of outdated data, failure to consult important papers in the field, etc. But it can't tell us whether string theory is correct or not.
Re: Peer review (Score:5, Insightful)
However, peer review does not solve all the problems. Most of the research takes a lot of time and effort and referees just read the papers. They do not reproduce the experiments or calculations. So peer review can weed out only obviously bad papers but not papers that looks OK but are wrong.
Re:Peer Review (Score:4, Informative)
Probabilities are soooo confusing... (Score:2, Insightful)
Can someone calculate, based on above, a better estimate on the chance of some paper being wrong?
note: don't publish this calculation on a paper, otherwise it will be subject to these probabilities and we'll have to recalculate all over again....
Sadly True (Score:4, Interesting)
Bad research==dangerous. (Score:5, Insightful)
What I do see as harmful is the attitude towards bad papers. To many academics try to accumulate more and more published papers the same way that slash-dotters try to build up karma. I understand that having papers published can reflect well on someone, but we need more accountability. Journals need to create a more strict system for reviewing papers that are to be published to weed out more of the crap plain and simple. If the evidence does not reflect the claims throw it away. If the research was conducted on a population that was too small or specific for a grand generalized claim about the topic as a whole, throw it out.
I understand that you will always have people just trying to throw their names around, but this needs to be looked at from the grander perspective.
"When I read the literature, I'm not reading it to find proof like a textbook.
Sure there are probably many scientists that think of it this way. But the problem is that bad research (or a bad paper) rarely dies after being published. They are often cited as evidence for years to come in other papers until enough evidence to the contrary comes out to raise questions. Plus, you have crazy professors giving this bad research for their classes to read, and often they don't explain to their classes where research is possibly flawed--so we find ourselves training generations of new scientific minds that run around spouting out bad research. I understand that we all need to take research with a grain of salt when we read it, however bad scientists trying to become famous with their bad ideas or bad papers can be very detrimental to any field.
Re:Bad research==dangerous. (Score:5, Insightful)
All that is true. But you left out possibly the worst effect of false papers - the effect they can have on the funding gatekeepers. A handful of just BAD research papers, all claiming to show what those that hold the pursestrings want to hear, and suddenly that false conclusion is a 'scientific fact' and anyone that wants their studies to be funded in the future had damn well better agree with that. Which becomes a self-perpetuating cycle.
Re:Bad research==dangerous. (Score:4, Informative)
- About half had very little novel content. Maybe one equation changed, a few different examples added
- Two or three had basic mathematical errors
- About half omitted details that were required to easily replicate their results or actually use their methods. I spent weeks piecing together what the authors meant from various clues scattered across appendices, tables and figures.
- Several had gaping holes in the method that were apparent to me, a first year PhD with no experience.
- All of them cherry-picked examples to show their methods in the best light, completely omitting any bad results.
If the research is done by Derek Smart, then what? (Score:2)
http://en.wikipedia.org/wiki/Derek_Smart [wikipedia.org]
Comment removed (Score:5, Insightful)
Re:Expectations (and Profits) (Score:3, Interesting)
I think both you and the poster hit on something very important here. First is that we (as people who are reading the research papers) are not looking for proof, etc. of something. When I grab the latest paper on a topic I work on, I am not going to read it and say, "Oh. They found x which contradicts with what I am seeing. They must be right." Instead, I am looking at their models, results, and the
Journal concept is outdated (Score:4, Insightful)
But seriously, reviewers are biased and sloppy, as are the editors. The fact that reviews are blind means that they are also unaccountable, which fosters even more bias.
Journals take months or years to respond to a submision, and often as not they respond with a rejection so the submitter has to give up or start the whole process over with another journal. There are so many scandals that one could quote. The whole process seems more designed to support the status quo than to promote knowledge.
I have discussed this with many people in academia and they react not with logic, but with horror that I would dare to question a system that they view almost mystical reverence.
WRONG can still be SOUND and USEFUL (Score:4, Insightful)
If a scientific paper is useless to the readership, that publication should reject it and recommend a different journal.
If a paper is wrong and the reviewers KNOW IT then they should send it back for corrections.
If it's WRONG but the reviewers don't or more typically can't know it because it is novel, then publish it. The rightness or wrongness will be sorted out soon enough.
Ever heard of Isaac Newton? Turns out his theories were incomplete in some very fundamental ways, but his theories regarding the motion of objects were the best approximations we had for hundreds of years and are still very useful for macroscopic objects traveling way below c.
So what? (Score:2, Insightful)
They've been replaced by something else. Sure, they're generally true, enough to be taught in physics classes, but all the specifics on gravitation etc. are incorrect.
They're being replaced with: (pick your theory) quantum gravity, string theory, quantum mechanics and more things I don't know.
But so what? Science, by its nature, is always being improved upon. Any time you correct someone else's theory, you could say their theory is now wro
Even Funnier (Score:2, Interesting)
Quote (Score:5, Insightful)
That's what my supervisor used to say to me when I got depressed about lack of progress.
In Related News (Score:3, Funny)
Science = Modeling (Score:3, Interesting)
I'd say it's more like 100% of scientific papers are wrong, it's merely a question of the limitations of the model.
Half empty, or half full? (Score:5, Funny)
I think I'll write that paper on statistics.
I'll take "Inappropriate Words" for $500, Alex. (Score:3, Insightful)
Indiana Jones:
Archaeology is the search for fact, not truth. If it's truth you're looking for, Dr. Tyree's philosophy class is right down the hall.
The Study was Examing *Medical* Science (Score:3, Informative)
While molecular biology papers can be prone to statistically insignicant, but factually stated conclusions, the biggest culprits are clinical studies and 'large-scale' analyses of data.
Good experiments are constructed to give a 'yes' or 'no' answer based on the presence or absence of evidence. The zeal of high-throughput studies and analysis have put more pressure on good statistical analysis. Unfortunately, statistical analysis requires math...which sometimes eludes doctors and biologists. Hence, the problem of missuing statistics and stating inadequately supported conclusions.
-Howard
Science and magic (Score:3, Interesting)
Science is not about finding the truth. It has nothing to do with the truth; people who look to it for truth misunderstand it. Science, and the scientific method, are based on one thing: reproducible effect. I do X, Y, and Z, and T results. If this can be confirmed, reproduced independently, you might have something scientifically useful.
Notice what this does not say: X, Y, and Z are "true"; Z is "true"; X, Y, and Z cause T. Nor does it state the meaning of X, Y, Z, or T. Nor does it say why, in the presence of X, Y, and Z, T occurs. These are irrelevant. The only thing science does, the only thing it is capable of, is one thing: testing if, in the presence of X, Y, and Z, we repeatedly get T. For most things, that's all that matters. This is the scientific method.
Thus it is that science is, quite literally, magic. Look over most fictional magic systems. We have things like "if we say this spell, this thing happens." "If we write these symbols, this thing happens." "If I visualize this thing in my mind, this thing happens." "If a mix a pinch of this and a hair of that, this thing happens." Because it's reproducible, it's useful. The mechanic does not matter: only reproducible effect matters. If waving ones hands and saying a phrase were to be followed consistently by a minor explosion, it would be just as scientific as mixing two chemicals to produce the same effect.
It doesn't matter why. Theories get revised consistently to fit the facts, to document reproducible effects. If phlostigen and ether were accurate and useful models, the fact they have been discarded for more useful models does not matter: science isn't about truth. It is about reproducible effects.
This is why not clinging to pet theories (yes, this includes everyone's favorite: natural evolution) is important: the theories do not matter. One should never fit facts to a theory. One should create a theory to fit the facts.
This is what makes science useful.
How do you measure truth (Score:3, Insightful)
People often quote Newton's physics as being "proved wrong" by Einstein's relativity (and those same people often barely understand the limits of relativity with respect to the quantum mechanical world). However Newtonian physics is good enough for most (though not all) space mission planning since it's still quite accurate so long as you don't get near a large gravity well like the sun or travel too fast. So Newtonian physics isn't "wrong" it's just accurate to within a certain margin and useful under less general conditions than previously thought.
That's what these non scientifically trained creationalist types miss. There is no right or wrong theory, even though that's how the popular scientific press reports it. There is only the ability of a theory to predict what can happen (or has happened) based on a set of conditions, and an accuracy under a given set of conditions. Newtonian physics is no more "wrong" than eating salad is. You just can't misuse it by applying it to the wrong set of conditions (don't eat that salad if you're allergic to the ingredients).
This is a model (Score:5, Insightful)
I think PLoS is peer reviewed, but that paper should never have survived peer review. Occasionally, bad papers slip through, even in the so called hard sciences. This one seems to be one of them. Since PLoS Medicine is pretty well respected for an open access publication, lets assume that this was a lark and more on.
But it makes me curious what the fraction of bad papers looks like in an open access publication like PloS versus a traditional journal like, say, Nature, The Lancet, or New England Journal of Medicine. One reservation people have about open access (or author pays) models was that since PLoS gets paid about $1500 from they authors, they might be accepting vanity papers, or don't triage as well as traditional journals. I don't think they are, but if this paper is any indication, PLoS might take a second look at their peer review process.
Re:This is a model (Score:3, Informative)
That means 50% are right! (Score:4, Insightful)
It would be better still if it was more than 50%, but we can just apply the process repeatedly to push up our confidence (50%, 75%, 87.5%, etc.). A little more attention to statistics would help us raise the base rate above 50%.
Doesn't anyone remember George Box? (Score:4, Insightful)
Papers, not theory (Score:3, Interesting)
Scientific papers are usually written by grad-students trying to earn a degree, and that is usually the only real purpose they will ever serve. The project I am working on now is a continuation of the work that was car
Now if this guy is right... (Score:3, Funny)
The death of science (Score:3, Insightful)
Ask them. To most kids, science is a class they take, where they have to regurgitate "facts" like why the sky is blue, or how hydrogen and oxygen combine to create water. It's a boring class, unless you happen to sit next to an attractive member of the opposite sex, but then, it's still not the class that's interesting...
Science is not 'fact' - Science is the best-known process by which truth can be reliably found.
Science is somewhat like the mathematical function x=1/y. Forever approaching truth, never (exactly) reaching it, forever leaving curious minds with new things to explore. Science is the magical combination of "what if" combined with the "feet on the ground" of experimentation, independent scrutiny, and validation of theories.
The "Scientific method" that is regurgitated by most Jr. High schoolers (in California, anyway) is never really *experienced* except in the case of the rare instructor who goes above and beyond the textbook curriculum. EG:What drudgery! If that really was science, I wouldn't be interested, either!
It's sad. Entire generations of people who never get to experience the awe, wonder, and magic of science, who then seek out that awe and wonder by (best case scenario) watching magicians and listening to Art Bell, or (worst case scenario) performing criminal acts and doing drugs.
How much of the interest in the pseudo-sciences (aliens, conspiracy theories, perpetual motion machines, telepathy, Scientology) comes from the fact that they have never really been exposed to the real thing?
"Not even wrong" (Score:3, Insightful)
not scientists are wrong, speculation is wrong (Score:3, Interesting)
I read many papers, I don't know numbers, but many dozens in a month. Usually I don't care if they seem good or bad, if they are correct or not. The ideas therein are what matter. Sometimes you get ideas on how to improve an old idea, sometimes you get new ideas from older papers. sometimes it's just nice to know what others are doing.
The matter is very much different when you have to review papers, but the seriousness of that review also depends very much on how much time you have, possible IRL problems, etc., but that's why there are >=3 reviewers+associate editor assigned to the paper at most of the serious journals.
Stating such things as a certain percent of all papers are crap is just crazy sh*t. It happens very seldom that I read a published (conference or journal) paper and I think it was useless. Anyway, if it would be true that would mean that this guy's paper is also half useless. You are free to choose which half
Re:Wait (Score:2)
Re:Reach (Score:5, Informative)
No, it's not.
Our biology does not provide us
Our biology provides us with excellent truth detectors: throughout most of primate evolution, if you were wrong about whether your food was poisonous or whether there was a lion hiding in the bushes, you didn't get to pass on your genes. You didn't get to debate social relativism with the lion before he made a tasty meal out of you.
Most of science is still ultimately about matters like that, matters that have good answers, at least in principle.
Some science has veered off course, however. Every major scientific discipline (physics, biology, chemistry, etc.) has subareas where people start conflating experimental facts with opinion, aesthetics, and prejudice.
So, scientific truth is not a matter of opinion, but a lot of what is published in science is not about scientific truth.
Re:Reach (Score:3, Insightful)
Actually, no. As Alvin Plantinga has pointed out, any number of false beliefs would accomplish the same thing. For example, if the early primates wanted to be friends with lio
Utterly useless rhetoric (Score:4, Interesting)
Indeed it is. Which is precisely why our senses are so easily fooled: given stimuli that do not correspond to those seen in survival tasks in the EEA (Environment of Evolutionary Adaptedness, i.e. hunting and gathering on the African plains or whatever), our brains do not necessarily respond correctly.
Now explain why a creator would have built brains that are so subject to misdirection, geometric optical illusions, etc. Why would he/she/it have done so?
belief in evolution is self-defeating because on the supposition that evolution is responsible for our reasoning ability, we have no confidence that the deliverances of reason (i.e. the theory of evolution) correspond to reality.
And if a creator built our reasoning capabilities, how do you know that he/she/it programmed it to accurately reflect reality? We'd be seeing whatever he/she/it wanted us to see, for whatever reasons. You'll get less mileage out of this argument for creation than for evolution, even.
Pointing out that evolutionary theory itself can't guarantee the accuracy of our reasoning faculties (which is true) gets you absolutely nowhere because Creation mythology is significantly worse. Consider:
Prove the above statement wrong. You certainly can't invoke anything it says in the bible, because that -- or rather your memory of it -- was created six seconds ago as well. It says exactly what the creator wanted it to say, for reasons of his/her/its own.
As soon as you invoke a creator, falsifiability is utterly gone, your conclusions can be ANYTHING, and future argumentation is pretty much futile. Thus creation mythology serves primarily as a tool for a person to project their own emotional needs and desires into their own understanding of reality.
Fortunately, there are other ways to evaluate the accuracy of our reasoning capabilities than evolutionary theory or creation mythology. Sparing a couple thousand years of philosophy, I'll stick to the pragmatic argument: Reason seems to work. It gives us effective tools for functioning, ergo we're best off assuming that our intellect and reason is what it seems to be, and make use of it.
End note: Your sig links to a story about Antony Flew "converting to religion". You'll notice he's a self-described Deist: a philosophy that is in no way contradictory to any contemporary understanding of evolution, physics, or any other branch of the sciences. He explicitly states he doesn't believe any sort of revealed religion. How does this bolster any point in favor of creationism or any other branch of post-Enlightenment fundamentalist thought? The point is lost, because Flew explicitly still rejects all that.
Re:Utterly useless rhetoric (Score:3, Insightful)
Because if you make complaints that evolutionary theory can't explain phenomenon X, then you should make an attempt to explain how yours can. It's unreasonable to poke holes at one theory and then argue that your
Re:Utterly useless rhetoric (Score:3, Insightful)
Re:Utterly useless rhetoric (Score:3, Interesting)
These are the sort of questions that a real scientific theory of Intelligent Design would have to answer. A scientific theory must make strong predictions that are testable, and that if proved false will lead to th
Re:Reach (Score:5, Funny)
Therefore anything that anyone says is simply an opinion.
That's just your opinion.