Follow Slashdot blog updates by subscribing to our blog RSS feed


Forgot your password?
Check out the new SourceForge HTML5 internet speed test! No Flash necessary and runs on all devices. ×

Submission + - Is Something Wrong with the Scientific Method? 6

Hugh Pickens writes: "Jonah Lehrer has an interesting article in the New Yorker reporting that all sorts of well-established, multiply confirmed findings in science have started to look increasingly uncertain as they cannot be replicated. This phenomenon doesn’t yet have an official name, but it’s occurring across a wide range of fields, from psychology to ecology and in the field of medicine, the phenomenon seems extremely widespread, affecting not only antipsychotics but also therapies ranging from cardiac stents to Vitamin E and antidepressants. “One of my mentors told me that my real mistake was trying to replicate my work," says researcher Jonathon Schooler. "He told me doing that was just setting myself up for disappointment.” For many scientists, the effect is especially troubling because of what it exposes about the scientific process. "If replication is what separates the rigor of science from the squishiness of pseudoscience, where do we put all these rigorously validated findings that can no longer be proved?" writes Lehrer. "Which results should we believe?" Francis Bacon, the early-modern philosopher and pioneer of the scientific method, once declared that experiments were essential, because they allowed us to “put nature to the question" but it now appears that nature often gives us different answers. According to John Ioannidis, author of “Why Most Published Research Findings Are False," the main problem is that too many researchers engage in what he calls “significance chasing,” or finding ways to interpret the data so that it passes the statistical test of significance—the ninety-five-per-cent boundary invented by Ronald Fisher. "The scientists are so eager to pass this magical test that they start playing around with the numbers, trying to find anything that seems worthy,”"
This discussion was created for logged-in users only, but now has been archived. No new comments can be posted.

Is Something Wrong with the Scientific Method?

Comments Filter:
  • I come across this all the time, but there's no need to panic. Basic physics and engineering are not threatened, just the justifications for the fad-du-jour in medicine or sociology or other semi-empirical studies. Alcohol is good for you; no, it's bad for you; no, it's very good for you in moderation; no, any amount is bad; and so forth. All of the conclusions on alcohol vs health are at 95% confidence, too, given the appalling misapplication of statistics to the relevant data (typically from inadequate st
    • In medical science especially, the fashion for meta-analyses can't help. Aggregating data from different studies results in all sorts of artifacts appearing.

  • There's something wrong with some scientists. TFS cites problems in psychology, ecology and medicine. That's no surprise. As it has been pointed in the above comment , there's a quite broad misuse of statistics, so in the end, the saying "lies, damn lies and statistics" comes true, since statistics are interpreted the way the scientist wants to better fit his hypothesis. In cosmology and physics there's also something going on: evidence for a pre-Big-Bang universe, evidence of multiverses, dark matter signa
    • Many if not most published climate-change papers describe some interesting results and then go on to draw broad conclusions that don't follow from the data, that are at best hypothesis the data suggests experimenting with. What percentage of scientists in a field have to get it wrong before the problem is with the methodologies and behavior accepted during peer review rather than the specific individuals?

  • In other words the "scientists" are sloppy and they produce results that fit their pet theory... Nothing new really, been going on for a long long time.

    It's troubling though that it's so widespread but I guess most of them need to produce results so they can justify their need for money/tenure etc. and that usually means taking shortcuts and fudge the data.

  • The problem is that many scientists, just like many politicians, can be bought and sold for the right price. The problem is that we live in a world that cares more about money than about science or culture or the public or the environment...

As far as we know, our computer has never had an undetected error. -- Weisert