How Science Goes Wrong

dryriver sends this article from the Economist: "A simple idea underpins science: 'trust, but verify'. Results should always be subject to challenge from experiment. That simple but powerful idea has generated a vast body of knowledge. Since its birth in the 17th century, modern science has changed the world beyond recognition, and overwhelmingly for the better. But success can breed complacency. Modern scientists are doing too much trusting and not enough verifying — to the detriment of the whole of science, and of humanity. Too many of the findings that fill the academic ether are the result of shoddy experiments or poor analysis (see article). A rule of thumb among biotechnology venture-capitalists is that half of published research cannot be replicated. Even that may be optimistic. Last year researchers at one biotech firm, Amgen, found they could reproduce just six of 53 'landmark' studies in cancer research. Earlier, a group at Bayer, a drug company, managed to repeat just a quarter of 67 similarly important papers. A leading computer scientist frets that three-quarters of papers in his subfield are bunk. In 2000-10 roughly 80,000 patients took part in clinical trials based on research that was later retracted because of mistakes or improprieties. Even when flawed research does not put people's lives at risk — and much of it is too far from the market to do so — it squanders money and the efforts of some of the world's best minds. The opportunity costs of stymied progress are hard to quantify, but they are likely to be vast. And they could be rising."

Re:Greed

[Anonymous Coward]

FYI, 'trust, but verify' is also a great rule of thumb for spell-check.

Can someone verify the numbers?

[ChronoReverse]

Are the numbers from this article just pulled out of a hat?

Re:Yeah, but it does depend on the area of science

[John Allsup]

That's why if you look through, say the TRIP database of medical studies, what you see is synopses of what look to be 'it really works, honest!' stories reminiscent of fake Amazon reviews, and little in the way of 'research downers' to make it look legit.  The kind of success rate you see in these research papers is like the poll outcomes you get in stuffed ballot box elections in the middle of some countries whose dictators don't quite understand how democracy is supposed to work.

Re:All scientific conclusions should be questioned

[Anonymous Coward]

Except those from Climate scientist... All of those conclusions should be taken on faith!!!

When I read this comment, I was laughing until I realized that it was modded down to -1. Clearly, you struck a nerve here at Slashdot.

Let it be know: On Slashdot, YOU SHALL NOT QUESTION THE FAITH OF CLIMATE CHANGE!!! So it is written so shall it be!!!

Re: Anti-science? See, now you have proof!

[Anonymous Coward]

It was autumn, and the Indians on the remote reservation asked their new Chief if the winter was going to be cold or mild. Since he was an Indian Chief in a modern society, he had never been taught the old secrets, and when he looked at the sky, he couldn't tell what the weather was going to be.

Nevertheless, to be on the safe side, he replied to his tribe that
the winter was indeed going to be cold and that the members of the
village should collect wood to be prepared.

But also being a practical leader, after several days he got an idea. He went to the phone booth, called the National Weather Service and asked, "Is the coming winter going to be cold?"

"It looks like this winter is going to be quite cold indeed," the meteorologist at the weather service responded.

So the Chief went back to his people and told them to collect even more wood in order to be prepared. One week later he called the National Weather Service again. "Is it going to be a very cold winter?"

"Yes," the man at National Weather Service again replied, "it's going to be a very cold winter."

The Chief again went back to his people and ordered them to collect every scrap of wood they could find. Two weeks later he called the National Weather Service again. "Are you absolutely sure that the winter is going to be very cold?"

"Absolutely," the man replied. "It's going to be one of the coldest winters ever."

"How can you be so sure?" the Chief asked.

The weather man replied, "The Indians are collecting wood like crazy."