Scientists Themselves Play Large Role In Bad Reporting 114
Hugh Pickens writes "A lot of science reporting is sensationalized nonsense, but are journalists, as a whole, really that bad at their jobs? Christie Wilcox reports that a team of French scientists have examined the language used in press releases for medical studies and found it was the scientists and their press offices that were largely to blame. As expected, they found that the media's portrayal of results was often sensationalistic. More than half of the news items they examined contained spin. But, while the researchers found a lot of over-reporting, they concluded that most of it was 'probably related to the presence of ''spin'' in conclusions of the scientific article's abstract.' It turns out that 47% of the press releases contained spin. Even more importantly, of the studies they examined, 40% of the study abstracts or conclusions did, too. When the study itself didn't contain spin to begin with, only 17% of the news items were sensationalistic, and of those, 3/4 got their hype from the press release. 'In the journal articles themselves, they found that authors spun their own results a variety of ways,' writes Wilcox. 'Most didn't acknowledge that their results were not significant or chose to focus on smaller, significant findings instead of overall non-significant ones in their abstracts and conclusions, though some contained outright inappropriate interpretations of their data.'"
Well, of course my abstract contained spin! (Score:5, Funny)
.
Re: (Score:2)
Let's face it, even the scientists don't know what quantum spin is all about.
Re: (Score:2)
Re: (Score:2)
MRI scans [wikipedia.org] couldn't exist without a thorough understanding of what quantum spin states are. Ditto for NMR spectroscopy [wikipedia.org].
It's only Natural (Score:4, Insightful)
1. Science is quite boring. By nature it's supposed to be, objective, logical, and devoid of feelings. But Scientists themselves are not typically boring people, they're humans, and humans are emotional beings.
2. Scientists aren't communications experts and suck at making dry discipline accessible to the public. Never was this more obvious than when I was in college. How many brilliant researchers really sucked at teaching? Pretty much most of them.
3. Scientists want to think their work matters, and therefore are inclined to extrapolate applications of their science to the public. When those applications get reported as a sure thing, then an exaggeration is bound to happen.
4. And of course, Science that can be show to be of great public benefit gets funding. Cha-ching!
Re:It's only Natural (Score:5, Insightful)
3 and 4 are the most concerning, as that is what peer review is for and that is where there is failure due to the large volumes of data vs time.
So, it is abused by those that just want money to do stupid things.
Re: (Score:2)
yes... must be troll to think scientists are not pure truth seekers.
If you needed any more proof there is a religion of science... marking the above as troll pretty much seals the deal.
Re: (Score:1)
I suspect that peer review actually encourages exaggeration of importance. After all, the reviewers are in the same field as the study author, and therefore inclined to believe that the niche is larger than it is. In my experience, reviewers have a much more positive response to a study that claims to have profound implications for [x] than to a study that reports findings and interprets them in the context of prior work. Reviewers are also under funding pressure, and the more important their niche is, t
Re: (Score:1)
While 4 is a problem, you have to consider the audience the article was written for. If the article is a scientific journal, which is what most scientist are writing articles for, then the spin is readily and easily recognized by other researchers as a kind of forecasting where this research might lead, rather than solid statements about the current work. In this case it may appear to be spin, but the intended audience knows what it is. On the other hand, if the article is one for the general public, the
Re: (Score:2)
There's also the fact that few scientists are in a position to lie about their results and not
Re:It's only Natural (Score:4, Insightful)
5. It's possible that scientists which include spin and get good news coverage receive additional funding the next year. Those who don't may not, and eventually end up an assistant to someone who does spin.
No idea if the above is true but if our carrot/stick system is setup this way but if it is then spin is guaranteed.
Re: (Score:2)
Re: (Score:2)
There is a lot of good science that sounds boring to the public, and more importantly to the funding agencies, so the investigators try to make it sound more exciting.
An interesting physics question about how a wave function collapses when a measurement is made becomes "quantum teleportation". Using X-ray pulses to saturated a L-shell (M?) transition in Aluminum becomes "transparent aluminum". A new technique for measuring fast chemical reactions on a surface becomes a 'breakthrough for hydrogen power".
Re: (Score:3)
4. And of course, Science that can be show to be of great public benefit gets funding. Cha-ching!
I would say number 4 should be at the top of the list--in 30pt. font and flashing bright red.
Science is supposed to be objective, above such matters as grant-whoring and self-promotion. But if such a creature actually exists, I've never met it myself.
Re: (Score:2)
Take evolution for example. I have rarely read or heard of scientists describing evolution in the most mundane but factually correct way -- the genetic change within a species resulting from natural selection*, a process that is merely the dying off of lineages that could not cope with the environmental conditions, or could not compete with other lineages -- except in textbooks. I do, however, remember guest scientists on documentaries and nature magazines wax poetically about a species' epic struggle of su
Re: (Score:2)
Not having read the article one also has to consider that there's likely a selection bias at work.
Say there are 10 studies for a journalist to choose from, 9 are all very cautious in their interpretation and communication, the remaining one plays up their result as much as possible. Which do you think is going to make the news?
The problem with science and the news is that its news, stuff we already know isn't news, it's the new and surprising stuff that's news, unfortunately in science the new and surprisin
Re: (Score:1)
Re: (Score:2)
You missed the main point: Scientific research needs to be relevant in order to qualify for funding.
Re: (Score:2)
That's nothing, my most recent paper has spin right in the title [doi.org]!
Re: (Score:2)
I found that short abstract interesting, but reading it I think I know one reason why science reporting sucks -- most reporters can't read at that level. Hell, most reporters would have trouble with the average Wikipedia post about any facet of science, let alone a PhD level paper.
A solution might be ignoring abstracts (Score:1)
Just make it standard for science reporters or editors for the science section to ignore the abstract entirely.
Re: (Score:1)
Re: (Score:1)
Re: (Score:2)
And the problem is the reporter not the scientist. I have read countless news article headings that had little to do with what the article reported, and then even less that what the actual research paper stated. News is about getting eye balls, either through subscriptions for papers or ratings for news shows.
Researchers do not need to gin up their research to gain funding. The agencies that provide funding are run by actual researchers th
Re: (Score:1)
First, fu for being a dick... no really... drink molten glass.
Second, researchers don't need to exaggerate? Then why do they... frequently?
This has been an ongoing and system wide issue in the halls of science.
Do journalists exaggerate as well? Oh god yes... more often then not frankly. But it's less acceptable for scientists to do it. And in any case, my only suggestion was that the abstract was ignored and have it be uncitable as a source. So the paper proper can be cited. But the abstract cannot be cited
Re: (Score:1)
This just in...media reports exciting news. (Score:5, Insightful)
Whereas the mundane gets nothing. For every person murdered, or in a car accident, there are thousands in the area who had a humdrum day. For every house that burns down, thousands don't.
People who hear about these bad things and think the world is going to heck, are forgetting that nobody cares to hear about nothing happening.
Attention whoring for funding (Score:5, Insightful)
Fund science like you fund business, and it becomes an exercise in marketing and hot topic buzzwords.
OK, it might take more energy to make a solar panel than we'll ever get back from it, but look at the economies of scale that we're leveraging!
And self promoting .... (Score:1)
Let's face it, to be successful in one's lifetime in any field requires some sort of self-promotion. I'm sure having a well known name makes it a LOT easier to get funding, tenure, book deals, etc ...
Re: (Score:3, Insightful)
As someone who is a Ph.D. student and research assistant, "whoring for funding" is pretty much SOP. It's pathetic and I hate it.
Re:Attention whoring for funding (Score:4, Insightful)
principle of least remorse (Score:3)
What a horrible, defeatist attitude. I can't stand bugs in software (the vast majority exist because low standards are cheap). So I should chose a different profession?
Re: (Score:2)
Re: (Score:2)
Replace 'bug' with 'poorly designed software' then.
Just because it's part of the design doesn't mean that design is a good one. You won't ever find a better way of doing things if you don't see anything wrong with the current one.
Re: (Score:2)
I've known since 1978 that "bugs were part of the job" and yet I persist.
I'd go further than that and say bugs ARE the job, when they stop being reported your product is dead, your business/corporate-department probably died earlier than your product. Note that by "bugs" I mean the software/documents don't do/say whatever the guy spending the money wants them to do/say. As I'm sure you know, a professional works on the basis that giving the customer/boss what they want does not necessarily mean giving them what they ask for, so right there you have your first major "bug" to iron
Re: (Score:2)
Ah, this explains why Microsoft is still around ....
Re: (Score:2)
I did choose another profession. Do I still burn for physics research and progress? Yes. Did I consider myself above the petty politics that are involved with getting funding? Yes. Did ANY of my classmates that originally set out to do research in Physics end up doing that? No. A couple of patent lawyers, a couple of quantitative analysts, a few went engineering, and some went in to IT, but with the politicking at the labs and the sensationalized self-promotion that you have to do, it feels like bein
Re: (Score:1)
My advisor is Chinese and has a job in China in addition to his one here. He knows scientists in virtually every country where there is science. Just last week, he said that he believes the (research/science) system in the US is the best in the world and that's why people want to come here. I take his statement very much to heart: I was convinced it was only a matter of time before he left his job in the US entirely and returned to China. What he said makes it very clear he will not do that.
So, things h
Re: (Score:3)
Re: (Score:2, Insightful)
Business when done properly will have a profitable result.
Science when done properly will have either a positive or negative result.
The scientific process for the Facebook Generation...
I have this crazy idea.
How to measure if my crazy idea works.
Lets run tests that measure my crazy idea.
Does the tests match my expected results?
If (Not even close) { Your idea was really crazy, try an other one }
If (close) { Your idea may have some backing but will need to be tweaked }
if (spot on) { These results may be a flu
Re:Attention whoring for funding (Score:5, Informative)
OK, it might take more energy to make a solar panel than we'll ever get back from it, but ...
Will you JUST FUCKING STOP spreading this lie? The energy payback time for photovoltaic modules according to most studies is between 1-4 years [nrel.gov], depending on the material and manufacturing process used. Their technical lifetime is 25 years or more.
(I know I'm late to the party and hardly anyone will read this, but this is for the three of you who will.)
Re: (Score:2)
Heh, I was thinking the same.
Re: (Score:2)
... are journalists, as a whole, really that bad at their jobs? Christie Wilcox reports that ...
I think we could probably patent the application of the uncertainty principle to journalism. Yeah I know skepticism has prior art, but if we don't use that word, your average examiner won't know about it....
In other news... (Score:5, Insightful)
So basically, most reporters just regurgitate press releases rather than doing any of that actual journalism stuff. That's not unique to science/medical reporting. It happens in political reporting, business reporting, hell even sports reporting. The bad science reporting is just more obvious because it's easier to debunk.
Re: (Score:1)
The bad science reporting is just more obvious because it's the local hobby.
All bad reporting is easy to debunk by anyone who takes a little time to get mildly familiar with the subjects. You notice bad science reporting because it is your hobby. I know that I do not notice bad sports reporting because I pay less attention to sports than I do to a youtube video of a lava lamp.
Re: (Score:2)
The system selects for CONmen and Shysters (Score:4, Interesting)
Re: (Score:1)
Re: (Score:2)
Fraud will not work in peer reviewed work, let alone grant applications - it's too easy to spot
Fraud, at least in the form of bending the truth, is common in both peer reviewed work and in grant applications (and particularly encouraged in the latter). The reality of funding is that scientists need to be increasingly devious in order to make the funding body believe that their work is more important and deserves to be funded over that other group of scientists who are doing similar work (and engaging in similar deception).
I have had trouble thinking of an appropriate solution to this. Most research t
Re: (Score:1)
Re: (Score:2)
Re: (Score:2, Insightful)
How that claim became your above "blame the scientist", I'll leave as an exercise for the reader.
Re: (Score:2)
Well I would blame society on a whole.
But it is important to try to get groups to realize that they are part of the problem.
Whenever you think you are part of a group that is some how immune to being part of the problem, then you open yourself up to be a bigger part of the problem.
For example...
The Catholic Priest Scandal: Priests for generations, heck for thousands of years, have been considered by the public to better people then the rest of us. So claims against them will suffer punishment from the vict
Solar Cells, Anybody? (Score:3, Insightful)
Re: (Score:3)
Press Releases? (Score:5, Informative)
To be fair, university press releases are not written by the scientists who did the research, and in my experience the scientist often doesn't even get the chance to proof and correct them. I myself had my 15 minutes of international fame several years ago (the phone literally didn't stop ringing, interview requests from around the world, etc), all on account of a shockingly inaccurate press release from the university about some interesting but not earth-shattering research that I did.
I wonder (Score:2)
Are are journals... (Score:5, Funny)
The question is not "are are journals, as a whole, really that bad" ... the question is...
IS OUR CHILDREN LEARNING YET?!!
Re: (Score:2)
Don't you mean:
"Are are children learning yet?!!"
From the Study's Abstract (Score:3, Interesting)
Re: (Score:2)
Surprising xkcd link (Score:3)
Science news cycle [phdcomics.com]
Re:Surprising xkcd link (Score:5, Funny)
Surprising xkcd link
Is that "surprising" in the sense of "not an"?
Whew (Score:1)
I guess scientists are supposed to do all of the work now, are they? They have to do the science, write the papers, market it for their funders, write the articles for the news corporations, AND be the fall-guys when something isn't 100% accurate?
Whew. I'm glad I didn't remain a researcher.
Parable (Score:2, Interesting)
A congressman was touring his district when he came upon a bunch of people in a big field with bows and arrows. They were all firing arrows in all different directions.
"What are you doing?" asked the Congressman.
"We are shooting arrows," said the archers.
"But there is nothing to shoot at," said the Congressman. "Those arrows are provided at taxpayer expense! How dare you waste them in this way?"
"Well," said the archers, "as you can see, we are very skilled archers. We can shoot arrows so far that they go ov
Key phrase "AND THEIR PRESS OFFICES" (Score:3)
Hardly "Scientists themselves", is it?
Re: (Score:2)
Bingo. As soon as I thought that, I immediately thought of this obligatory xkc^H^H^H PHD comic: http://www.phdcomics.com/comics/archive.php?comicid=1174 [phdcomics.com]
I hate Odds Ratios (Score:2)
One of the worst "bad abstract tricks" is putting your findings as Odds Ratios. What's an Odds Ratio? You probably know that the "probability" of an event is "Event over Total". The probability of rolling a 6 on a standard die is 1/6. The "odds" of an event is "Event to Not Event". The odds of rolling a 6 are not 1:6, they are 1:5 for (or more often said, 5:1 against). So then the odds ratio (OR) of two groups is the ratio of ratios, or the ratio of the odds for one event compared to the odds of anot
Re: (Score:2)
Wait a minute: how the heck OR 20:1 implies probability ratio 3.5:1? The probability of event A is 1/21 = 0.048, probability of B is 20/21 = 0.95. The ratio p(A)/p(B) = 0.051 or ~1/20. It's seem that the OR approaches probability ratio as the OR goes away from 1:1. It's seem to me that OR is farthest away from probability ratio when OR = 1:1. Or else I'm not getting what you mean by OR 20:1 implying 3.5:1 probability ratio. Probability ratio of *what*?
Three words: Dr John Ioannidis (Score:2)
This article is very well worth reading:
http://www.theatlantic.com/magazine/print/2010/11/lies-damned-lies-and-medical-science/308269/# [theatlantic.com]
Abstracts vs. articles in medical science (Score:1)
If all you look at is the abstract and "conclusions", of course you're going to get an unbalanced view of what the study said. Think of all the other information that is contained in the body of the paper. There's a discussion of the methodological limitations of the study, there's a discussion of all the outcome measures which DIDN'T reach statistical significance, there's a discussion of adverse events, and there's usually also a discussion of where this study fits into our knowledge of the topic as a w
Reporters need to do their job (Score:2)
This seems like a weak attempt to shift the blame for bad reporters. Their job is to get at the facts and report what is really true. That's what reporters do - at least if they're any good. So scientific press releases contain spin? Shocking! Just like press releases in absolutely every other field. Any reporter who just parrots a press release without understanding it and getting at the truth is a bad reporter.
Yes, science is complicated. Yes, it takes specialized knowledge to understand. Just lik
I think they ARE ARE (Score:1)
Re: (Score:2)
You must be new here.
Oh wait, you are. Rage on.
Re: (Score:2)