Climategate and the Need For Greater Scientific Openness 701
The Guardian follows up on the recent news that CRU climate scientists were cleared of scientific misconduct with an article that focuses on how the controversy could have been avoided, and public trust retained, had the scientists made more of an effort to be open about their research. You may recall our discussion of a report from Pennsylvania State University; that was followed by another review with similar conclusions. Quoting:
"The review, led by Sir Muir Russell, does not mention the media. Instead, it examines the reaction of the scientists at the UEA's Climatic Research Unit (CRU) to the pressure exerted by bloggers: 'An important feature of the blogosphere is the extent to which it demands openness and access to data. A failure to recognize this and to act appropriately can lead to immense reputational damage by feeding allegations of cover-up.' The review adds: 'We found a lack of recognition of the extent to which earlier action to release information might have minimized the problems.' Pressure on the scientists, whose once esoteric work creating records of past temperatures had gained global significance, was intense. In 2005, CRU head Phil Jones replied to a request: 'We have 25 or so years invested in the work. Why should I make the data available to you, when your aim is to try and find something wrong with it?' But, the review implies, the more they blocked, the more the Freedom of Information requests flooded in."
Re:Response (Score:3, Informative)
The data is all available to the public from NASA and the NOAA. Go run your computations and get back to us on your results, professor.
The Guardian hosts a debate on Climategate (Score:3, Informative)
Re:The investigation was a farce (Score:4, Informative)
But the data IS available and it WAS available, they didn't even fudge the data. The only accusation made against them was that they started getting obstinate and refusing to give the data to climate doubters and the chart they had in the WMO report was misleading if you didn't read the report carefully.
Re:Response (Score:5, Informative)
Not only that, these "found no malfeasance" results are absurd on their face. They DID hide, throw away and improperly manipulate the data.
No, actually, they didn't.
That's the point of the review-- it turns out they didn't hide, throw away, or manipulate the data.
They did publish one figure (out of a total of many hundred figures in a large number of articles) that was misleading-- but (here's the interesting thing) they explained exactly what they did and why in the figure. If you read the figure, but not the text explaining it, it might have been misleading-- or might not; there is some real controversy about tree ring data, and it's pretty clear that they thought that they were presenting the data in the clearest form. In any case, if you read the text, you would have known exactly what the figure was graphing.
In fact, Nature (the journal in which the arguably-misleading figure was published) had their own review, which concluded that there was no need for a correction, because the article did explain exactly what the figure showed, and why it showed that particular data; it did not need a correction because it was not incorrect.
Re:Response (Score:3, Informative)
Re:That's how science works... (Score:5, Informative)
This wasn't a case of "you will give us your readings as you measure them". This was a case of "you published these figures in this study, can we now see how you arrived at these figures?".
This is a canard (Score:5, Informative)
Popper's notion of science is, frankly, obsolete. It was already obsolete when I was reading Philosophy of Science in the 1970s. He envisages a world in which falsifying an hypothesis invalidates a theory. But modern science - and this includes quantum mechanics as well as climatology - depends on statistical analysis and probability theory. You could almost say that when Schroedinger and Heisenberg defined the Uncertainty Principle and the probabilitistic Wave Equation, physics changed in a way that obsoleted Popper and the whole Victorian idea of science.
Jones is replying to people who don't want to take large amounts of data and mine them, but to find single errors and then claim that this invalidates the lot. He was actually right to tell them to get stuffed - but, because we live in a world dominated by PR and spin, this was misused against him. You are demonstrating the effect of this - you clearly have never read Popper, but you're trying to use a sound-bite as an argument.
Exxon-Mobil funding [Re:Impressive] (Score:5, Informative)
No, he just doesn't want a bunch of people funded by exxon-mobil selectively quoting tiny portions of his data to support bullshit positions,
Funnily enough, none of the people who asked for the data were funded by Exxon-Mobil. Its boring how facts get submerged by a straightforward lie.
Uh, except actually they were. It's not even particualry a secret-- take a look at who funds the "Heartland Institute" (Hint: Exxon Mobil). Google the "American Petroleum Institute [euronet.nl]".
For a while they were even offering a payment of ten thousand dollars [politicalaffairs.net] to every scientist who published a paper casting doubt on global warming. (They stopped this when it got publicized in the Guardian.)
Openness vs Harrasment (Score:5, Informative)
Internal communications of the IPCC to authors of the scientific review now say the following [nytimes.com]:
As a climate scientist and a computer scientist and an advocate for openness and replicability my position is greatly weakened by people using "openness" as an excuse for harrassment and witch-hunting.
The inevitable short result of this approach to openness is going to be that scientists will do as much work as possible on their laptops and their yahoo email accounts. Using their funded platforms will be only for production runs and final drafts of publications; this will minimize the amount of exposure of their actual work to hostile parties. We will also see far fewer really good people getting into work with any controversy, lest they be subjected to public abuse; eventually only work of little consequence will attract the intellectually adventurous.
I really want the open science movement to be about making science more accessible and more appealing and more part of the culture. This subversion of the open science movement in the name of derailing climate science, which in turn hides the real intent of delaying climate policy until all the fossil reserves are cashed in, is a disaster on more fronts than one. One unfortunate aspect is that it drives important segments of the scientific community to treat the open science movement as a threat to science. Advocates of open science would do well to think twice about the motivations and actions of this gang.
Re:not cleared (Score:4, Informative)
Yes, it is. Except the report did not claim anywhere that it was intentional. Nor was it, considering that the dropping of tree ring data was made explicit in the original paper where the graph was used:
In one of the most notorious leaked e-mails, Jones, referring to the WMO report graph, described how he had "just completed Mike's trick of adding in the real temps to each series for the last 20 years". Jones was referring to the fact that climatologist Michael Mann of Pennsylvania State University in University Park had used direct temperature measurements to reconstruct temperatures over the past 20 years or so in a graph in an earlier Nature paper [2] [nature.com]. However, while Mann and his colleagues had clearly labelled which temperature lines were derived from direct measurements and which referred to proxy data, the graph submitted by Jones for the WMO report did not.
- UK climate data were not tampered with [nature.com]
If they were intentionally misleading the public, why had the same graph already been published with the missing information?
"What does bother me is the attempt to pass off the results of incompetent software engineering as valid science."
The evidence of your post tells me that the misrepresentation of facts doesn't seem to bother you at all.
Re:Impressive (Score:5, Informative)
None of the people who asked for the data were amateurs. But more importantly, the data that Jones was trying to hide had already been lost - by Jones.
From Ars [arstechnica.com]
Data they were trying to hide
"In order to test the principal allegations of withholding data and making inappropriate adjustments, the Review undertook its own trial analysis of land station temperature data. The goal was to determine whether it is possible for an independent researcher to (a) obtain primary data and (b) to analyse it in order to produce independent temperature trend results. This study was intended only to test the feasibility of conducting such a process, and not to generate scientific conclusions." In other words, if we can do it, anyone can.
They found that the data was readily available at at least three different websites. They downloaded the data, selected every station that had an adequate amount of data and performed some smoothing and spatial averaging operations on them. In effect, they replicated the CRU's main research results, producing nearly identical instrumental temperature records, in very little time.
Broken FoIA system
The key findings here are pretty bleak. Basically, the UEA logged FoIA requests, but that was about it. After that, everything was down to the individual researchers figuring out if the data had to be, or, indeed, should be released, and then figuring out how to release it properly. Essentially, the entire system was dysfunctional, and the CRU made no attempt to make life easier for anyone.
In my opinion, it seems like bureaucratic incompetence rather than malice or ideology.
Re:Impressive (Score:3, Informative)
None of the people who asked for the data were amateurs.
You're kidding, right? The people who used FOIA requests to get the data were ALL amateurs. And Jones and others knew what happens when you release the
data to amateurs. When Mann released data to McIntyre, he got endless requests for explanations on the format, and the meaning of this or that piece of data.
Scientists are used to having other scientists requesting data, but that means the guy requesting is actually qualified to understand the data.
Most of the FOIA requests that were made had one purpose only: to harass the scientists.
But more importantly, the data that Jones was trying to hide had already been lost - by Jones.
Jones only lost his copies (in the eighties, when keeping that much data was very expensive) All the data was/is still
available from the original sources (individual state weather services), which is where those interested SHOULD request them from.
That's a lie and not insightful. (Score:5, Informative)
We don't get paid by paper. We get funding to research a specific things. And a whole lab might get a funding of a few million for a subject, over a few years, but nobody PERSONALLY get that much money as a scientist. That's not even counting the TIME spent on doing that paper. So take your 1 to 1000 factor and stuff it. 10 grand per paper is an ENORMOUS sum of money for the average scientist.
Re:Impressive (Score:5, Informative)
Bullshit to your bullshit. Have you read anything on ClimateAudit? Steve McIntyre is not claiming to be a climatologist, but he had issues with the analysis and handling of proxy data. And none of these "inquiries" ever addressed those issues. In fact, he has had some success in getting corrections made to published work. How many inquires asked him about his complaints? I haven't seen any.
On top of that, Steve McIntyre has never claimed that his requests should be seen as a statement on climate change. Please don't take my word for it. Go to climate audit and read his own words.
Re:Response (Score:5, Informative)
The problem is that the tree rings accurately track temperatures until fairly recently, when some of the trees in some of the forests show "divergence," while others continue to correctly track temperature. So the trees do not merely diverge from the temperature record, but from one another.
Obviously, the fact that such divergence can occur limits confidence in the use of tree rings for climate reconstruction. On the other hand, it also shows that such divergence can be detected by comparing results from different forests. The fact that the historical tree ring data does not exhibit this kind of discrepancy between different sets of trees argues against divergence being a problem.
Here is a review [sciencedirect.com] that discusses the divergence problem and possible causes.
Re:Impressive (Score:4, Informative)
Re:Can you spell W H I T E W A S H ? (Score:3, Informative)
Blogs are not the place for reputable criticism. I could just as easily point you to http://realclimate.org [realclimate.org] which are actually scientists at CRU, NOAA, and NASA to debunk all of your conspiracy theories. I would like (and have searched for, but maybe just not hard enough) published, peer reviewed evidence that climate science as we know it is actually incorrect. Not just Mann, et al (who do sound like they are douchebags), but the entire rest of the field.
Re:Impressive (Score:5, Informative)
Bullshit. Steve McIntyre is NOT a climatologist, and neither is Anthony Watts.
Nope. Steve is a statistical analyst with a degree in mathematics and Anthony Watts is a meteorologist. The data was fed into statistical models. They were and are very well qualified and able to analyze what was done.
Which is why the Hockey Team was so mortally afraid of them that Jones and Co destroyed pertinent information requested under FOIA.
Quite what this has to do with politics only your psychiatrist can tell.
Re:Exxon-Mobil funding [Re:Impressive] (Score:3, Informative)
Being paid to research is far different from being paid to produce a specific result.
Re:Impressive (Score:3, Informative)
No, he's an ex-TV weather presenter. I realize some people call that a meteorologist, but it's not the same thing at all.
1970s cooling consensus = myth (Score:5, Informative)
Indeed, either they now have it wrong, or they did less than a century ago, when they concluded that the trend was towards "cooling".
Myth: In the '70s, the best scientific knowledge indicated that the Earth's climate was headed for a trend of long-term cooling, rather than warming.
Fact: The 1970s global cooling scare was little more than a runaway media circle jerk.
The Myth of the 1970s Global Cooling Scientific Consensus (PDF) [scienceblogs.com]
A more TL;DR-friendly article on the topic [newscientist.com]
Re:Out of whose budget? (Score:3, Informative)
Who funds the process of replying?
Generally the requester is charged, however, it depends upon the FOIA request. There are some exemptions for certain types of data and certain types of requests. For example, news media are given extra leeway under FOIA.
Re:Response (Score:3, Informative)
Re:That's how science works... (Score:5, Informative)
Don't you be silly. Nearly all the raw temperature station data has been available for years. It's called the Global Historical Climatology Network. Go and look it up.
Anybody wanting to construct their own global or regional temperature records has been able to do so. In fact NASA GISS temperature record is constructed from this freely available dataset.
The fact that after years of whining, the skeptics never did so says one of two things - they are incompetent or for not hard to fathom reasons, unwilling. Far easier and suited their purposes better to defame working scientists.
In fact in the last 6-9 months, several science bloggers have done so, including (at last) one skeptic. And the results all are in close agreement with the published HAdCrut, GISS and NCDC temperature records.
So exactly what data is withheld that actually affect in any meaningful way the core conclusions of climate science?
Re:Impressive (Score:3, Informative)
That's funny you mention "big oil", because BP is a big financial backer of the CRU.
Guess who stand to benefit from a cap and trade regime? The oil companies. They are the players primed to exploit the trading of carbon credits.
Re:!Science (Score:5, Informative)
In your post above, what's missing is the bit where you explain what was wrong with Steve McIntyre's analysis.
I don't need to. Others have done it for me. Except those others happen to be climate scientists, so I guess that doesn't count.
Because although you delight in the ad-hominem against him, you will note that his criticisms are hard to dispute, particularly the excellent work he did on Briffa and the Yamal series.
No they're not. Real Climate article http://www.realclimate.org/index.php/archives/2009/09/hey-ya-mal/ [realclimate.org]
And at the end there is even a convenient link to a published peer reviewed research article refuting the claims.
The problem here is that none of the inquires bothered to interview those in a position to "verify" the claims. Indeed, I don't believe any of them even asked McIntyre for a statement, even though he was the subject of a lot of the bile in the emails.
And what claims would those be exactly? And why would thy ask McIntyre for a statement? Unlike the jackass who dumped their mail server and selected only the emails that he/she/it wanted to show, the investigators had access to all of them, plus whatever else they wanted.
You obey the law and honour the request. Your ego is not a higher authority on these matters, unless you're as arrogant and self-regarding as the Climate-Gate clique and their supporters
I never said he made the right decision. People hold up that quote and turn him into the "Evil Climate Conspiracy Monster". He had his reasons for doing what he did, and it had a lot more to do with the feeling being unnecessarily harassed than "hiding the data".
If your data, methods and claims are water-tight, you've got nothing to worry about.
Oh my. You are so naive.
It took years of fighting before anything was done about smoking. It took years of fighting before anything was done about acid rain. It took years of fighting before anything was done about CFCs and the ozone hole. In all cases there was "water-tight" research (and lot's of it) showing that bad shit was happening. But that doesn't matter when you have a billion dollar PR machine on your side. You don't even need to refute the research. All you need to do is plant a little seed of doubt and all that water-tight science won't mean jack shit in front of the public or congress.
You can't show research proving that something a multi-billion dollar conglomerate is doing is bad and expect your research to be the end of the discussion. They will attack you, drag you through the mud, crucify you, and leave you to the crows.
If they're cobbled together from poorly documented, poorly maintained, part-deleted data, with dubious analysis, as was the case here, then the claims *you* are making, upon which trillions of dollars depend, should not be taken seriously.
How many times does this need to be repeated? CRU != Climate Science Community. It doesn't. Nobody is making a trillion dollar decision based soley on the research done by CRU. No one. Really.
However, when you have thousands of climate scientists from every major national research group on the planet all reporting research that indicates the same thing, maybe, just maybe, people should take notice.
Even if CRU falsified EVERYTHING, there's still thousands upon thousands of research papers out there all showing the same damn thing.
You're also making a lot of claims in regards to the quality of their research, which really you don't seem to know much about. You don't get published in research journals by putting a pile a crap together and sending it in. You won't even make it past the initial review.
Climate science, like any other science branch, is highly competitive. If you send in some POS paper other researchers will be m
Re:!Science (Score:2, Informative)
Ozone hole has turned out to be a natural phenomena not related CFC. (Never mind the pesky volcano spewing all sorts of ozone-depleting stuff near the damn ozone hole...)
Acid rain -- noone, anywhere, ever even slightly inconvenienced by the "acid rain".
While it is true that smoking can cause lung cancer, you might want to look up the rate of lung cancer in non-smokers...
Re:Impressive (Score:3, Informative)
Yeah right, that's never what happens. These kooks latch on to whatever data you give them that they think proves their point, and none of your explanations ever budge them from their a-priori position.
That's why Mann didn't want to release his data; it wasn't the first time that that jerk McIntyre had asked him for it. The first time, Mann gave it willingly - and was then amazed at how it was misinterpreted, because McIntyre doesn't have the background to do this sort of thing. Seriously, look up his qualifications - he's got BS in Mathematics from the 1960s, and that's it.
That's why Mann was apathetic towards the idea of giving McIntyre (or really any of those people) his data - they just don't know what to do with it. Of course, now that this has become an issue, he's made the data publicly available [psu.edu], but for some reason you don't see McIntyre talking about that.
Re:Impressive (Score:3, Informative)
Albert Einstein was an amateur in 1905, the year he released Special Relativity, his work on brownian motion, his work on the photoelectric effect and the equivalence of matter and energy.
No, he wasn't. He was a doctor of physics, the exact field in which he published his work. He only lacked a position in academia - he was employed by the patent office rather than the university.
Even if they were amateurs in a useful sense, you cannot call either McIntyre or Watts unknowledgeable about the subject. Not without the rest of us laughing at you, anyway.
At first, looking at how they requested input data from scientists, I thought McIntyre was inexperienced in science. The normal way to get input data is to look up the sources cited in the paper you want to verify, and get them from there. That way, you are requesting someone's results (which they are always pleased to give) and not someone's input (which they are under no obligation to give).
I no longer think that - not since it turned out that McIntyre DID get the data from the original author (Yamal), but kept harassing Mann for the same data, anyway. That means he was not after the data.
Re:Do as i say not as i do (Score:5, Informative)
Especially since Co2 is immaterial to global warming since the primary mediator of heat in the atmosphere... wait for it... is WATER!
Closing open windows is immaterial to temperature inside a house because the primary mediator for heat in a house... wait for it... is WALLS!
There is indeed a huge quantity of water vapor in the atmosphere, and it is indeed the gas with the single largest impact on trapping thermal radiation. However there are two key points. The first point is that the existing warming effect is 50 degrees F. So yes water is the "primary mediator" towards the existing 50 degree greenhouse effect. Without water and other natural natural level greenhouse gases the earth would be 50 degrees colder than current temperatures. Without the natural existing 50 degree greenhouse effect most of the planet would be covered in ice. The second key point is that there are different frequencies of thermal radiation, like different colors of light. At certain "colors" of heat radiation water is completely "black". The vast quantity of water vapor traps that portion of heat energy almost completely. Increasing or decreasing the amount of water vapor has little effect because the quantity of water vapor already traps that heat energy almost completely. At those frequencies the water vapor acts like the walls of a house. However at other frequencies, other "colors" of thermal radiation, water vapor is completely transparent to heat energy. Those frequencies are like holes in the walls, they are like open windows in the walls. Heat energy does freely escape and allow the earth to cool back down at those frequencies, through those open windows. This issue is that CO2 is "black" at those frequencies. Dumping CO2 into the atmosphere covers up those "open windows" that exist in H20. Dumping CO2 into the atmosphere is like closing the windows in a house and nailing an extra plywood wall over them.
So yes H20 is the gas with the single largest greenhouse effect, and that effect is already 50 degrees. H20 has "open windows" and H2O has zero ability to trap heat at certain frequencies. Dumping CO2 into the atmosphere and closing those open windows WILL have a very real effect increasing the greenhouse warming several degrees about the existing 50 degree greenhouse warming.
This is basic physics. There is absolutely no scientific dispute over the physics. There is absolutely no scientific dispute that increasing CO2 levels in the atmosphere does have the effect of trapping heat in this manner. None. This Anthropogenic Global Warming effect is absolutely indisputable basic physics. This effect absolutely is significant because CO2 is trapping frequencies of thermal radiation that water doesn't affect at all.
Global Warming is a social controversy and a political controversy. The fundamental facts of Global Warming are not a scientific controversy.
-
Re:Impressive (Score:3, Informative)
I have, and I could feel it sucking the intelligence out through my pupils.
"he had issues with the analysis and handling of proxy data. And none of these "inquiries" ever addressed those issues."
Not only have you missed the section on tree-rings in the Muir report from TFS. You must also have missed the Spanish ^H^H^H^H^H^H senate inquistion into Mann's "Hockey stick". The National Academies of Science were asked to adjudicate on the veracity of his Nature paper (since they have at their disposal most of the planets top statistical experts), their sworn testimony [nationalacademies.org] came down clearly in favour of Mann. They did however make some criticisims about Mann's confidence levels for tempratures further back than (IIRC) 800yrs. To his credit Mann's addressed those criticisims and extended his reconstruction in a paper publised in Science (the internationally recognised journal published by his critics)
The whole idea of cliamte audit is nonesense, McIntyre's two peer-reviewed papers were published in obscure non-ISI listed journals and did not stand the test of time. When he realised he could not pass muster with the standard scientific audit of peer-review he went of on a crusade to "audit climate science", naturally he appointed himself as chief auditor. In other words he spat the dummy and took himself out of the science game. He is now just another crank heckling "those who can" from a his blog and harrasing them with an avalanche of FOI requests, mostly for stuff that is already readily available in the litrature.
Aside from aligning himself with the scumbag anti-science lobbyists at the Heartland Institute, CEI, etc. Sourcewatch throws some further light on the possible motivations for his propogandist rants...
"McIntyre was also exposed for having unreported ties to CGX Energy, Inc., an oil and gas exploration company, which listed McIntyre as a "strategic advisor." He is the former President of Dumont Nickel Inc., and was President of Northwest Exploration Company Limited, the predecessor company to CGX Energy Inc."
You've been played for a sucker my friend, but don't dispair McIntyre is an accomplished anti-science propogandists and it happens to us all at one time or another.
Re:Impressive (Score:5, Informative)
The "hide the decline" and "divergence problem" issues are this: A method was developed to use tree ring data as a proxy for past temperatures for which we have no measured temperature records. The "decline" or "divergence problem" is that the method proved to be unreliable when used to "measure" temperatures in the present - the real temperature record went one way, and the tree ring data went another way.
Rather than take the more scientifically reasonable position that this inconvenient truth invalidated the entire method, the "trick" they used to "hide the decline" was to just splice the actual temperature record onto the end of the proxy data, and present this patched together result as the result of their research. I do not find this scientifically defensible.
Steve McIntyre's area of expertise is in statistics, and the choice of the correct methods to apply to various kinds of data. It was McIntyre who discovered the mistake in Michael Mann's statistical methods that resulted in the now-discredited "hockey stick" graph that shows 20th Century temperature records as something unusual. He found that the statistical method used creates a hockey-stick type graph regardless of the data that is fed into it - tree-ring data, random numbers - it all comes out the same. Again, a valid scientific examination of the methods used. Peer-review did not uncover this fundamental flaw in Mann's research - it was McIntyre.
The material you quoted from Steve's site read perfectly as perfectly reasonable to me, but that is because I am familiar enough with his context to understand what it is he is referring to with his verbal shorthand. If you see the guy being interviewed on video you quickly realize that the only axe he has to grind is that he wants the climate scientists to do their jobs properly.
Re:Impressive (Score:2, Informative)
To assume only the side you oppose has a vested interest is just plain ignorant.