Statistical Accuracy of Internet Weather Forecasts

markmcb writes "Brandon Hansen considers the statistical accuracy of popular on-line weather forecast sources and shows who's on target, and on who you probably shouldn't rely. Motivated by a trip to a water park that was spoiled with hail despite a 'clear sky' forecast, he does a nice job of depicting deviations, averages, and overall accuracy in a manner that stats junkies are sure to love."

Can we believe the forecasts?

[gavink42]

Personally, I take weather forecasts with a couple of grains of salt.

However, the last cold blast that came through Memphis was forcast almost a week ahead of time. Weather radar of the middle part of the country showed about 90% clear of storms. So, I had a hard time with that one.

To my surprise (and right on time), down came the blast of cold air. Soon after was the promised snow/ice.

It still seems like an inexact science... with a touch of art and a pinch of luck thrown in for good measure.

Free forecast

[140Mandak262Jamuna]

These web sites provide these weather predictions for free, and it is worth every penny you paid for them. Compared to some other people in prediction business, tarot cards come to my mind, these sites are not doing that badly.

Re:Can we believe the forecasts?

[Anonymous Coward]

Replace weather with climate and see the responses. We seem to trust climate predictions more than weather predictions, when both are inherently and historically inaccurate.

Re:Statistics don't lie Statisticians do!

[Overzeetop]

Actually, you're only considering the accuracy of the scientific forecast. That is essentially useless to the end user - what they (I) want is accuracy of the reported forecast. The difference? I don't care when the actual prediction was made, I want to know that when I look at the forecast, it is likely to be correct. A very accurate forecast that is only updated once every three days is not nearly as useful as a farily accurate forcast updated every ten minutes. The former would be the best by your yardstick, but wouldn't necessarily help determine if the likilihood of a hail storm was high for this afternoon as much as the latter.

Who cares about temp? Is it gonna rain?

[Eideteker]

I'd rather know if there's going to be any precipitation so I can plan my motorcycle gear correctly. Let me know when someone compares precip. forecasts.

Re:Can we believe the forecasts?

[Daniel Dvorkin]

What I find funny is that the entire practice of weather prediction is based on a logically fallacy. They take the data from previous years and say, ok, last time conditions looked like this x happened, so we predict x will happen again. Anybody who's taken an introductory logic class knows that you can't correlation does not equal causation.

And anyone whose understanding of correlation goes beyond "an introductory logic class" knows that in fact, as long as you're very careful about what you're doing, you can in fact very often use observed correlations to make valid predictions.

There's this whole field of study called "statistics," see. Not the "X% of people surveyed believe Y" type of thing you hear on the news, but an actual science, grounded in rigorous mathematical theory and growing more sophisticated all the time at producing useful knowledge from mountains of data. People get PhD's in it and stuff. Really. Maybe you ought to read about it some time. Maybe even take a class.

Or perhaps you'd rather remain secure in your prejudices, repeating "correlation does not equal causation" like a mantra, snickering at people whose knowledge you choose not to understand.

We got global warming down, why not the short term

[lessthan0]

Since we can predict with absolute certainty what the weather of the Earth is going to be 100 years from now (latest IPCC report), why can't we accurately predict the weather 10 days from now? Unless maybe we can't predict the weather 100 years from now. Hmmm.

Re:We got global warming down, why not the short t

[Waffle Iron]

Since we can predict with absolute certainty what the weather of the Earth is going to be 100 years from now (latest IPCC report), why can't we accurately predict the weather 10 days from now?

It's the same reason why you can't predict what you might win if you play a slot machine for an hour, whereas the casino can predict the annual profits from its slot machines to a high degree of accuracy.

Nice work? huh?

[rhombic]

That's a nice piece of work?? Looking at his charts, I'd be amazed if there was p0.1 significance to any of the differences, much less p0.05. Not significantly different. And sampling bias out the kazoo-- whatever differences there are in models are likely to be very different over areas with different weather and different times of the year. This would be like looking at the arrivals/departures board at your local airport, and deciding based on that which airline is most reliable.

Not to mention, the guy's inspiration for this was an unexpected storm-- but his analysis is limited to only hi and low temps???? Kind of like looking at the airport arrival/departure time board to decide which airline is most reliable in delivering luggage, isn't it? Excel will let you chart anything you want, but please don't try to pass it off as statistics.

Re:Nice work? huh?

[berzerke]

...the guy's inspiration for this was an unexpected storm-- but his analysis is limited to only hi and low temps????

Living in Houston as the author of the study does, I can tell you that rain can be rather spotty. There are many times when I've been off and stayed home all day and saw no rain, while my wife, who works about a 15 minute drive away saw torrential downpours. Some areas get rain, others don't. Seems to be the same for other parts of Texas too. Trying to do a rain analysis in Houston would be rather difficult. Do you use a certain location, or just where you are during the day?