Statistical Accuracy of Internet Weather Forecasts 189
markmcb writes "Brandon Hansen considers the statistical accuracy of popular on-line weather forecast sources and shows who's on target, and on who you probably shouldn't rely. Motivated by a trip to a water park that was spoiled with hail despite a 'clear sky' forecast, he does a nice job of depicting deviations, averages, and overall accuracy in a manner that stats junkies are sure to love."
What is your source? (Score:5, Informative)
One thing that struck me is the 'abnormal diversity' of weather information sources. In Canada, weather models are computed in one place, a ~1000 processors computer in a basement which does only one thing: forecasting weather (the constant real-world observations that are ingested are used to adjust the models). Only one 'real' source (of course, there's the american, british, french, etc. official forecasting models to which we compare 'scores' on a daily basis). However, there's plenty of other canadian websites which will give you weather forecasts (one example [meteomedia.com]). From what I know, these "other websites" have a significantly smaller workforce of meteorologists to interpret the models results than the Meteorological Service of Canada [ec.gc.ca] (the CMC is part of the MSC). That's why I would favor the 'original' source instead of a 'second-hand' source. I must however admit, commercial online sources of weather forecasting sometimes offer value-added products, such as the number of ski trails opened, offer general weather information capsules, etc.
And by the way, the official Environment Canada weather website [ec.gc.ca] is the most visited website in Canada (or at least, that's what they tell us, the employees!
Statistics don't lie Statisticians do! (Score:5, Informative)
The National Weather Service collects all the weather data used by forecasters, they also provide the 1st forecast. AccuWeather and others take the National Weather Service forecast then watch the new data (using National Weather Service provided data) to offer a refined forecast a few hours latter. Who do you think is going to be the most accurate the guy who provides the first forecast or the guy who waits for more data and then refines the for cast? AccuWeather's has statistics that show they are more accurate then the National Weather Service but if you used the AccuWeather forecast then waited for the next National Weather Service update I bet National Weather Service would be more accurate.
I am surprised that this guy used the weather.com and not the National Weather Service for the actually temp for all his calculations. (It doesn't matter b/c I am sure weather.com is right from National Weather Service data). He did point out that AccuWeather is the only one who provides forecasts > 10 days in advance.
My preference for weather forecasts is:
National Weather Service
AccuWeather (easy to understand graphics and 2 week forecasts)
The Weather Underground (Years ago they were the1st to provided free access to hurricane computer models)
Re:NOAA/NWS (Score:3, Informative)
Re:Reliable forcasting method... (Score:4, Informative)
Persistence is the yardstick all forecasters use to determine if they should find another line of work (or be asked to do so by others). If you can't demonstrate an understanding of the processes and data such that extend beyond the data source everyone else has (ie., the weather their experiencing), it's just snake oil.
(IAAM)
Re:NOAA/NWS (Score:4, Informative)
Amen to that. NWS is one example where a government funded program actuallly WORKS. Clean, simple, Flash-free. It's been reliable enough for me. I was able to successfully plan for an outside project almost a week in advance with their long-range forecasts. You have to learn how to use these things a bit, based on your area. For example, here in DC during Summer, it may or may not rain in the afternoon, and nobody can predict if it will actually rain on a particular spot. That's because most of the rain comes from brief thunderstorms that pop up. In winter, they can tell you if a snowstorm will be nearby, but not if it will actually snow or how much. OTOH, sunny vs. rainy and general temperature predictions work pretty well. As an experienced user, I've learned to recognize which types of weather systems are predictable, and which aren't (e.g., Alberta Clipper -- easily predictable temperature drop vs. Gulf low snowstorm--extremely difficult to get the snow total in advance). Other areas have their own peculiarities too I'm sure.
Re:Reliable forcasting method... (Score:5, Informative)
On any significant sample, weather reports were never worse than this.
Currently, models are able to make 85% or a little more accuracy.
This may sound paltry, but where this really works out is in longer term forecasts. At 75% you are probably wrong at 3 days forecast. Even if you take the assumption that forecasts are independent from day to day, 85% means you are probablt wrong after 5 days.
The extra two days you can predict for is what the money is going towards.
slashdotted! (Score:2, Informative)
Source Integrity (Score:4, Informative)
One reason for countries to maintain their own weather forecast agency is to ensure the integrity of the data. This ensures that a country isn't receiving tainted data, or denied data. Models could be skewed to favor accuracy in one country over another, giving that country agricultural and energy trading competitive advantages. During many conflicts, countries where the conflicts occur cease dissemination of weather data so that the opposing force can't use the data. The US DoD maintains its own weather forecasting computers to ensure that access can't be denied, even if there is an NWS outage. If a country maintains its own systems, data integrity isn't in question.
A reason to use multiple models is that each model has different strengths. One model may tend toward forecasting precipitation over the midwest more often than it is likely to occur, and another may tend to forecast precipitation less often then actual. By using both models, we can get a better idea of the actual weather. In this case, if both forecast dry, it would likely be dry, and if both forecast precipitation, we would expect precipitation, and if they split, the forecasters would have to go back to old time forecasting techniques and get the coin and dartboard out. (Just kidding about the coin and dartboard. They'd really have to unfold their broaches, hats, and Pterodactyls, and start using the charts for what they were intended.)
Large metro areas (Score:4, Informative)
Too long ago, when I was an undergraduate taking Meteorology, we visited the weather department in a Twin Cities (MN) television station. The anchor on duty was pretty blunt: if there's a 100% chance of rain on one end of town and a 10% chance on the other end, the broadcast would distill that as a 55% chance of rain. He argued that it was the best his department could offer given the commercial realities of limited airtime and the mandate to serve the entire metro area.
Senator wanted to halt NWS website. (Score:2, Informative)
FWIW, former senator Rick Santorum (R-Pa) was gunning to prohibit the NWS from providing forecast information directly to the public. Why? AccuWeather, a Pennsylvania-based company was lobbying him to do so on the basis that the government should not be undermining private corporations business interest. In other words, Accuweather wanted to continue to sell their forecast products without the free competition from the NWS forecast products.
As someone who relies upon information from the NWS, I'm glad it went nowhere, and also glad to see that he didn't get re-elected so he doesn't have the chance to reintroduce such a stupid idea.
Re:The more the merrier? (Score:4, Informative)
Re:The more the merrier? (Score:3, Informative)
Just a note, MSN doesn't have any meteorologists on staff. The weather forecasts at weather.msn.com are provided by The Weather Channel [microsoft.com].
-AceRe:Whom (Score:4, Informative)
-- Winston Churchill, on the practice of rearranging sentences to make sure they don't end in a preposition.
It is interesting to note the origin of such practices. Nothing in English forbids ending with a proposition. In Victorian England, the educated middle classes invented grammatical shibboleths [wikipedia.org] to differentiate themselves from the uneducated lower classes. To do so, they took rules from Latin that had never before applied to English.
Rules such as this one and not splitting an infinitive were not originally part of the English language. They were invented for elitist reasons. People who insist on them should in their nuts be kicked.