Statistical Accuracy of Internet Weather Forecasts

markmcb writes "Brandon Hansen considers the statistical accuracy of popular on-line weather forecast sources and shows who's on target, and on who you probably shouldn't rely. Motivated by a trip to a water park that was spoiled with hail despite a 'clear sky' forecast, he does a nice job of depicting deviations, averages, and overall accuracy in a manner that stats junkies are sure to love."

The more the merrier?

[Reverse Gear]

What a nice piece of work.

I can't help but smile a bit that MSN weather in this test turns out to be the absolutely worst when it comes to accuracy in almost all categories.

I would think there is a lot of difference on how the forecasts are made in the different channels, some of them probably do get a lot of their information from meteorologist working on their own stations. I wouldn't wonder if MSN doesn't have a meteorologist (or maybe only one) working to provide their forecasts.
Computers and simulations play a big role in predicting the weather today, but human eyes are worth a lot still.

I don't myself live in the USA, so my primary use of these are to check on when there is severe weather in areas where I know someone.
I have gotten used to check on weather underground for this information, I haven't checked on many other weather channel, but I feel quite well capable of following what is going on in the USA with tornadoes and such here from Denmark.

For a long time we only had one weather forecast service here in Denmark, a national institute. Since a primarily private TV station (TV2) a few years ago started their own weather forecast service, I really feel the national institute have been pulling themselves together and have provided many services that they didn't provide until now. So even though some of the services provides terrible accuracy they might still serve a good purpose in giving the other services competition and thereby forcing them to improve also.

When I am really dependent on the forecast I tend to study the information behind the simple prediction of the given weather, that way I am also much better prepared for possible scenarios, knowing which front move where and can better "read the skies".

Re:The more the merrier?

[Gabrill]

It seams odd to me that he started the project because of rain, and then completely ignored rain in his observations. Otherwise, the study was very cool.

Re:

[chrismcdirty]

The absolute most reliable place to go for weather prediction (and emergency information) in the US is the Nation Weather Service. [noaa.gov]

Re:The more the merrier?

[Hoi Polloi]

That's where I go for my weather online. Let's face it, all of the other weather services depend on NOAA for their base data anyway so you might as well go to the source. I do find that the NOAA predictions tend to err on the conservative side though (always predicting a chance of precipitation on dry weeks, etc).

Re:

[AceGopher]

Just a note, MSN doesn't have any meteorologists on staff. The weather forecasts at weather.msn.com are provided by The Weather Channel [microsoft.com].

-Ace

Nice work? huh?

[rhombic]

That's a nice piece of work?? Looking at his charts, I'd be amazed if there was p0.1 significance to any of the differences, much less p0.05. Not significantly different. And sampling bias out the kazoo-- whatever differences there are in models are likely to be very different over areas with different weather and different times of the year. This would be like looking at the arrivals/departures board at your local airport, and deciding based on that which airline is most reliable.

Not to mention, the guy's

Re:

[berzerke]

...the guy's inspiration for this was an unexpected storm-- but his analysis is limited to only hi and low temps????

Living in Houston as the author of the study does, I can tell you that rain can be rather spotty. There are many times when I've been off and stayed home all day and saw no rain, while my wife, who works about a 15 minute drive away saw torrential downpours. Some areas get rain, others don't. Seems to be the same for other parts of Texas too. Trying to do a rain analysis in Houston would b

Re:

[JWSmythe]

Actually, checking arrival and departures does help estimate if your luggage will arrive with you. :)

If you get off flight 1 with 5 minutes to get to flight 2, your bags may not be unloaded from flight 1 until flight 2 is already on the runway.

Been there, done that, got my luggage the next day.

But, I agree, he's looking in the wrong place. I can imagine it would be a bit tricky to calculate if rain estimations are right. If they gave a 10% chance

Re:

[einnar2000]

I can't help but smile a bit that MSN weather in this test turns out to be the absolutely worst when it comes to accuracy in almost all categories.

That's because MSN is trying to control the weather, not predict it. Trends are for sissies.

Re:

[GPSguy]

Not very accurate. In the US, a well-tuned numerical prediction is skillful for about 72 hours. At best. Anomalies happen. The forecasts I did, for example, at the time of Hurricane Katrina, were pretty decent. Those I did when Hurricane Rita came through were 'way off on track... but then, so were those from the National Hurricane Center. Model verification and tweaking are the fodder of PhD projects and dissertations.

For what it's worth, the ECMWF model tends to be good for about another 24 hrs beyo

NOAA/NWS

[zoward]

I use the NWS website, mostly because I hate all the annoying flash ads on most of the other sites. I was also under the impression that most of these sites get their information from the NWS and pass it along to you (along with a bucket of ads). There was a lot of complaining amongst the popular weather sites when the NWS opened its own web site.

Re:

[senatorpjt]

Yeah, I was surprised. Apparently, all the sites are different.

Re:

[limecat4eva]

I'll second that. Personally, I love the 48 hour graphs [noaa.gov] for being clean, simple, and easy to understand. And quick to load.

Re:

[smokin_juan]

Absolutely! Even if the graph is "wrong" you still get an approximate idea of what's supposed to happen. Hope you like the daylight indicating shading... it was my idea ;). Too bad they didn't implement the suggestion to change "48 hour period" to a drop menu that'd let you show 48, 72, 96 hours or the whole 5 days.

Anyway, I'm surprised this is the first time I've seen (would've seen if the site weren't 'dotted) forecast accuracy data. With everyone claiming their weather is the most accurate, you'd t

Re:

[GPSguy]

The various Private Sector "Partners" do track their forecast accuracies but mostly to see what they can improve in their forecasts. I know the Weather Channel runs its own versions of at least 2 models and does verificaiton. I expect AccuWeather does as well (well, OK, I know they do), as do WeatherBug. It's not likely they'll show their verification stats, though. As few folks really understand statistics, showing something that degrades toward a coin toss after a couple of days (not really, but I'm m

Re:NOAA/NWS

[istartedi]

Amen to that. NWS is one example where a government funded program actuallly WORKS. Clean, simple, Flash-free. It's been reliable enough for me. I was able to successfully plan for an outside project almost a week in advance with their long-range forecasts. You have to learn how to use these things a bit, based on your area. For example, here in DC during Summer, it may or may not rain in the afternoon, and nobody can predict if it will actually rain on a particular spot. That's because most of the rain comes from brief thunderstorms that pop up. In winter, they can tell you if a snowstorm will be nearby, but not if it will actually snow or how much. OTOH, sunny vs. rainy and general temperature predictions work pretty well. As an experienced user, I've learned to recognize which types of weather systems are predictable, and which aren't (e.g., Alberta Clipper -- easily predictable temperature drop vs. Gulf low snowstorm--extremely difficult to get the snow total in advance). Other areas have their own peculiarities too I'm sure.

Senator wanted to halt NWS website.

[Radon360]

FWIW, former senator Rick Santorum (R-Pa) was gunning to prohibit the NWS from providing forecast information directly to the public. Why? AccuWeather, a Pennsylvania-based company was lobbying him to do so on the basis that the government should not be undermining private corporations business interest. In other words, Accuweather wanted to continue to sell their forecast products without the free competition from the NWS forecast products.

As someone who relies upon information from the NWS, I'm g

Re:

[Danse]

Actually, former Senator Santorum was correct.

You've gotta be trolling, but even still, that statement is moronic. Santorum wasn't trying to abolish NWS, he was trying to make it easier to AccuWeather to sell us what we have already paid for. That's the worst of both worlds right there.

Re:

[Danse]

I said further, as in further than what Senator Santorum wants.

It's not further than what Santorum wants, it's something completely different! Santorum is just trying to help AccuWeather profit. I definitely didn't see anything about him making AccuWeather rely only on their own satellites rather than government satellites.

Re:

[Danse]

Um, did you KNOW that Santorum was one of the 20 Most Corrupt Politicians last year? So "just trying to help AccuWeather profit" was most likely putting dough in his own campaign chest or wherever.

Of course it was, that was sort of my point. I'm sure he wasn't doing it out of the goodness of his heart. They promise support and contributions and he helps them get rid of an obstacle to higher profits for them. Win-win for them, big loss for the public.

Re:

[knightri]

In the US today, two major organizations release weather forecasts, the National Weather Service and AccuWeather which is based in central PA. However, both of these services get their information from the same source, the satellites, and base their forecasts on the runs of the various computer models. The major computers are the GFS, the NAM, the UKMET and the ECMWF. There are many more though and each is fed the satellite data/ground observations etc to help make a forecast. Some computers, like the N

Re:

[pbrammer]

Who do you think owns the source of GFS, NAM, and those satellites that you speak of? NOAA! AccuWeather seems to miss the mark frequently. I trust the NWS over ANY online source because of their *local* forecasting offices. Not some weather company who issues local forecasts from the other side of the country (or GLOBE!). Also, just because it snows, doesn't mean that the temps drop 15 degrees (F, I presume). The colder it is, the less of an effect the snow cover has on temperatures. Unless of course

Re:

[djdavetrouble]

Going outside and looking? Ask at your local home center about 'windows', they'll know. Hint: not the operating system.

You must be new here. We all live in our parents' *windowless* basement and drink copious quantities of mountain dew
while playing WOW for 4 days at a stretch.

Summary

[DraconPern]

Only trust 6 days forcast. Avoid MSN Weather.

What is your source?

[Lord Satri]

(I work at the Canadian Meteorological Centre, but I am not a meteorologist myself)

One thing that struck me is the 'abnormal diversity' of weather information sources. In Canada, weather models are computed in one place, a ~1000 processors computer in a basement which does only one thing: forecasting weather (the constant real-world observations that are ingested are used to adjust the models). Only one 'real' source (of course, there's the american, british, french, etc. official forecasting models to which we compare 'scores' on a daily basis). However, there's plenty of other canadian websites which will give you weather forecasts (one example [meteomedia.com]). From what I know, these "other websites" have a significantly smaller workforce of meteorologists to interpret the models results than the Meteorological Service of Canada [ec.gc.ca] (the CMC is part of the MSC). That's why I would favor the 'original' source instead of a 'second-hand' source. I must however admit, commercial online sources of weather forecasting sometimes offer value-added products, such as the number of ski trails opened, offer general weather information capsules, etc.

And by the way, the official Environment Canada weather website [ec.gc.ca] is the most visited website in Canada (or at least, that's what they tell us, the employees! :-).

Additional comments...

[Lord Satri]

(the fact that you must post quick at Slashdot or your comment will be buried really lowers post quality since people (or at least I) sometimes rush their comments uselessly... ;-) Additional comments (after reading parts of TFA).

We must not forget online forecasting websites often offer a trend for a whole day, but (in Canada's weather office case, see parent) it is worthed to read the accompanying text to know how the weather will evolve throughout the day. If you need close to real-time observations, use

Re:

[mgblst]

It always seems a bit odd to me, that when you model the weather, you need to model the entire world, and all the different models get their data from the same sources. Why then, don't we have a world weather computer? What I mean, is combining all these different computer resources into one huge model? I know that each nation does its own little tweaks to produce the "best" model, but surely the ability to throw even more machines at such a problem would produce better results? Are we heading this way, or

Source Integrity

[wasted]

It always seems a bit odd to me, that when you model the weather, you need to model the entire world, and all the different models get their data from the same sources. Why then, don't we have a world weather computer? What I mean, is combining all these different computer resources into one huge model? I know that each nation does its own little tweaks to produce the "best" model, but surely the ability to throw even more machines at such a problem would produce better results? Are we heading this way, or is their just too much prestige for a country to work out its own weather?

One reason for countries to maintain their own weather forecast agency is to ensure the integrity of the data. This ensures that a country isn't receiving tainted data, or denied data. Models could be skewed to favor accuracy in one country over another, giving that country agricultural and energy trading competitive advantages. During many conflicts, countries where the conflicts occur cease dissemination of weather data so that the opposing force can't use the data. The US DoD maintains its own weather forecasting computers to ensure that access can't be denied, even if there is an NWS outage. If a country maintains its own systems, data integrity isn't in question.

A reason to use multiple models is that each model has different strengths. One model may tend toward forecasting precipitation over the midwest more often than it is likely to occur, and another may tend to forecast precipitation less often then actual. By using both models, we can get a better idea of the actual weather. In this case, if both forecast dry, it would likely be dry, and if both forecast precipitation, we would expect precipitation, and if they split, the forecasters would have to go back to old time forecasting techniques and get the coin and dartboard out. (Just kidding about the coin and dartboard. They'd really have to unfold their broaches, hats, and Pterodactyls, and start using the charts for what they were intended.)

Re:

[mgblst]

I spoke to someone about this last year, and I was led to understand that their was a huge pool of data, colleceted by all the weather agencies around the world, and everyone had access to it. It is not like Canada has weather recording stations spread throught the world, in case they don't get data from somewhere.

Re:

[wasted]

All raw weather data is usually broadcast freely, as is most model data. Countries can cease dissemination on their own at any time, however. For instance, if my memory serves me correctly, it was difficult to get Iran data in the early eighties and Iraq data in the late eighties. The weather models still generated worldwide forecasts, even without the missing data. It was difficult to verify the models in the areas with missing data, though. Now, satellites can make up for the most of the missing data

Re:

[Lord Satri]

"Why then, don't we have a world weather computer?"

Read my other comment [slashdot.org]. There are several reasons. One is that models runs at higher resolutions for specific regions. i.e. Canada runs higher resolution models over Canada, a thing which, obviously, France or Britain won't do. There are also numerous products (e.g. marine, aviation, emergencies, etc.) which are more or less region-specific (or that no one wants to produce for the whole world).

There are countries, Australia is an example, which decided to p

Re:

[FuzzyDaddy]

"climate [aesto.or.jp] is what you expect. Weather is what you get."

-Mark Twain

Re:

[Anonymous Coward]

The fun part.... My personal weather station is only lacking radar, I do have old slow scan satellite imaging from a ham radio receiver. and I can predict weather at my location far more accurate than the NWS and the local stations.

By looking at the rate of barometric pressure drop and the rise in humidity and wind changes I can tell you within a 1/2 hour when it will rain or storm and typically how badly.

What blows my mind is why does the National weather service not install a crap load of cellular connec

Re:What is your source?

[CmdrGravy]

Within half an hour ! That's pretty gosh darn impressive work there sir !

If you wouldn't mind going into a bit more depth I'd love to hear the details of how you knock up these forecasts, at the moment I can only predict the weather for definite maybe 20 - 25 minutes ( using basic optical observations ) in advance and I'd love to shave off those extra 5 minutes but I wonder if it's worth the cost of investing in something like a PC. I've seen portable weather stations you can install in your living room from hippy shops, do you think these would be suitable ?

Re:

[geoffspear]

What blows my mind is why does the National weather service not install a crap load of cellular connected mini weather stations across the country in the military grid coordinates?

Lack of an unlimited budget?

Who knows what they'll be able to accomplish now that Rick Santorum isn't around to try to prevent them from competing with PA-Based Accuweather anymore, though?

Re:

[operagost]

Uhh... clearly, since the NWS collects data whether or not they are allowed to release it to the public, Rick Santorum's presence had nothing to do with it. Accuweather needs NWS data, so it's in their best interest that that data be accurate. Your anti-Santorum troll is useless and misinformed.

Re:

[chipperdog]

They do use data from State DOT roadside sensors (such as found at rwis.state.mn.us [state.mn.us] (nearly every state's dot uses these google "RWIS and your state"), APRS stations such as those found at findu.com [findu.com], and Citizen Weather Observer Program [wxqa.com]

Re:

[geoffspear]

Well, of course they use data that the stares are collecting anyway, but I hardly think we can expect them to get funding to do what GP suggesting, which was paying for their own giant grid of weather stations blanketing the entire country. This would be both extremely expensive and probably not particularly effective; the problem of forecasting weather accurately beyond a few days time isn't necessarily going to get any easier simply by adding more data points, unless we find a way to track the motion of

Re:

[Helios1182]

Sites like http://www.wunderground.com/ [wunderground.com] take in feeds from small weather stations (schools, homes, etc.) and display them in a list. At least in Chicago there are enough to get a good idea of the real weather (not just at O'Hare).

Re:

[JWSmythe]

    That's really neat around Los Angeles, especially in the hills. Someone had a PWS only a few miles from my house, so it gave me great results. The closest "official" weather station was 10 miles to my west, and 1500 feet below me. Temperature, wind, and even humidity were always wrong from the "offical" site.

    Eventually, I plan on putting up a PWS. Who knows when though. :)

Re:

[GPSguy]

Radar is but one tool in the toolchest we all use in numerical prediction. Satellite data are useful, as are in-situ observations. In fact, the US National Centers for Environmental Prediction are incorporating these data into the models (google for 3dVAR and data assimilation).

For a given isolated point (your house/apartment) and given sufficient experience (you've lived there for a couple of years watching the weather as an avocation) it's probable that you can predict for that site better than the Nati

Re:

[JWSmythe]

You don't need to be lacking the radar. :)

You'd have to google around for the page, but I had seen a site, where a guy had taken a naval weather radar unit, that he had picked up surplus, mounted the antenna on his roof, and attached the rest to his computer.

But, you can grab weather radar images from a whole slew of sites, and use only what's specific to you. While your instrumentation can see YOUR location, and your own radar could see 15 miles or so, using ot

Re:Just a suggestion...

[flyingfsck]

In Calgary, the weather forecast is just a suggestion. Actual weather can change 20C up or down in a few minutes. One side of the city can have snow and the other side sunshine. Sometimes it snows in July. So I have no idea why all these other wimps are complaining about a thunder shower on a forcasted clear day. I mean, what is the problem with that? The guy should be glad that he didn't get a heat wave followed by 2 feet of snow after the thunder shower...

Re:

[iabervon]

I believe that all of the weather sites get their model runs from the NWS, but they each interpret them partially individually. The big computer in the basement spits out a lot of values for various things, but the daily low in Houston isn't exactly one of them. There's a lot of "the model says this, but that pocket of cold air over there is just as likely to go north or south of the model output, so we'll say something different". Also, there are a number of different models with different behavior. The NW

Reliable forcasting method...

[AmIAnAi]

I remember some years ago a radio presenter saying that you could achieve greater accuracy than supposed weather forcasters simply by using the assertion: today's weather will be the same as yesterday. Have we moved on from this position?

Re:Reliable forcasting method...

[Bob(TM)]

In the forecasting vernacular, that method describes forecasting based on "persistence".

Persistence is the yardstick all forecasters use to determine if they should find another line of work (or be asked to do so by others). If you can't demonstrate an understanding of the processes and data such that extend beyond the data source everyone else has (ie., the weather their experiencing), it's just snake oil.

(IAAM)

Re:Reliable forcasting method...

[Anonymous Coward]

persistence is 75% reliable.

On any significant sample, weather reports were never worse than this.

Currently, models are able to make 85% or a little more accuracy.

This may sound paltry, but where this really works out is in longer term forecasts. At 75% you are probably wrong at 3 days forecast. Even if you take the assumption that forecasts are independent from day to day, 85% means you are probablt wrong after 5 days.

The extra two days you can predict for is what the money is going towards.

Re:

[OldBus]

persistence is 75% reliable

Won't this depend where in the world you are? E.g. in desert areas most days are likely to be hot and sunny. Here in England, we consider stable weather for 10 minutes to be persistent.

Re:

[armb]

> > persistence is 75% reliable
> we consider stable weather for 10 minutes to be persistent

Making a forecast of "it will be variable" almost 100% reliable, and a forecast of "it will be variable in the same sort of ways as today" about, oh, 75% reliable?

Re:

[CmdrGravy]

It's kind of sleeting here at the moment, the question is will it get colder and turn into snow necessitating an immediate retreat from work or will it get warmer and turn into boring old rain.

Re:

[edremy]

Persistance is a great way to do it, except that the information out of it tends to be pretty much useless. Think about what you care about- temperature/humidity and preciptiation. Frankly, I don't care so much about these if they continue the way they are right now. It's cold and clear outside right now, but I'm dressed for cold+clear. If it was rainy I'd have an umbrella, if it was really snowy I'd have driven my wife's Subaru to work.

What I care about (for lack of a better term) is the weather delt

Counter example

[Secret Rabbit]

Well, that might be true is some (most?) places. But:

In Winnipeg (which I just moved away from a couple months ago) can have violently different weather day to day and even sometimes within the same day. It as explained to me as: there are different "spheres of influence" when it comes to weather patterns and Winnipeg sits on the boarder of two of them. Knowing that weather is never stable means that this "line" constantly moves, sometimes quickly. Thus, the violent weather patterns.

In fact, in Southern

rain?

[Shooter6947]

What about predictions for precipitation? The author claimed that he was motivated by a washed-out waterpark visit. Variations in the high temp by 3 degrees don't really matter a lick -- what matters is if it predicts sun and you get rain, or vice versa. How accurate are those "20% chance of rain" predictions really? Inquiring slashdot readers want to know!

Re:rain?

[Wordplay]

You'd need a lot of data for that one, if you want to establish the accuracy of the probabilities. Unless I'm missing a more mathematically clever way to do it, I'd assume you'd require lots of 20% days to determine whether rain happened on anything near 20% of them. Similarly, 10, 30, 40, 50, etc would require their own groups of lots of days.

It'd be even trickier in, say, the SF Bay Area, where it only rains for two or three months a year, and then almost every day. Your 0% and 80-100% groups would be well-stocked, but not so much the other ones.

20% Chance of Rain

[iamlucky13]

I don't know about the rest of the world, but here in Portland, "20% chance of rain" does not mean that rain should be expected on 20% of the days with that forecast. It means that it will rain for 20% of the day.

The forecasters here are also fans of synonymous statements. Check out this week's forecast:

Fri: Mostly cloudy with a chance of rain
Sat: Showers likely
Sun: Occasional Showers
Mon: Rain and scattered sunbreaks
Tue: Scattered showers
Wed: Possible morning mist followed by afternoon drizzle
Th

I want one of these for stock analysts.

[Anonymous Coward]

Anyone know who tracks the accuracy of individual stock analysts?

Re:

[demonbug]

I hear this [irs.gov] group keeps track pretty well...

Statistics don't lie Statisticians do!

[wesborgmandvm]

Kudos to this guy for the work he put into the effort but it is really comparing apples and oranges. A forecast is a time sensitive product. You can't look at the forecast provided on day x from two different sources and compare them unless the forecast was provided at the same time of day.

The National Weather Service collects all the weather data used by forecasters, they also provide the 1st forecast. AccuWeather and others take the National Weather Service forecast then watch the new data (using National Weather Service provided data) to offer a refined forecast a few hours latter. Who do you think is going to be the most accurate the guy who provides the first forecast or the guy who waits for more data and then refines the for cast? AccuWeather's has statistics that show they are more accurate then the National Weather Service but if you used the AccuWeather forecast then waited for the next National Weather Service update I bet National Weather Service would be more accurate.

I am surprised that this guy used the weather.com and not the National Weather Service for the actually temp for all his calculations. (It doesn't matter b/c I am sure weather.com is right from National Weather Service data). He did point out that AccuWeather is the only one who provides forecasts > 10 days in advance.

My preference for weather forecasts is:

National Weather Service
AccuWeather (easy to understand graphics and 2 week forecasts)
The Weather Underground (Years ago they were the1st to provided free access to hurricane computer models)

Re:

[TapeCutter]

"He did point out that AccuWeather is the only one who provides forecasts > 10 days in advance"

He must have missed Australian seasonal outlooks [bom.gov.au].

Re:

[TapeCutter]

I was simply trying to point out that longer term "predictions" are available and have a fair degree of accuracy. I fully understand that climate != weather => outlook != forecast.

Re:Statistics don't lie Statisticians do!

[Overzeetop]

Actually, you're only considering the accuracy of the scientific forecast. That is essentially useless to the end user - what they (I) want is accuracy of the reported forecast. The difference? I don't care when the actual prediction was made, I want to know that when I look at the forecast, it is likely to be correct. A very accurate forecast that is only updated once every three days is not nearly as useful as a farily accurate forcast updated every ten minutes. The former would be the best by your yardstick, but wouldn't necessarily help determine if the likilihood of a hail storm was high for this afternoon as much as the latter.

Can we believe the forecasts?

[gavink42]

Personally, I take weather forecasts with a couple of grains of salt.

However, the last cold blast that came through Memphis was forcast almost a week ahead of time. Weather radar of the middle part of the country showed about 90% clear of storms. So, I had a hard time with that one.

To my surprise (and right on time), down came the blast of cold air. Soon after was the promised snow/ice.

It still seems like an inexact science... with a touch of art and a pinch of luck thrown in for good measure.

Re:

[nomadic]

To my surprise (and right on time), down came the blast of cold air. Soon after was the promised snow/ice.

I remember when a year and a half ago one of the hurricanes was in the Gulf of Mexico, heading almost straight west, and the meteorologists all insisted that the hurricane would make a complete 180 degree turn and head back east and smack into Florida. I didn't believe them. On more than one occasion I publicly stated that this was ridiculous.

I ate a lot of canned food that week.

Re:

[poot_rootbeer]

the meteorologists all insisted that the hurricane would make a complete 180 degree turn and head back east and smack into Florida. I didn't believe them.
I ate a lot of canned food that week.

And learned a valuable lesson about the relative merits of the consensus opinion of many scientists using multi-million dollar supercomputers, and one layperson's hunch.

Re:

[Mr2cents]

Ever since I've been riding to work with my bicycle, I'm a big fan of online weather radars. You can see where it's raining in real-time and where the rain is heading. It's only predictable up to a few hours but many times that's all I need. It has kept me dry on many occasions.

Of course, if a rain zone develops right above your head you're out of luck. But most of the time, when it rains it's just an active rain zone passing over your head.

If you live in the Benelux, chek out http://www.buienradar.nl/ [buienradar.nl]. It'

Re:

[BootNinja]

What I find funny is that the entire practice of weather prediction is based on a logically fallacy. They take the data from previous years and say, ok, last time conditions looked like this x happened, so we predict x will happen again. Anybody who's taken an introductory logic class knows that you can't correlation does not equal causation. Plus, we've only been studying the skies for about a hundred years, so we don't have enough data to make a decent correlation in the first place.

Re:Can we believe the forecasts?

[Daniel Dvorkin]

What I find funny is that the entire practice of weather prediction is based on a logically fallacy. They take the data from previous years and say, ok, last time conditions looked like this x happened, so we predict x will happen again. Anybody who's taken an introductory logic class knows that you can't correlation does not equal causation.

And anyone whose understanding of correlation goes beyond "an introductory logic class" knows that in fact, as long as you're very careful about what you're doing, you can in fact very often use observed correlations to make valid predictions.

There's this whole field of study called "statistics," see. Not the "X% of people surveyed believe Y" type of thing you hear on the news, but an actual science, grounded in rigorous mathematical theory and growing more sophisticated all the time at producing useful knowledge from mountains of data. People get PhD's in it and stuff. Really. Maybe you ought to read about it some time. Maybe even take a class.

Or perhaps you'd rather remain secure in your prejudices, repeating "correlation does not equal causation" like a mantra, snickering at people whose knowledge you choose not to understand.

Re:

[smithmc]

Or perhaps you'd rather remain secure in your prejudices, repeating "correlation does not equal causation" like a mantra, snickering at people whose knowledge you choose not to understand.

But he's right - correlation does not prove causation. Phenomenon Y can correlate to phenomenon X with r=0.99, and that still doesn't mean that X causes Y. Both X and Y might be caused by W, and therefore they always appear together. That's not to say that one phenomenon might not be a good predictor of another

Free forecast

[140Mandak262Jamuna]

These web sites provide these weather predictions for free, and it is worth every penny you paid for them. Compared to some other people in prediction business, tarot cards come to my mind, these sites are not doing that badly.

Whom

[mdsolar]

A preposition is awkward to end a sentence with. But, "whom" is the word "on" is followed by.
--
Solar follows the rules for grammer. http://mdsolar.blogspot.com/2007/01/slashdot-users -selling-solar.html [blogspot.com]

Re:Whom

[spun]

This is an outrage up with which we will not put.
-- Winston Churchill, on the practice of rearranging sentences to make sure they don't end in a preposition.

It is interesting to note the origin of such practices. Nothing in English forbids ending with a proposition. In Victorian England, the educated middle classes invented grammatical shibboleths [wikipedia.org] to differentiate themselves from the uneducated lower classes. To do so, they took rules from Latin that had never before applied to English.

Rules such as this one and not splitting an infinitive were not originally part of the English language. They were invented for elitist reasons. People who insist on them should in their nuts be kicked.

Re:

[mdsolar]

Here! Here! No. Listen! Listen! No, No. Just bring me a shrubbery.

Re:

[mdsolar]

Here! Here!

Interpretation of the models is everything

[Aliks]

I am not a meteorologist, but I have worked with them a few times.

Generally the competing weather models will show a range of possible outcomes with various probabilities. You can average across all scenarios and come up with a 60% probability of rain, but the more days out you go more the scenarios diverge, so the less useful a single average will be.

Most people would not find it useful to hear that "there will be probably be thunder on Wednesday if it remains hot enough, but if it cools down on Tuesday then the thunderstorm will be off to the north somewhere"

Additionally, a lot of weather conditions are influenced by thin layers of cloud high up, so thin that precise measurements are critical so precise forecasts in one location more than 3 days out are difficult.

Possibly the best slashdotted error message ever.

[sammy baby]

"Total Freaking Database Error!"

Best 500 error I've ever seen. (Although I'm not sure it actually sent a 500.)

YIC!

[Guppy06]

Where's the Wii Forecast Channel?

PHEW!

[LoudMusic]

I'm just glad MSN didn't come out on top. That would have ruined my day. Besides that, meh, it's an educated guess. So long as they can tell me "it's gonna be hot" or "it's gonna be cold", I'm cool with it. The weather forecasts for my area are almost always more incorrect than his findings, especially when it comes to precipitation.

Eh, at least they get the Moon phases right ;)

slashdotted!

[sid77]

so, here's the mirror dot link [mirrordot.com]

You can't fool mother nature

[TheHornedOne]

I work for Mother Nature; So I am really getting a kick out of most of these replies. Some of you guys are very good at making it sound like you know what you are talking about. But trust me.... You don't. I think you just want to make yourself sound smart, when in reality you don't know what you are talking about. This is how bad info gets passed around. If you dont know about the topic....Dont make yourself sound like you do. Cos some slashdotters believe anything they hear."

/wrong metasite
//slashies
//dont' kill me

Who cares about temp? Is it gonna rain?

[Eideteker]

I'd rather know if there's going to be any precipitation so I can plan my motorcycle gear correctly. Let me know when someone compares precip. forecasts.

I did this in 8th grade.

[nharmon]

My 8th grade science project was to see who could predict the weather the best. I collected forecasts and then compared them to actual meteorological conditions. I totally forget what my findings were but it is too bad Slashdot wasn't around back then, I could have had some great publicity. :)

My experience with the BBC weather forecast...

[OneSmartFellow]

for my area is that they are usually accurate down to a period of about 3 hours. As an example.

http://www.bbc.co.uk/weather/24hr.shtml?world=41 79 predicted that it would be snowing yesterday morning by 0600. Sure enough, I woke at six, and it was snowing. I awoke earlier in the night (about 0400) and it hadn't yet started.
Similarly, that site predicted that the snow would drop off by noon, and turn to sleet or rain by 1600. Again, this prediction came true, within an hour of the predicted time.

Generally speaking I find the BBC weather site to be accurate significantly more often than not (guesstimate 80% accuracy) with the 24 hour forecast being almost universally correct, and the 5 day forecast being the least reliable. (as expected)

This is a FAR cry from the weather predictions when I was a lad. Then the weather forecast on TV was simply a way to poke fun at the meteorologist, who clearly was doing the best he could, but invariably got it wrong.

Obligatory Stats Joke

[frank249]

Nice article but the sample only uses an 'n' of 14 days. I would have more confidence in the means, standard deviations and correlations if the author had used a bigger 'n'. For in stats, as in ethics, the n's do justify the means.

Re:

[the phantom]

Ow! That hurt! I am going to have to remember that one...

Re:

[frank249]

'N' denotes the whole population while 'n' denotes a subset or sample [wikipedia.org] of the population.

We got global warming down, why not the short term

[lessthan0]

Since we can predict with absolute certainty what the weather of the Earth is going to be 100 years from now (latest IPCC report), why can't we accurately predict the weather 10 days from now? Unless maybe we can't predict the weather 100 years from now. Hmmm.

Re:We got global warming down, why not the short t

[FhnuZoag]

Man, I was waiting for one of these.

The answer is very simple. Because weather != climate. Climate is a statistical average over long periods of time and large geographical areas. (And we don't know with absolute certainty, in any case. Everything has to be qualified with error bounds, which is very obvious, really, because much of what will happen is dependent on what we will do in response to predictions.) Weather is localised temporary fluctuations in phenomena.

It's like saying that we can pretty much gu

Re:

[Waffle Iron]

Since we can predict with absolute certainty what the weather of the Earth is going to be 100 years from now (latest IPCC report), why can't we accurately predict the weather 10 days from now?

It's the same reason why you can't predict what you might win if you play a slot machine for an hour, whereas the casino can predict the annual profits from its slot machines to a high degree of accuracy.

Re:We got global warming down, why not the short t

[rrkap]

Since we can predict with absolute certainty what the weather of the Earth is going to be 100 years from now (latest IPCC report), why can't we accurately predict the weather 10 days from now? Unless maybe we can't predict the weather 100 years from now. Hmmm.

You're confusing climate and weather. I can tell you from my observations of the climate that it will be hot and sunny in Sacramento on July 18 2007, but I can't tell you if it will be 90 or 115. Climate prediction is about determining the range

I predict...

[monkeyboythom]

very stormy weather for the poor website linked to in the TFA. I believe the outlook will be dark, followed by intense periods of slashdotting...

Large metro areas

[Anonymous Coward]

Geographically large metro areas -- especially those with hills or large bodies of water -- make a weather forecaster's job all the more difficult. The chance of rain may be higher on one end of town, but it's difficult for a TV or radio announcer (or a newspaper spread, for that matter) to present the distinctions clearly and quickly.

Too long ago, when I was an undergraduate taking Meteorology, we visited the weather department in a Twin Cities (MN) television station. The anchor on duty was pretty blunt: if there's a 100% chance of rain on one end of town and a 10% chance on the other end, the broadcast would distill that as a 55% chance of rain. He argued that it was the best his department could offer given the commercial realities of limited airtime and the mandate to serve the entire metro area.

ForecastWatch and ForecastAdvisor

[AceGopher]

I was quite curious about weather forecast accuracy as well. So three years ago I started collecting weather forecasts from the primary providers (Accuweather, Weather Channel, NWS, CustomWeather, Intellicast, etc.) and comparing them to actual observations. It's tougher than you might imagine, and there are a lot of factors that need to go into creating usable verification statistics.

I have a public site with some statistics for about 800 locations in the US available at ForecastAdvisor.com [forecastadvisor.com]. There is also a blog with more in depth analysis (like how do temperature forecasts fare relative to how deviant the actual temperature is [forecastadvisor.com]...in other words how well do forecasts do the further away from normal the actual is, and how to forecasts fare the further out they forecast for [forecastadvisor.com], and how does forecast accuracy compare over time [forecastadvisor.com].).

ForecastWatch.com [forecastwatch.com] is used by meteorologists and professionals. Accuweather, The Weather Channel, and several private meteorological companies use this system to help them understand and improve their weather forecasts.

And a geek note: ForecastWatch.com [forecastwatch.com] runs on Quixote (a Python web framework), while ForecastAdvisor.com [forecastadvisor.com] runs on Ruby on Rails. The back-end forecast and actual collection, and calculations are Python with a MySQL database. Both sites are close to migrating to Django, a new Python web framework and ORM.

-Ace

Why not just go to the source?

[LordPhantom]

As, in NOAA [noaa.gov]???

Interesting - but are they better than dice?

[bcarl314]

I've always wondered, but never had the time nor determiniation to find out, whether weather forecasts are more or less accurate than coin tosses and dice. I've always wanted to run an experiment similar to this :

Take the 5 day temp forecast from a national weather site (NWS/weather.com, etc) and write the high / lows on one row.

Then, flip a coin and roll a single die.
Start with yesterday's high and low.

If you toss "heads" then add the number rolled on the die to the high temp for that day. If you toss "tai

Why trust the intertubes?

[Asklepius M.D.]

When I did SAR in Alaska, a good weather forecast was essential. It didn't take long to realize that every source for information analyzed their data differently and came up with slightly different conclusions. The solution? Learn to forcast yourself. Learn to read an isobar map, learn to predict wind/temp/precip for your local area and you'll get a far more accurate feel for what's going to happen than any regional analysis could give you. Practice locally - you'll learn what worked and what didn't, t

Interesting Report

[BanjoBob]

As I read the report, the first thing that hit me was using The Weather Channel as an accurate reference foundation to base all the other services against. I don't know about his geographic region but where I live, along the front range of the rockies, TWC is frequently wrong often reporting rain when the skies are clear and clear skies when it's snowing and 25 ft visibility.

Weather guessers here are rarely, if ever accurate as the mountains and Palmer Ridge play havoc with the weather. This caused me t

KNMI.

[rew]

The KNMI (Koninklijk Nederlands Meteorologisch Instituut), our offical government-sponsored weather forcast service screwed up royally yesterday.

They predicted a snow-storm for yesterday. Enough snow to completely disrupt the whole country. Weather-alarm!!!

The only thing that happened is that they scared a lot of people into staying at home, or go home early. The evening rush hour therfore had a max total traffic jam of only 12km. Instead of the normal 250km!

Oh, and indeed streets were white for a couple of

Re:

[Overzeetop]

That's because the Mac guy just looked out the window and saw that it was raining and decided to cancel the picnic while the PC guy went outside to eat soggy sandwiches and get soaked by the afternoons "partly cloudy" weather.

I believe it was a DC forecaster who once said "I just shoveled 8 inches of partly cloudy off of my driveway" after missing the forecast the day before. It pays to have a sense of humor when you're in the meteorology business.

Re:

[iambarry]

How about :
"[The article] shows who's on target, and who(m) you probably shouldn't rely on asshole ."

Re:

[/dev/trash]

if the temp is off by 5 degrees then you could get rain or snow or whatnot.