Researchers Mine Old News To Predict Future Events 99
Posted
by
samzenpus
from the what's-old-is-new dept.
from the what's-old-is-new dept.
hypnosec writes "Microsoft Research has teamed up with the Technion-Israel Institute of Technology to develop software that can predict events like outbreaks of disease or violence by mining data from old news and the web. The project, if successful, will result into a tool that would provide information that is more than just educated guesses or intuition. The team consisting of Eric Horvitz from Microsoft Research and Kira Radinsky from Technion-Israel Institute tested the program with articles from New York Times spanning over 20 years from 1986-2007."
Past performance... (Score:5, Funny)
... is Not Necessarily Indicative of Future Results.
Do these guys not even read their own prospectus?
Re:Past performance... (Score:4, Interesting)
True, though it can to the extent that there are recurring patterns and you find the right ones.
On the other hand, there's also a circularity problem. Say you find, from analyzing 20 years of correlations, that certain events tend to happen some period after certain news reports. This might impact whether that relationship continues to hold in the future. That's already quite common for financial events: if you can reliably predict that when News Report Type X happens (for a possibly complex "X"), then Stock Move Y will happen, you can profit from it, but only until it becomes known by enough people, after which the arbitrage opportunity will close.
Re: (Score:1)
Feudalism-->Capitalism-->Socialism-->Communism-->Collapse-->Feudalism-->
Our hubris prevents us from realizing we're prone to the same problems that cause the issues in the previous cycle so we're doomed to repeat it. I say this as America thinks moving to a more socialist society means prosperrity...we forget about the "...some animals are more equal than others" clause.
One Useful Prediction ... (Score:2)
I hope that the collaboration can focus in one aspect of old news ... ... even if it is not accurate ... even if Past Events Are Not Necessarily Indicative Of Future Results
What I hope they will focus in is the possibility of violence caused by religion - no matter which religion, no matter how religion is defined
I'm sure it will do mankind a lot of good ...
Re: (Score:2)
I do agree with you that violence is indeed an innate part of human beings.
What I need to stress is, religion (and patriotism, and all other "ism" that unify a certain portion of the human society) does channel and amplify that innate violence deep within each of us and tip it over the critical mass.
Re: (Score:2)
Re: (Score:2)
"By the third generation, the wealth is gone."
Yeah? Tell that to the Rothschilds and the Rockefellers.
But seriously, this whole thing reminds me of Asimov's Foundation series.
Re: (Score:2)
Re: (Score:2)
yes, because Microsoft built all those fake Apple stores in China.
Re:Past performance... (Score:4, Informative)
jokes typically place consistency with reality a little further down in the list of priorities than whatever you were expecting as a first post.
Re: (Score:1)
maybe they should read more Asimov as well.
there was a second Foundation who's job it was to make sure the predictions came true.
Re: (Score:1)
And even the second foundation failed to do so when the Mule came.
Re: (Score:2)
Well , as long as we're not predicting global warming, the superbowl or any of the other myriad of things out there that require far more criteria than is (mis)represented by news media. My main concern is the level of propaganda printed over the timeframe covering everything from politics to commerce to science and even religion that
get the "ol' quicky newsclown one sided, one eyed coverage" by personnel biased and unfit to cover grandmothers tea party, let alone add criteria to digital clarvoyance. Their
Re: (Score:2)
Dark matter theory vs. MOND - who is owned by the bias?
CC.
So Microsoft is inventing Psychohistory. (Score:1)
I just want to know when Hari Seldon [wikipedia.org] joined Microsoft?
Re: (Score:1)
For example, did you know... (Score:5, Insightful)
There will be a Presidential election in 2016 and 2020
An NFL team will win the Super Bowl.
Many new TV shows will be cancelled each fall.
Old people will die.
Re: (Score:2)
More nightly reports of murders and burglaries may be increasing your city's crime rates.
Viewers who are asked teasing questions about the weather makes them more likely to keep watching until the end of the news.
You can predict a candidate's election success by how much air time is spent on them during the evening news.
What could be lurking under you kitchen sink that might kill you, won't do so until after a word from our sponcers.
The Observer from NZ (Score:1)
Cliff has been doing this for years. The most interesting aspect is that there is weighting given to emotional "charge".
http://www.halfpasthuman.com/
Re:Consider.. (Score:5, Insightful)
Actually, the weather report for the next day is pretty accurate most of the time.
Re: (Score:2)
Asimov was here (Score:5, Insightful)
Re: (Score:2)
The shade of Asimov raises his head..... Does this seem a little like Psychohistory to anyone else? Where's the Mule?
More importantly, which is the empire that will break down?
Re: (Score:2)
Easy, the US - mind that it probably takes a century.
CC.
Re: (Score:2)
OK, good, someone beat me to it. I'd be disappointed otherwise.
Re:Asimov was here (Score:4, Interesting)
Re: (Score:2)
well known by stock market types. which makes me wonder what the purpose of financial reporting is.
Pythia was here (Score:1)
"All this has happened before. All this will happen again."
Queue the ... (Score:2)
Re: (Score:2)
would you consider a Douglas Adams?
“Anything that happens, happens.
Anything that, in happening, causes something else to happen, causes something else to happen.
Anything that, in happening, causes itself to happen again, happens again.
It doesn’t necessarily do it in chronological order, though.”
Re: (Score:2)
See Sig
Memory is deceptive because it is colored by today's events. - Albert Einstein (For AC's)
Re: (Score:2)
(For AC's)
And for future readers finding your post long after you've changed your signature to something completely different.
Nobody should ever refer to sigs (not their own, and certainly not other people's) without quoting them.
Re: (Score:2)
This has all happened before... (Score:2)
...and will happen again.
Hey that makes a pretty good tag line...someone should make a scifi tv series based on that concept. Maybe two even!
Re: (Score:2)
Re: (Score:1)
Re: (Score:2)
Re: (Score:3)
Re: (Score:2)
care to elucidate further on that thought?
Science! (Score:3)
Discovers experience.
Re: (Score:2)
garbage in, garbage out (Score:1)
Here, Let Me Save Them Some Time (Score:5, Informative)
Unrest in Middle East
African Regime Unstable, May Collapse
Congressman Indicted
Experts Say Hurricane "Extremely Dangerous"
Audit Finds Serious Misuse of Funds
Green Energy Firm Declares Bankruptcy
Apple's Latest Product Selling Like Hotcakes
Patch Released for Serious Windows Vulnerability
Unemployment Rises, Unexpectedly
There's your headlines for the year.
Re: (Score:3)
Re: (Score:1)
Re: (Score:1)
A week ago I put on Buffalo 66 for someone, and this person said, "Blue Bird," out loud when a frame of the movie showed a Blue Bird brand bus. Then he asked me about this brand, and I honestly said I had no idea, never noticed the name before. The following night, very late, I walked by a normally vacant street, and there was another BB, idling, full of o
Re: (Score:2)
You forgot "Lance Armstrong movie set to break previous sales record", and "Windows 9 releases classical desktop edition earlier than expected".
Welcome... (Score:1)
Its taken them this long? (Score:2)
Slashdot! (Score:2)
What better place for mining old news?
Eureka! We've struck gold!
Disease outbreaks? (Score:3)
You mean by looking at twenty years of newspaper articles they were able to predict that there will be a large increase in the number of cases of influenza during November 2013 to March 2014 compared to the preceding 5 months?
Or, when disaster causes infrastructure to break down and crowds refugees into unsanitary temporary housing there's a high likelyhood of more cholera breaking out than at other times?
Gee. Color me impressed.
How is this greatly different than many different types of analysts have been doing for decades via headline counts in world newspapers and the like? (See John Naisbitt of Megatrends fame, for example. And he certainly wasn't the first.)
The math used to find the probabilities may be a bit better, and it may be more automated, but it's not particularly new.
That's why we study history (or used to) (Score:2)
Yes, i'm sure this audience will always quote that old hack Asimov but perhaps Mark Twain is better - "history does not repeat itself, but it does rhyme". Humans respond to much the same situations in much the same way - emphasis on 'much', because there are always differences of culture, place and circumstance.
That is why we study history - to guide us in our own decisions. Do some research on Heine and Neitzche and the Doctrine of Eternal Recurrance which ponts to recurring patterns of human behaviour,
Re: (Score:2)
But tell me please - this baby boomer asks why do so few gen X and Ys show any interest in history?
Because the Founding Fathers were Christian through and through; the Civil War was about slavery and nothing else; and we're the chosen people, spreaders of truth and justice and democracy, having never committed wholesale genocide or locked up our own citizens in concentration camps.
Also, they hate us for our freedumbs.
Seriously, though, most history teachers bore kids to death by doing little more than demanding they memorize places, names and dates. While some of them are shitheads who get off on this s
Re: (Score:2)
If you're investing a considerable sum in money and precious years of your youth then getting a real payback is a valid target; not everyone is rich enough to be able to dedicate resources to indulge in "intellectual masturbation".
This isn't to decry history, or to deny that many find it interesting - just one potential explanation on
Will it predict the future, or media cycles? (Score:3)
I have two family members in the journalism business. The media business has a cycle of covering particular topics and moving on when the public gets bored of it. Plenty of news does not get coverage at all, at least by English language media, if the country is too remote or the topic is too cliche.
Only as good as the source material (Score:3)
The NYT? For only the last 20 years? That's worse that basing global warming predictions on just the last 20 years.
Wise after the fact (Score:1)
Re: (Score:2)
That assumes that the paper is reporting everything. Not only is it physically impossible for papers to do so but editorial policy will emphasize one type of story and demphasize or completely ignore another. In addition, does this prediction system take into account the location and size of story? What shows up on page one above the fold in one paper (with one editorial viewpoint) may only appear several pages down in another and the number of words can also be vastly different. A paper could easily ch
Re: (Score:1)
I think we're a little more short-term than climate change.
Re: (Score:2)
Hardly. Back in the 80s and early 90s, nobody ever heard of global warming. In fact, during the 70s, there was a lot of ginned up hand-wringing about another ice age.
Re: (Score:1)
Hardly. Back in the 80s and early 90s, nobody ever heard of global warming.
Public recognition, regardless of what any ad agency may tell you to the contrary, does not regulate reality.
In fact, during the 70s, there was a lot of ginned up hand-wringing about another ice age.
No. There was ONE article about it in (I believe) Time Magazine.
Well this is odd. (Score:1)
Reminds me of the movie "Paycheck" but other than that it has been pointed out to me by a financial adviser that FDR converted this country from a republic to a democracy as the people gave him credit for bringing the US out of the great depression. Only it wasn't the government but the oil industry. This is the history part. The future part is strongly suggesting Obama is going to convert this country from a democracy to Socialism and he too will take credit for turning the economy around, and again it won
Re: (Score:2)
I believe, the result was "Five months before Palestinian State will be established with stable borders and elected government".
Microsoft Research is best ignored (Score:2)
Buuuuut (Score:1)
Researchers discover Time Machine, Predict Future (Score:2)
Made mostly of cellulose, and some of the most expensive fluids known to mankind, this newly discovered vessel is capable of transmitting information from past to present and can acurately predict many future events.
Scientists dub "Calendar" as discovery the year.
Future? (Score:1)
Isn't this a job for the Time Bandets?
Machiavelli's theory (Score:2)
Was that nothing had essentially changed from the beginning of man, so you could scientifically use Greek or Roman battles to navigate the contemporary wars with Pisa and Cesare Borgia.
Usually, though, such an enterprise would only be fruitful after the event (e.g. no scientific predictions) on account of Fortuna. Notably, Machiavelli died bitter and beaten.
Point of this is that you cannot predict the future; perhaps at best a possible outcome with some better chance than other counterfactual scenarios, whi
GIGO (Score:1)
"Garbage in, Garbage out".
But, it's probably worth a try...