Using Twitter Data To Approximate a Telephone Survey 68
cremeglace writes "A team led by a computer scientist at Carnegie Mellon University has used text-analysis software to detect tweets pertaining to various issues — such as whether President Barack Obama is doing a good job — and measure the frequency of positive or negative words ranging from 'awesome' to 'sucks.' The results were surprisingly similar to traditional surveys. For example, the ratio of Twitter posts expressing either positive or negative sentiments about President Obama produced a 'job approval rating' that closely tracked the big Gallup daily poll across 2009. The analysis also produced classic economic indicators like consumer confidence." By averaging several days' worth of tweets on presidential job approval, the researchers got results that correlated 79% with daily Gallup polling. Lead researcher Noah Smith said, "The results are noisy, as are the results of polls. Opinion pollsters have learned to compensate for these distortions, while we're still trying to identify and understand the noise in our data. Given that, I'm excited that we get any signal at all from social media that correlates with the polls." Here is CMU's press release.
Brb making multiple twitter accounts (Score:3, Interesting)
Just like traditional pollsters, social media researchers will have to address how representative Twitter users are of the general population. And unlike telephone surveys, small groups of people can wildly skew the results of Internet data,
Yes I did STFA (Skim the fucking article).
It mentioned the two main problems I see with this, cheating the system and whether twitter really is a large enough sample and a random enough sample to be considered a viable alternative.
Twitter has a whole range of people who don't actually use the damned thing. As with any poll though, people are going to say that the minority polled is what everyone says.
"The American people want to do x! Our poll says 80% of the American people want it!" No. No it doesn't. It just means 80% of the people you polled want it.
I despise how easy it is to use statistics and polls to manipulate people.
Now that the word is out it's useless. (Score:3, Interesting)
As soon as something like this comes to light it's only a short matter of time until turfers screw it up. Turfers are like spammers, as soon as there's a new medium they abuse it into uselessness.
Both "McCain" and "Obama" mean Obama is good? (Score:3, Interesting)
Umm... from TFA:
Likewise, both the Twitter-derived sentiments and the traditional polls reflected declining approval of President Obama's job performance during 2009, with a 72 percent correlation between them.
Okay... not a great correlation, but let's continue....
But the researchers found that their sentiment analysis did not correlate as well with election polling during 2008. For instance, increased mentions of "Obama" tended to correlate with rises in Barack Obama's polling numbers, but increased mentions of "McCain" also correlated with rises in Obama's popularity.
WTF? Is all of this built on how many times "Obama" or "McCain" is uttered on Twitter? And, given the obvious skewed demographics on Twitter (i.e., younger people, which tended to poll way toward Obama), increased conversations about McCain probably were bad in general.
Well, how do they explain this? Ah, the next sentence....
Improved computational methods for understanding natural language, particularly the unusual lexicon of microblogs, will be necessary before Twitter feeds can be reliably mined to predict elections, the researchers concluded.
Ah yes, the "unusual lexicon of microblogs," which probably consisted of sentiments like "I luv Obama!" and "McCain too old - WTF?"
Perhaps if they bothered to measure more than "mentions" of a candidates name, the data might have some (albeit still vague) meaning...
If this is the best stuff from the study which they actually mention in a press release, how much crap results are they not reporting?
Re:Demographics Anyone (Score:2, Interesting)
I think telephone polls only reach one demographic - people willing to take a survey over the phone, who don't instantly hang up on random callers, who don't have anything better to do, and who think other people give a crap what they think. This demographic does not represent me and I doubt it covers most of the US.