"Wisdom of Crowds" Works For Individuals Too

ideonexus writes "Take a crowd of people and have them guess how many jelly beans are in a jar, and the average of their answers will be remarkably accurate. Now researchers have found the same goes for asking one person to guess about the same thing several times. Accuracy improved when the individual was given longer periods of time between guesses." The anonymous author of the Economist piece, not quoting the researchers, says the finding bolsters the "generate and test" model of creative thinking.

The Delphi Method

[Illbay]

http://en.wikipedia.org/wiki/Delphi_method [wikipedia.org]

Another product of the RAND Corporation.

Short vid about Wisdom of the Crowds

[cplusplus]

Here's an interesting little bit that was on Nova Science Now the other night explaining (in a fun way) about the Wisdom of the Crowds: http://www.youtube.com/watch?v=r-FonWBEb0o [youtube.com]

The wisdom of averages

[thethibs]

So what we've always thought was the wisdom of crowds turns out to be the wisdom of averages. That does make more sense.

Re:Ah duh!

[John Hasler]

I predict that if you ask the same person the same question over and over again even at wide intervals the answers will converge but not necessarily to the correct value (of course, that value may often be "You asked me that before. Bugger off!")

Re:In related news...

[Falkkin]

Good observation about the Gaussian distribution being necessary. Thought experiment: I am thinking of a number between one and a million. What's the likelihood that the average of a bunch of people's guesses are anywhere near the number I am thinking of?

Minsky's Emotion Machine

[br00tus]

In Minsky's book "The Emotion Machine" he describes what we know about the human brain from observation and such. When one encounters a tough problem, one turns different parts of the brain on and off in an attempt to solve it. First might be a trial-and-error brain agent, then an analogy brain agent that searches memory for some similar situation and so forth. That is why there is a difference between blitz chess and tournament chess - in tournament chess, where you have several minutes to make a decision for each move, you can draw on memory, make tactical and strategical decisions and the like quicker than the snap decisions made in blitz chess. It's also why we often go to sleep working on a tough (programming etc.) problem and wake up with the answer - our "unconscious" brain put the answer together while we slept.

Re:Wisdom of the Crowds"

[DaoudaW]

Clearly if someone _knows_ the answer, then the wisdom of crowds doesn't work. The 1000 rednecks are clearly not going to out-guess the guy that packed the pickles in the jar if he counted them. But that's totally not the point. Experimental data shows that a group of rednecks beats any individual redneck, that a group of Harvard graduates will beat any individual Harvard graduate, and sometimes a large group of rednecks will beat a small group of Harvard graduates. I did an experiment in the high school math classes I teach during the NCAA basketball tournament. On the day of the final game, I asked each student to predict the score. I was shocked at the results. In each class (5), the class average beat the best individual guess and the aggregate average beat each of the class averages. The final score was predicted to be 75-69 with Kansas winning, in reality they won 75-68. Try it yourself. It definitely works.

Re:So how long is the emperor of China's nose?

[juuri]

Nearly every one of them said yes.

The problem with this particular example from P&T (who are awesome despite doing this from time to time) is that this isn't an appeal to find a concrete value or fact, it is instead an appeal to a person's knowledge. Just like the questions asking about weapons of mass destruction were framed in a manner which directly appealed to the information people were being fed by the administration and in turn the media.

Had the questions been framed more like "If Iraq possesses weapons of mass destruction that could reach Iran? Should we invade?" (Which is a false and leading appeal like most poll questions) you can bet most of the Americans who agreed with the invasion would have said, "No."

The polling system in the US is greatly flawed, yet many people quote the stats from polls without actually reading the questions. Ever wonder why so many Americans believe in god? Because the typical poll question asks about a higher power rather than any sort of identifiable god. The responder is then primed for the followups.

Before quoting a poll, make sure you know what the questions actually asked, you may be surprised.