"Wisdom of Crowds" Works For Individuals Too 158
ideonexus writes "Take a crowd of people and have them guess how many jelly beans are in a jar, and the average of their answers will be remarkably accurate. Now researchers have found the same goes for asking one person to guess about the same thing several times. Accuracy improved when the individual was given longer periods of time between guesses." The anonymous author of the Economist piece, not quoting the researchers, says the finding bolsters the "generate and test" model of creative thinking.
So how long is the emperor of China's nose? (Score:5, Insightful)
Wisdom of the Crowds" (Score:5, Insightful)
The idea that a group guessing is more accurate than an individual guess, and if you make more than one guess the mean or average of the guesses is more accurate than a single guess?
So, in real world terms, 1000 rednecks are going to be more accurate than one Harvard graduate? (assuming the graduate in question isn't our current President) (if we were guessing the number of pickled eggs in a pickle jar, I'd have to agree... Otherwise, I'm somewhat skeptical of how well this translates beyond the estimation of things.
Re:In related news... (Score:5, Insightful)
Not quite... but you are close. It sounds like you're pointing out that anyone will get lucky if given enough chances. These guys are claiming that the average will converge to the ground truth over time. This would need to have guesses with some Gaussian distribution about the correct answer.
If the guesses were uniformly distributed then the average wouldn't tend to the correct answer over time. Of course what is described in the summary has nothing to do with the wisdom of crowds as it is commonly thought of (i.e in markets) where shared information is vital. Instead it is simply an artifact of sampling (which is why the longer gaps are necessary for better accuracy)
Re:So how long is the emperor of China's nose? (Score:5, Insightful)
Exactly. Penn and Teller asked a group of people if the chemical Dihydrous Monoxide should be banned. Nearly every one of them said yes. The wisdom of crowds is not in and of itself some sort of magic. It is merely an interesting observation.
That your own guesses seem to exhibit the same 'average' correctness as a crowd is bad science IMO. Once you guess at a problem, you're subconsciously directed to think of that problem, thus getting more than a knee jerk reactionary guess. The longer you have to think about it, the longer you have to assimilate information pertaining to the answer.
Durr (Score:4, Insightful)
Um, three weeks is plenty of time to look up such an intriguing factoid on the Internet.
Should this be a surprise? (Score:5, Insightful)
I thought this was understood.
This is how you are able to catch a ball. Your brain doesn't do a physics calculation and determine where the ball will land. It guesses, watches, refines the guess, repeats, and eventually the guess is close enough so your hand is in the right spot to catch it.
Re:Ah duh! (Score:5, Insightful)
The point is that while thinking long and hard about some problems can be helpful (e.g. designing something complex and technical), for other kinds of problems, added thought can hinder (e.g. when there are many confounding unknowns).
Re:Explains (Score:5, Insightful)
No, I don't think so. It wouldn't be "one of them is bound to be right" -- it would be something more along the lines of "with enough posts, the consensus is likely to be close to reality."
This assumes, of course, that everything in life is like a jar of jellybeans.
Re:Should this be a surprise? (Score:1, Insightful)
...which ultimately amounts to doing a physics calculation anyway, just using training rather than a priori formulas.
I call bs on the concept.. (Score:2, Insightful)
I read through the first few chapters of James Surowiecki's book in the bookstore. The only thing I found was a small (statistically speaking) number of anecdotes. Nothing really well researched (perhaps there were actual studies done later on).
I would say my main gripe is that the idea is often presented in an extremely poor manner. Like the author above does with the jelly beans.
It implies that the "popular mean" can express knowledge that isn't strongly represented in the group already. I.e. Clearly people voting on what medical procedure should be done for a given set of symptoms is radically different than people voting on what they would like to be fed for breakfast and likely puts the patient in a worse position rather than a better one. Now I get that with the idea of jelly beans is the belief that more people with overestimate or underestimate than guess right and that these two sides "balance" but, to my knowledge anyway that hasn't been actually demonstrated in a statistically valid way or for that matter in a way where proper bias control was done (the first example in the book IIRC was about the weight of cattle - clearly that could be biased by the sample used - especially since it was self-selected)
This brings us to the question: "How is this useful?" It doesn't introduce us to a new concept. We already believe that the "popular mean" is a better judge of some things but not others. It doesn't give us any better idea HOW to judge which things are better judged by crowds and which do not.
Re:Ah duh! (Score:4, Insightful)
They didn't say that the second answer was better. They said that the average was better. It would be interesting to know if the second answer was, on average, better than the first.
Re:Explains (Score:4, Insightful)
The "generate and test" idea is something I've made great effort to more consciously embrace in my creative endeavors. People decry "quantity over quality," but what I've found is that you simply can't just brood over an idea and "work on" the idea until it's "perfect" and then execute it--you have to create prototypes and test them, and the more you do this, the better you get at creating good prototypes in the first place. Still, it's remarkable how difficult it can be to convince yourself of this.
Re:In related news... (Score:4, Insightful)
You state the really cool thing about this but somehow completely miss it!
You say, "If the guesses are distributed around the correct value...." Well, why would they be? They're guesses! There's no reason to expect one person's guesses to be centered on the correct value if they don't know the correct value. But this study shows that they are centered near the correct value, even though the person doesn't know what that value is.
Re:In related news... (Score:2, Insightful)
The Gaussian distribution is completely unnecessary. The only necessity for the law of increasing averages to hold is that the distribution is centered on the average.
Re:In related news... (Score:4, Insightful)
That would be a flaw if I ever discussed "a bunch of people", but I never did.
The interesting thing here is not that the individual can guess a number close to the true value. What's interesting is that if he guesses more than once, the average is closer to the true value than his initial guess. This is unexpected and a little weird.