"Wisdom of Crowds" Works For Individuals Too 158
ideonexus writes "Take a crowd of people and have them guess how many jelly beans are in a jar, and the average of their answers will be remarkably accurate. Now researchers have found the same goes for asking one person to guess about the same thing several times. Accuracy improved when the individual was given longer periods of time between guesses." The anonymous author of the Economist piece, not quoting the researchers, says the finding bolsters the "generate and test" model of creative thinking.
In related news... (Score:4, Funny)
In related news, students were found to do far better on multiple choice tests when given an unlimited number of guesses at each question. Even students that didn't study eventually got As.
Re:In related news... (Score:5, Insightful)
Not quite... but you are close. It sounds like you're pointing out that anyone will get lucky if given enough chances. These guys are claiming that the average will converge to the ground truth over time. This would need to have guesses with some Gaussian distribution about the correct answer.
If the guesses were uniformly distributed then the average wouldn't tend to the correct answer over time. Of course what is described in the summary has nothing to do with the wisdom of crowds as it is commonly thought of (i.e in markets) where shared information is vital. Instead it is simply an artifact of sampling (which is why the longer gaps are necessary for better accuracy)
Re: (Score:2)
I'd hazard a guess that us oldies would need longer between guesses than the young 'uns, since we're more likely to remember the previous guesses than a rap-music listening, zero attention span teen.
That is, the effect of increased accuracy should fall off with age.
Re: (Score:2)
That's an interesting idea. It would be a good follow up to measure the short-term attention span of the subjects and see if there was a correlation between memory and accuracy.
In general though, doesn't short-term memory decrease with age? Or don't you remember what we were talking about :)
Re:In related news... (Score:4, Informative)
Re:In related news... (Score:4, Insightful)
You state the really cool thing about this but somehow completely miss it!
You say, "If the guesses are distributed around the correct value...." Well, why would they be? They're guesses! There's no reason to expect one person's guesses to be centered on the correct value if they don't know the correct value. But this study shows that they are centered near the correct value, even though the person doesn't know what that value is.
Re: (Score:2)
They are guesses, but not random guesses.
Re: (Score:2)
They have some degree of randomness. You would expect a second guess to be close to the first guess, but there's no reason to expect the average of multiple guesses to be closer to the true value.
Re: (Score:2)
Well yes, there is some degree of randomness, but that is different from random. If they were random then a guess of 1 would be as likely as a guess of 1,000,000. Because the guesses are informed by an approximation of the size of the jar and its contents, answers that are way off are less likely than guess that are in the ballpark.
Re: (Score:2)
You're confusing "random" with "random along an even distribution". There are many random distributions, and most of them won't put the probability of 1 equal to the probability of 1 million.
From my naive grasp of statistics I'd expect an individual's guesses to be randomly distributed along a bell curve, with the center of the bell curve based on that person's knowledge of the situation. The more he knows, the more accurate his guesses will be, and the closer the center of that curve will be to reality.
But
Re: (Score:2)
No, that's not what the study shows. The study shows the average of all the guesses made approaches the correct answer. It does not say that there is any increase in the accuracy of subsequent guesses on their own.
Re: (Score:2)
True, enough, but the only way for the one to happen without the other is if the other guesses stay equally far away but hop back and forth on either side of the correct value. However it happens, it's something that you would not expect to see.
Re:In related news... (Score:4, Insightful)
That would be a flaw if I ever discussed "a bunch of people", but I never did.
The interesting thing here is not that the individual can guess a number close to the true value. What's interesting is that if he guesses more than once, the average is closer to the true value than his initial guess. This is unexpected and a little weird.
Re: (Score:2)
There's no real meaningful difference. "Estimate" implies that more thought and evidence went into the result but it does not require it.
Re: (Score:2)
I didn't say on the correct value, I said near. I should have said nearer.
Re: (Score:2)
Sorry I was a bit unclear with the way that I phrased it. I did mean the literal comparison between a "Guassian distribution about the correct answer" and just a uniform distribution i.e. not about the correct answer. Of course you are correct that any symmetric distribution would work as long as it was was centered around the correct solution.
It is with an image of Ferris Bueller playing the sax(?) that I say: and without a single stats class ever :)
Check out the Central Limit Theorem and be amazed! (Score:2)
GP said:
and you said
You are both right. However, GP is more correct than you.
You speak of a distribution where the expected value is the correct solution, but you also said that it was a "symmetric" distribution (and thus it would be "centered" around the correct solution). This is sufficient but not necessary. Amazingly enough, no matter how asymmet
Re:Check out the Central Limit Theorem and be amaz (Score:2)
Well spotted, and a very good explanation.
*bows*
For a clear example (video): (Score:2)
Have a look at this Derron Brown episode! [youtube.com]
Re: (Score:3, Interesting)
Good observation about the Gaussian distribution being necessary. Thought experiment: I am thinking of a number between one and a million. What's the likelihood that the average of a bunch of people's guesses are anywhere near the number I am thinking of?
Re: (Score:2)
Re: (Score:2, Insightful)
The Gaussian distribution is completely unnecessary. The only necessity for the law of increasing averages to hold is that the distribution is centered on the average.
It depends... (Score:2)
It depends... are you thinking of the number 500000 :D
Re: (Score:2)
Actually, this would be an interesting experiment to try out just for that reason. People are notoriously bad at random number generation; I bet the mean guess wouldn't be near 500000.
Re: (Score:2)
Since people can see the jelly beans before a guess, but have no reference at all to what number you could be thinking(70,004) I don't think the comparison is fair.
Re: (Score:2)
It depends. 0 right now, because you thought of the question, and then choose a sufficent outlier to make the number unlikely to be reached.
First posts? (Score:1)
I'm going to guess there will be 3 "first posts" on this...
What time is it? I guess 03:04PM (Score:2)
In other news, I prefer cold drinks, hot women, and slow dances. Let's average everyone's opinion on those too.
So how long is the emperor of China's nose? (Score:5, Insightful)
Re: (Score:2)
Re:So how long is the emperor of China's nose? (Score:5, Insightful)
Exactly. Penn and Teller asked a group of people if the chemical Dihydrous Monoxide should be banned. Nearly every one of them said yes. The wisdom of crowds is not in and of itself some sort of magic. It is merely an interesting observation.
That your own guesses seem to exhibit the same 'average' correctness as a crowd is bad science IMO. Once you guess at a problem, you're subconsciously directed to think of that problem, thus getting more than a knee jerk reactionary guess. The longer you have to think about it, the longer you have to assimilate information pertaining to the answer.
Re: (Score:2)
Once you guess at a problem, you're subconsciously directed to think of that problem, thus getting more than a knee jerk reactionary guess. The longer you have to think about it, the longer you have to assimilate information pertaining to the answer.
Or maybe after 3 weeks you've forgotten your previous answer or have lost the psychological attachment to it and are not as nearly as likely to pick a new number in close proximity to the original.
What I'm saying is that maybe 6%* just represents the avg amount individuals are willing to stray from their original guess.
What the study really needed was intermediate data points between "immediately" and "3 weeks".
*shouldn't it be a 12% difference between the 1st & 2nd guess, which avgs to 6%?
Re: (Score:2)
Exactly. Penn and Teller asked a group of people if the chemical Dihydrous Monoxide should be banned. Nearly every one of them said yes. The wisdom of crowds is not in and of itself some sort of magic. It is merely an interesting observation.
The problem with your example is that it is not remotely comparable. That is not a guess, that is just being stupid.
Re: (Score:3, Interesting)
Nearly every one of them said yes.
The problem with this particular example from P&T (who are awesome despite doing this from time to time) is that this isn't an appeal to find a concrete value or fact, it is instead an appeal to a person's knowledge. Just like the questions asking about weapons of mass destruction were framed in a manner which directly appealed to the information people were being fed by the administration and in turn the media.
Had the questions been framed more like "If Iraq possesses
Re: (Score:2)
I do a fair bit of survey-based research and here's the thing: You always put your instrument in the appendix. If you're going to be making claims about what you found in your research, you need to show the goods and let people decide if they are founded or not.
This is my irritation with all reporting of all statistics to the general population. People are told useless things like "10% increase" or "higher average," etc., but without knowing the design of the experiment and at least the rest of the des
Re: (Score:2, Informative)
Most polls said people expected that we would find WMD and WMDs were found.
Re: (Score:2)
That's right. WMDs were even used... Though not by Iraqis: http://news.bbc.co.uk/2/hi/middle_east/4440664.stm [bbc.co.uk]
Re: (Score:2)
Re: (Score:3, Informative)
How is this informative? This is wrong. Read the wikipedia article [wikipedia.org]. Money quote:
Re: (Score:2)
It does not take the large quantities and all the world's intelligences, excluding a few places like Cuba, said existed in order to make the original posters statement a lie. BTW read General Franks complete speech he was also talking about the large quantities that the intelligence agencies talked about.
The above definition of lie is the dictionary de
Re: (Score:2)
Wisdom of crowds only works when the crowd has some information about the situation.
Or when the crowd isn't a self-reinforcing/recruiting echo chamber for ignorant fuckwittery..... like the one-time alleged poster boy of the movement, Digg.
Re: (Score:2)
A little biased (Score:5, Funny)
"600 jelly beans?"
"Higher"
"900?"
"Looower...."
How to guess jelly beans (Score:2)
The trick to guessing how many jelly beans or whatever are in a jar is to estimate how many beans tall wide and long the jar is and then multiply it out as if the jar was rectangular. You can then take off a little bit to adjust for a round rather than rectangular jar, but your estimate will be so inaccurate anyway that a little bit like that won't matter much anyway. You'll still be way off, but much closer than anyone that just looks at it and makes a guess.
If the items in the container are very small li
The Delphi Method (Score:5, Interesting)
Another product of the RAND Corporation.
From the I-am-large-I-contain-multitudes dept.? (Score:5, Funny)
Do I contradict myself?
Very well then! I contradict myself!
Re: (Score:3, Funny)
My name is Legion, for we are many
Wisdom of the Crowds" (Score:5, Insightful)
The idea that a group guessing is more accurate than an individual guess, and if you make more than one guess the mean or average of the guesses is more accurate than a single guess?
So, in real world terms, 1000 rednecks are going to be more accurate than one Harvard graduate? (assuming the graduate in question isn't our current President) (if we were guessing the number of pickled eggs in a pickle jar, I'd have to agree... Otherwise, I'm somewhat skeptical of how well this translates beyond the estimation of things.
Re:Wisdom of the Crowds" (Score:5, Funny)
Re: (Score:3, Interesting)
OT : Is "redneck" the new black? (Score:2)
I regularly observe on this site that a "redneck" is stereotypically utterly stupid, and it reminds me of the now universally unacceptable stereotype that black people were dumb. So is "redneck" the new black, in that it has become our new 'generic group of intellectually inferior people' with the advantage of not being deemed racist because "redneck" is conveniently not regarded as a race? Doesn't it make us bigots anyways to pick regularly on this group of people? Am I the only one who feels offended desp
Re: (Score:2)
Re: (Score:2)
Being a "redneck" is a lifestyle choice and not a biological or geographical designation, just like being an asshole.
*Bzzzp*, wrong. Originally, rednecks [wikipedia.org] are Celts (which is a very "biological" or rather ethnic thing) from Ireland and Scotland who emigrated to the Appalachians and the South of the USA, and are known for being poor and poorly educated. Where's the choice in that? If your parents are rednecks, then you'll be a redneck too. Rednecks are as much a precise group as the Amish are. Now how is it not bigotry to hate them like we do?
Oh wait, we cannot be bigots, we're oh so well educated liberal and libertarians g
Re: (Score:2)
Except that's not masses, that's two selected groups.
Explains (Score:5, Funny)
This explains why there's so much informative discussion here at slashdot. N o one knows much of anything, but if you throw enough wild assed guesses at something, one of them is bound to be right, right?
Re:Explains (Score:5, Insightful)
No, I don't think so. It wouldn't be "one of them is bound to be right" -- it would be something more along the lines of "with enough posts, the consensus is likely to be close to reality."
This assumes, of course, that everything in life is like a jar of jellybeans.
Re: (Score:2, Funny)
This assumes, of course, that everything in life is like a jar of jellybeans.
Unfortunately, it's not. Life is like a box of chocolates. Ask Forrest's momma.
Re: (Score:2)
I wonder if the lower one reads at, them more inaccurate the guess becomes?
Re: (Score:1)
I'm not sure if you're joking or not, but no, it's something quite different. The ideas they're talking about (or at least the idea involving multiple people) is that with a large number of guesses there's a good chance that the majority of guesses will balance out between being too high or too low allowing an average of those guesses to be relatively accurate.
On Slashdot each post is a disparate peice of information, you can't average anything. The number of informative/interesting/insightful posts is a mu
Re: (Score:2)
Actually I was kidding, really. It just wasn't that funny.
Re:Explains (Score:4, Insightful)
The "generate and test" idea is something I've made great effort to more consciously embrace in my creative endeavors. People decry "quantity over quality," but what I've found is that you simply can't just brood over an idea and "work on" the idea until it's "perfect" and then execute it--you have to create prototypes and test them, and the more you do this, the better you get at creating good prototypes in the first place. Still, it's remarkable how difficult it can be to convince yourself of this.
Re: (Score:2)
intuition is a product of subconscious information processing. The brain is a pattern-recognition machine, and is generally very good at that. I would hazard a guess that if you average out everybody's intuitions ("first guesses"), some of the people are "overthinking" things, but many are just going with their gut, and the pattern recognition and extrapolation that's going on constantly anyway in your brain is often onto something.
I think you hit on why I like Derron Brown's shows so much. He actively gets people to perform amazing feats [youtube.com] by (among other things) getting them to listen to their subconscious. He even goes to the extent of using hypnosis.
Re: (Score:2)
BTW, a colleague of mine used to live in the same building as him, and used to hang out with him. Which is quite cool. (Unless you're lying, Dowie!)
Re: (Score:2)
Your comment violated the "postercomment" compression filter. Try less whitespace and/or less repetition.
Huh, that's a new one.
Gubmint (Score:2)
N.B. -- this does not apply to politics. In fact, the phrase "Political Science" may be turning into the biggest oxymoron of all time.
Measure Twice, Cut Once (Score:2)
Ah duh! (Score:5, Funny)
Re:Ah duh! (Score:4, Funny)
Re:Ah duh! (Score:5, Insightful)
The point is that while thinking long and hard about some problems can be helpful (e.g. designing something complex and technical), for other kinds of problems, added thought can hinder (e.g. when there are many confounding unknowns).
Re:Ah duh! (Score:5, Funny)
The point is that while thinking long and hard about some problems can be helpful (e.g. designing something complex and technical), for other kinds of problems, added thought can hinder (e.g. when there are many confounding unknowns).
So that explains why most /.ers are single.
Re: (Score:2)
For once, a post on Slashdot about singleness that's actually a seriously good point. I should think about these things WAY less. Mod parent informative.
Re: (Score:2)
Re:Ah duh! (Score:4, Insightful)
They didn't say that the second answer was better. They said that the average was better. It would be interesting to know if the second answer was, on average, better than the first.
Re: (Score:3, Interesting)
I predict that if you ask the same person the same question over and over again even at wide intervals the answers will converge but not necessarily to the correct value (of course, that value may often be "You asked me that before. Bugger off!")
yea, no (Score:2, Funny)
Yes, a little right. No, not at all. Total bullshit. Yet also 100% right. Doorknob. Right about 30% of the time. Wait, what was the question?
Wisdom of crowds? (Score:2)
Usually what you get out of crowds is some form of mob rule, not wisdom.
Seriously? (Score:1)
Durr (Score:4, Insightful)
Um, three weeks is plenty of time to look up such an intriguing factoid on the Internet.
Pure Brain Power? (Score:1)
Should this be a surprise? (Score:5, Insightful)
I thought this was understood.
This is how you are able to catch a ball. Your brain doesn't do a physics calculation and determine where the ball will land. It guesses, watches, refines the guess, repeats, and eventually the guess is close enough so your hand is in the right spot to catch it.
Re: (Score:1, Insightful)
...which ultimately amounts to doing a physics calculation anyway, just using training rather than a priori formulas.
Re: (Score:3, Funny)
This is how you are able to catch a ball. Your brain doesn't do a physics calculation and determine where the ball will land. It guesses, watches, refines the guess, repeats, and eventually the guess is close enough so your hand is in the right spot to catch it.
Apparently that feature of the brain is broken for the players of the [Insert Name of Hated Sports Team].
(Hey...I try to make my put downs fun for everybody!)
Re: (Score:2)
No it isn't. We're nerds for a reason.
Re: (Score:2)
I call bs on the concept.. (Score:2, Insightful)
I read through the first few chapters of James Surowiecki's book in the bookstore. The only thing I found was a small (statistically speaking) number of anecdotes. Nothing really well researched (perhaps there were actual studies done later on).
I would say my main gripe is that the idea is often presented in an extremely poor manner. Like the author above does with the jelly beans.
It implies that the "popular mean" can express knowledge that isn't strongly represented in the group already. I.e. Clearly
Sorry, can't resist... (Score:3, Funny)
Crowds not wise in sports betting, that for sure (Score:2)
I was deeply involved in sports betting for a while. One of the first things any serious sports bettor learns is that if everyone else likes the same team as you, then start to worry.
There are several websites dedicated to come up with a public consensus on wagers. They were always a must-see for me but only as another piece of information (oddly, it's just as dumb to bet along with the crowd as it is to bet against it.)
Yes, this is gambling, but it's not like betting red or black in roulette. There here ha
Short vid about Wisdom of the Crowds (Score:3, Interesting)
Intervals (Score:2)
It sounds like this is just an example of someone making a confidence interval; "I know the answer is in a certain range, so I'll make two guesses and it should probably be in there."
If each guess is made using a different model, then you're adding more "information" to the guess. Then there's more total information in the average, than in each guess on it's own.
But what do I know, I'm not a psychologist. I could just be making stuff up.
The wisdom of averages (Score:4, Interesting)
Sounds Strangely Like (Score:2)
Forest for the trees. (Score:2)
Things get awkward when people talk about harnessing the power of crowds to improve complex predictions. A question like, "What will the price of oil be in five years?" is damn near impossible to answer without a time machine because it completely fails to factor in unpredictable economic & political disruptions (9/11, some guy inventing a portable fusion generator in his basement, alien invasion, global war, or pandemic). In fact, the financial markets are a great example of "the wisdom of crowds" loo
That's What The Mice Wanted (Score:2, Funny)
Minsky's Emotion Machine (Score:3, Interesting)
Re: (Score:2)
You needed the pseudoscientific research of a charlatan like Minsky to figure out why blitz chess and tournament chess are different? Minsky so has you in the palm of his hand that he's convinced you that you need his help to understand something that you already know and understand and have understood since childhood. As long as people like Minsky can convince intelligent people like you that you are ignorant, as he's clearly
Re: (Score:2)
>>It's also why we often go to sleep working on a tough (programming etc.) problem and wake up with the
>>answer - our "unconscious" brain put the answer together while we slept.
I always kind of thought that if you go to sleep thinking about a problem, it means that you're tired. If you're tired, you are capable of only a fraction of your normal brain function. When you wake up well-rested, your time spent thinking (say from waking up until 10 minutes later) is simply how long it would have taken
noise reduction (Score:2)
This is just like noise reduction. You either shoot with more sensors at the same time, and do an average between them to eliminate sensor noise, or shoot with the same sensor repeatedly and average with itself.
The noise and other sensor defects are reduced.
Apparently, it's the same with people.
The original article (Score:2)
Accuracy of mass randomness (Score:2)
This sounds like an experiment a coworker once relayed to me: Given a object you want to measure, drop a ruler onto it from a height of a few feet. Do this repeatedly and each time record the value on the ruler at the edge of the object. The average of a sufficient number of such values will approach the actual measurement.
Just applied to humans. Go figure, the world is self-similar like that.
All hail chaos.
Re: (Score:2)
As an individual, you wouldn't be able to understand. At least not right away. Keep trying.