Busting the MythBusters' Yawn Experiment 397
markmcb writes "Most everyone knows and loves the MythBusters, two guys who attempt to set the story straight on things people just take for granted. Well, maybe everyone except Brandon Hansen, who has offered them a taste of their own medicine as he busts the MythBusters' improper use of statistics in their experiment to determine whether yawning is contagious. While the article maintains that the contagion of yawns is still a possibility, Hansen is clearly giving the MythBusters no credit for proving such a claim, 'not with a correlation coefficient of .045835.'"
Well... (Score:5, Insightful)
Re: (Score:2)
Re:Well... (Score:5, Insightful)
TV is entertainment, not science (Score:5, Insightful)
Re: (Score:3, Interesting)
Re: (Score:3, Insightful)
Isn't the real crux of science documentation and repeatability? I mean, if someone comes by examines their methods, and finds out that they did it wrong and can show it, isn't that proof that they're acting in the spirit of science?
Re: (Score:3, Insightful)
Layne
Re: (Score:3, Funny)
Re:Well... (Score:5, Insightful)
Besides, I think most of us already know that the best ways to test most myths would be so boring it would never make TV in the first place.
Re:Well... (Score:5, Funny)
Submitter gets an F on this one (Score:4, Insightful)
Re: (Score:3, Insightful)
I mean, since there can't be any fractions of a person, if we know there are 50 people, we know that there are 50.0, 50.00, 50.000000000000 people, right?
It doesn't seem like sig-figs is applicable here.
Re: (Score:2, Redundant)
Re:Submitter gets an F on this one (Score:5, Funny)
1.54? Are you sure? Do you work for NASA?
Re:Submitter gets an F on this one (Score:5, Informative)
The number of significant figures in an answer depends on how the function propagates errors. It's INCORRECT in general to think that if the inputs are given with two significant digits (say), then the output is only good for two significant digits.
The CORRECT way is to perform error analysis [wikipedia.org] on the function being computed. If the function is linear, then the error magnitude is essentially multiplied by a constant. If that constant is close to 1 (and only then) will the output accuracy be close to the input accuracy.
In general, a function being computed is nonlinear, and the resulting number of significant digits can be either more or less than for the input. Examples are chaotic systems (high accuracy in input -> low accuracy in output) or stable attractive systems (low accuracy in input -> high accuracy in output).
Re: (Score:3, Informative)
The mean of 1.0 and 4.0 on the other hand is 2.5. Assuming you know your '1' and '4' in the above to that level of accuracy. Otherwise especially your 4 could be anywhere from 3.0_1 to 4.9_9 because it might only be accurate to half a digit.
Re: (Score:3, Insightful)
You're making the assumption that either we cannot measure anything to the precision guarenteed by a scale, or that we can measure past the precision guarenteed by a scale. In either case you'd be wrong. In the former case, you'd even be suggesting that measurement is
Re: (Score:3, Insightful)
Sure, it's popular on
Re:Well... (Score:4, Insightful)
Re:Well... (Score:4, Funny)
Re:Well... (Score:5, Insightful)
Mythbusters is no different than Bill Nye, Mr. Wizard or who ever the hell came first. They use the same basic methods for all of their problems. There are some differences though:
When I was ten I know I'd much rather watch two guys drive two semis into a small economy car rather than watch Mr. Wizard mix baking soda and vinegar again and again. Mythbusters rocks, because it is exactly what the 10+ set is capable of and it also shows them the constraints of their knowledge because the Mythbusters actually do discredit themselves on the show, you'll hear them say things like "I think you're way off base with your method" or "I'm really happy with the results" and if you hear that from the old guy in the beret it's usually because it was an effective (or ineffective as the case may be) low-level experiment. It's a simple formula:
Now I'm not saying that all of their experiments are 100% right for all levels of science, I'm just suggesting that they are about as good as you get with pre-algebra to algebra level math. And that isn't that bad, after all that's where we get things like the lever, steam engine, plumbing, and a lot of other cool crap (like higher math). I remember building a trebuchet for a lower level physics class (10*?), they mostly sucked but we did the algebra (Newtonian mechanics) some of us got A's, most of us didn't, but when we were done we had learned a little (by trial and error) about trajectories and conflicting forces, not to mention recording our results. It wasn't in vain, it was a nice precursor for things to come. Between Mythbusters and American Idol I'd easily rather have my kids watch Mythbusters even if they're wrong 80% of the time.
I'm not apologizing for the Mythbusters in the least.
Re: (Score:3, Insightful)
Re:Well... (Score:5, Interesting)
Yes it fucking well is a bad thing when they don't teach you how to do it. They're teaching skepticism but then they're teaching hillbilly scientific practice instead of logic and the scientific method. The result is you get a bunch of kids who are rude, and think they know everything just because they can provide a counter-argument backed up with nothing but the shoddiest proof. That is very much a bad thing.
The Mythbusters basically piss on the scientific method in every show, drawing wild conclusions from a single lll thought out experiment, often with no controls (or weak ones), and often testing a single instance or brand and then generalising for all of that type of product.
Another poster put it correctly. People watch because they blow shit up, which is fine as far as entertainment goes. However no other show presents bad pseudo-science as science and fucks up the minds of kids who then think they understand science, when at best they understand skepticism.
Every time I've said this here I've been modded down but fuck it, it needs to be said.
Re:Well... (Score:5, Insightful)
Re:Well... (Score:5, Interesting)
At the very least, scientific TV shows encourage people to learn more about science and the scientific method.
Carl Sagan taught me more about science with his Cosmos series (that has stuck with me) than any government school ever did. When I heard about this search engine named "Google" back on Slashdot so many years ago, I can still remember thinking back to the Cosmos episode where Sagan was talking about large numbers, like googol and googolplex. To see him try to roll out a piece of paper not with a googolplex of numbers on it, but merely the standard notation of googolplex (1 followed by a googol zeroes), it sticks with you. And on the smaller scale, to watch him place a drop of oil on a lake, and come back an hour later to explain that the entire surface of the lake now had a microscopic layer of oil across the entire surface. Or to demonstrate Einstein's theories of gravity with a stretchy sheet of material and some heavy balls of different sizes. Or demonstrating the 4th dimension by showing a "shadow" of a 4th dimensional item as a 3 dimensional item, much as we can see the shadow of a 3 dimensional item drawn on paper. I haven't seen Cosmos in a decade, and can still remember things he talked about.
This is something government schools rarely ever do, unless you happen to be assigned to the one-in-a-million inspirational teacher.
Another example -- planet earth, now running on Discovery HD Theatre. An absolutely stunning piece of scientifically interesting video.
Re: (Score:3, Interesting)
It's only a good thing if they're first taught how to think critically and how to prove things. Too much of people "going out and proving things for themselves" involves three steps;
1/ Whoah! This is very complicated! It would take years of study to fully understand it.
2/ Screw that, I'm just going to apply my in depth knowledge of what looks right, and what seems to me
Re: (Score:3, Insightful)
It's only a good thing if they're first taught how to think critically and how to prove things. Too much of people "going out and proving things for themselves" involves three steps;
<trim>
It all depends on what your question is. "Do things always fall down?" is quite a different question than "Why do things always fall down?" Describing the nature of gravity is an entirely different field than describing its effects. There is a lot more math and laboratory rigor in designing and proving cold fusion than there is in launching a car off of a large mound of dirt to see if it would still be drivable*. There is also a lot less of the process that would make for compelling television. The drive,
Re: (Score:2)
Its "Science entertainment", much as the super 12, rugby 7's or one day cricket are "Sports entertainment".
Sorry I can't give any USA-centric examples of sports entertainment... maybe baseball?
Re: (Score:2)
well... truthfully... (Score:5, Insightful)
not to mention that they always try to prove stupid crap like "a rolling stone gathers no moss". I'm waiting for them to try "the grass is always greener on the other side", or "it takes one to know one".
Re: (Score:3, Informative)
But to test this, they used SUVs (if you are concerned about fuel efficiency, are you driving one?) going at about 40 mph (air drag I think increases by the square of the speed at those speeds, so highway speeds could significantly change the re
Re: (Score:2, Insightful)
Re: (Score:2)
Re:well... truthfully... (Score:5, Insightful)
Re:well... truthfully... (Score:4, Interesting)
Re:well... truthfully... (Score:4, Insightful)
Speaking of that: You seem to have some misunderstanding of their conclusions: They specifically stated that high-velocity weapons seemed to have problems penetrating the surface, not that 3 feet of water will keep you safe from any gun.
Their testing seemed pretty good to me (not exactly scientific, but enough to warrant the claim "supersonic bullets in general do not seem to be effective after a few feet in water"), and you have so far provided zero evidence to the contrary. Come on, why just say "there are other arguments as well" -- if you know about some evidence, please link to it and don't just weasel out...
Re: (Score:3, Interesting)
Re: (Score:2)
is perfectly reasonable and useful.
Re: (Score:2, Informative)
Re: (Score:3, Informative)
Wiki has the details: http://en.wikipedia.org/wiki/MythBusters_(season_3 )#AC_vs._Windows_Down [wikipedia.org]
Re: (Score:3, Insightful)
What was stupid is how small their sample was - they were planing on driving a whole tank off but then said that would take too long so they sucked it down to a gallon in each car or something. Which I don't think is a fair test; how do you know if the AC performs better as it runs longer or something?
Also it'd dumb that they had the AC running full blast as that's not a realistic scenario - once the c
Re: (Score:2)
Re: (Score:2)
I've wanted to write an article like that for a while now.
Re: (Score:2)
Re: (Score:3, Informative)
That's one thing I've seen a lot online. People watch maybe 80% of the show and then go online and say "they made a gross procedural error!" when in fact they're testing something subtly different or not doing what those people think. From
Re: (Score:3, Informative)
Or, more likely, test a subset and claim it applies to the entire set.
E.g., the Cell phones on a plane [wikipedia.org] episode, where they claim a cellphone will not interfere with the avionics of a jet. Unfortunately, this is true for the cellphones they tested, plus the jet they tested. It unfortunately does
Precision? (Score:5, Insightful)
These "statistics" are COMPLETELY flawed. (Score:3, Informative)
The Fact that they use a standard deviation to test an Hypothesis, you know, instead of Hypothesis Testing [wikipedia.org] makes me certain that he doesn't know jack about statistics.
you do _NOT_ use descriptive statistics to study samples!!!
I can't believe how wrong this analysis is... What you're supposed to test is that when seeded with a yawn, you're more susceptible then wh
Re:Precision? (Score:5, Funny)
Heck, I yawned when I read the summary.
Uh... wait a second... this is Slashdot. Why are you reading the article!?! :-)
Conversation slayer (Score:4, Funny)
Mythbusters is not scientific (Score:5, Interesting)
Sometimes they don't things more than once (even when required), other times they don't adequately recreate the conditions of the "myth."
The show is entertaining as hell, and sometimes they do conclusively prove things.
Re: (Score:3, Insightful)
Re: (Score:2, Funny)
{Commercial break. }
{ Adam and Jamie read slughead's comment. }
{ Jamie chuckles. Adam laughs really hard, pisses his pants, and continues to laugh really hard. }
{ Commercial break. }
Announcer: BUSTED! slughead can apparently only state nothing but the blatantly obvious!
Re:Mythbusters is not scientific (Score:5, Informative)
Surprising how many people take them seriously! (Score:2, Insightful)
I'm all for watching their show for its entertainm
Re: (Score:2)
My favorite episode is when they proved that diving in to water is effective in evading gunfire. They placed a gelatin mold 18inches under water in a swimming pool and fired a .50cal at it. The gelatin wasn't pierced. They repeated this test time and time again at several calibers (IIRC even went to 12inches below surface).
Sometimes they don't do so good, but other times they do extremely well.
Re: (Score:2)
So let's do it like they do on the telly Myth Busters.
Re: (Score:2, Insightful)
Re: (Score:2)
Re: (Score:3, Funny)
R2-D2, Official Operator? This guy has more nerd creds than everyone in this thread put together!
That charlatan.... (Score:3, Funny)
Science (Score:5, Insightful)
The above is why I wouldn't trust Mythbusters as far as I could throw them. The entire show screams entertainment rather than Science. Unfortunaely I can't find the name of a program that aired in the UK about 6 months ago. It took a team of 4 people to a deserted island and each week they had a task to complete each, they were only allowed to use what was on the island and what was given to them each week (as well as a tool set because, well no tools = screwed). They had to do things like make fireworks, record a song and various other "minor" things which required them to render down various things to achieve the chemicals they needed to complete each task. What they did and what it resulted in was very clearly labeled, having real science explained behind it.
Saddly as I recall it basicly got replaced with some crappy school based soap opera where the kids say "innit" and the teachers fuck anything with two legs (including the kids as the current trailer at least implies). So after this long rant, I guess we just give up on science and go back to the discoery channel, maybe we can catch the 3 minutes of it that isn't Nazis or some form of sport!
Re:Science (Score:5, Informative)
Would that be Rough Science [open2.net]? In particular, it sounds like the second series [wikipedia.org]. I've seen a couple of the series over the past few years, and I believe it did a pretty good job of being a science show – the interest comes from watching people who actually know what they're doing, designing and building ingenious solutions (admittedly with very convenient tools and materials available) to problems that aren't inherently interesting (like making toothpaste or measuring the speed of a glacier), rather than relying on 'interesting' problems that are large/dangerous/explosive and lacking focus on the solution process.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re:Science (Score:5, Interesting)
Of course that's not to say there isn't room for more demanding science shows on television, and you cite a good example, because whether TV forces you to think or not is purely down to the quality of the programming. There is a serious issue in terms of the bias TV has towards undemanding entertainment, but where should the blame lie? Ultimately the people behind these stations are trying to make money, and they do that by giving people what they want (or what they think they want). We've created a monster.
Lies (Score:2)
Lies, Damn Lies and !!!!
Not quite, OmniNerd (Score:5, Informative)
Re: (Score:2)
It always bugs me that they don't even try and collect data to fit a normal distribution or do any proper means testing. But oh well, it's TV, thems the breaks.
references (Score:3, Insightful)
reference 5 is an episode that won't air for 2 days (maybe he's from the future!)
references 7 and 8 are forum posts (ref. 8 has just 2 replies)
two references are news stories..
these do not suggest a thorough exploration of the matter, but he cites them as if they are authoritative sources
Whoa there... (Score:3, Informative)
Re: (Score:3, Interesting)
That doesn't mean that the Ministers are running around getting under-age girls pregnant, but rather that during the drought people turned to "simple pleasures" and sought spiritual easing of their hardships. Often a high correlation im
Re: (Score:2)
The image above is not safe for work, nor is it particularly funny.
Doesn't anyone know statistics any more? (Score:5, Informative)
There is a very well-known test, the chi-square test, that deals with exactly this case. (Given the small sample sizes, the Fisher exact test may give better results.) Someone should point Hansen to the Wikipedia page on the topic.
For example, if there are 16 non-primed people, with 4 yawning and 12 not (for 25%), and there are 34 primed people, with 10 yawning and 24 not (for 29%), the chi square test gives a p value of 0.74.
The values Hansen supposes are significant 4,12 and 12,24 are not: p = 0.29.
You have to go all the way to 4,12 and 17,19 (i.e. 47% on a sample of 36) to get significance.
MythBusters was wrong to conclude that their results were significant, but Hansen was equally wrong to conclude that he had shown that Mythbusters was wrong.
Re:Doesn't anyone know statistics any more? (Score:5, Informative)
But all this aside, I'm not sure I like the experiment. Why bore people? Why have so many in the room. the 4,12 number is way too high, I'd say the were better off looking at narrow time slices and natural yawns (i.e. do yawns happen at random or do they set off avalanches). Then there is only one group and you're just testing the Poisson process assumption of uncorrelatedness.
Re: (Score:2)
Re: (Score:2)
The advantage of their way is that you know the causality of an increased yawning rate. In your design, it's harder to rule out temperature fluctuations or conversations about, say, the appropriateness of a correlation test for binary data.
Busting the MythBusters busters (Score:3, Informative)
MythBusters numbers may mean that someone is 20% more likely to yawn if seeded. Now, what's important is to evaluate the margin of error for this statement given the sample size.
What the article is definitely wrong about is that the sample size does not change anything. The sample size basically reduces the probability of error. The higher the sample size, the more likely that the statement "someone is 20% more likely to yawn if seeded" is true. However, at their sample size, it is not unlikely that the error marging is comparable with that 20% difference, which would invalidate the experiment.
The detailed calculations for sufficient sample size are left as an excercise for the reader.
Re: (Score:2)
Correlation is indeed a measure of a relationship between a cause and effect
No, it's not. Correlation does not mean causation. Assuming that correlation means causation is considered a logical fallacy.
Correlation is to causation what a rectangle is to a square.
If you take two groups, introduce factor X into one of them, and see a correlation between something and factor X, you've got a piece of data for causation. Do it a few more times, explain the mechanism if you can, and you've got a causal relationship.
While seeing correlation and assuming causation is indeed a misunderstanding of statistics, believing that there is no relationship between them is a more serious error.
Re: (Score:2)
this episode was 2 years ago (Score:2)
it's already been done (Score:3, Funny)
Not the greatest research (Score:2)
Here's the thing, not everything they do is crap, sometimes they get it more or less right, or at least right enough that one should pause. Most of they time they do a pretty mediocre job at it.
Don't confuse science with entertainment. It seems that the myth busters work from a layman's perspective and as such, fancy methodologies would confuse the audience.
Yawning *is* contagious (Score:4, Interesting)
-Ted
Re: (Score:2)
I'm very susceptible to "contagious yawning", When I watched this episode of Mythbusters, it set off a yawning fit that lasted for hours. Now, I refuse to watch it when it is re-broadcast.
Just reading a discussion about the episode has set me off. It's not as bad this time, but I had no urge to yawn 5 minutes ago, and now I'm yawning about twice a minute.
I expect that some people aren't affected by "contagious yawning". Maybe, even most people aren't affected. But,
Oy (Score:5, Insightful)
Look, to spare everyone the continued arguing over which statistical test to use at what probability level and the lack of proper control groups, let me say that MythBusters has never claimed to be a science show like Mr. Wizard. The guys are special effects designers for crying out loud! They are good at what they do, and while their scientific methodology and statistics may be a bit wonky at times, there are some experiments I've seen in peer-reviewed journals that aren't much better. Science education in the United States gets worse all the time, and if these guys can inject some life and curiosity into the current generation to get them interested in science, I applaud the effort.
An hour of tv can't prove a scientific fact? (Score:2)
The show is about entertainment and fun, not about the scientific method.
*Yawn* (Score:2)
Why the busting on the Mythbusters? (Score:5, Insightful)
Look, the show never said it was teaching people about science. Adam and Jaime themselves have said many times they're more entertainment than science. They're special effects people by trade, not scientists. They build things and blow shit up. It's what they enjoy doing. You can even see it on Jaime's face when they're doing myths that don't involve blowing things up (e.g. Adam building a wind tunnel for the penny drop myth).
When the show first started, there wasn't even mention of science. They looked at urban legends such as rocket car and getting airborne in a lawnchair. The show was about the stories themselves, not the methods. Only in about season 2 or so did they start including things like "controls" and "variables" (probably by Discovery's request), but they never lost sight of the fact that they're a TV show, and television (by and large) is meant to entertain.
But that leads to an interesting question: even if they DID follow proper scientific method, how do you even apply that to some of the myths they examine? For example, they did a myth where a hillbilly chased a raccoon into a sewer pipe, decided to throw gas down it, attempted to fill the thing with fire to kill the raccoon and was purportedly "shot out". How on earth do you test that scientifically? Nowhere in the myth does it says how big the pipe was, how much gasoline was used, etc. Nowhere does it mention if he was stuck (which is important, as they found the man could only be shot out of he was wrapped in a sabot). All they have is a fun story to go off of.
If nothing else, Mythbusters gets people interested in the process of examining life, not teaching how to use proper scientific method. If their only accomplishment is making people critically question things that are usually taken at face value, they'll have succeeded in my mind.
Statistics (Score:4, Funny)
Re: (Score:2)
Hey, me too! It *is* contagious!
And to think, I usually dismiss 'mythbusters' as garbage...
Re: (Score:2)
Re: (Score:2)
Re: (Score:2, Funny)
Where's the +1 Redundant moderation when you need it.
Peter
Re: (Score:2)
I can't wait to see the irony of that getting modded "Redundant"
Re: (Score:2)
That reminds me of the time my dad split an arrow, around the time the mythbusters aired saying that it was impossible.
Got a picture? Was your dad using manufactured arrows, or did he make them himself?
The "myth" was that it was simply a matter of skill -- it's not. It's a combination of having enough skill to hit your own shot, and the luck of having an arrow whose grain allows for a down-the-line split.
Don't believe me? Get an axe and a bunch of old arrows, and try and split them down the middle with a single cut.
Re: (Score:3, Funny)
*yawn*
No, it's just a coincidence.