Slashdot stories can be listened to in audio form via an RSS feed, as read by our own robotic overlord.


Forgot your password?

Slashdot videos: Now with more Slashdot!

  • View

  • Discuss

  • Share

We've improved Slashdot's video section; now you can view our video interviews, product close-ups and site visits with all the usual Slashdot options to comment, share, etc. No more walled garden! It's a work in progress -- we hope you'll check it out (Learn more about the recent updates).

NASA Space

Why the "NASA Tested Space Drive" Is Bad Science 315

Posted by samzenpus
from the not-so-fast dept.
StartsWithABang writes Just over a century ago, N rays were detected by over a hundred researchers and discussed in some three hundred publications, yet there were serious experimental flaws and experimenter biases that were exposed over time. Fast forward to last week, and NASA Tests Microwave Space Drive is front page news. But a quick analysis shows that it isn't theorists who'll need to struggle to explain this phenomenon, but rather the shoddy experimentalists who are making the exact same "bad science" mistakes all over again.
This discussion has been archived. No new comments can be posted.

Why the "NASA Tested Space Drive" Is Bad Science

Comments Filter:
  • by Anonymous Coward on Thursday August 07, 2014 @07:51PM (#47627267)

    The result from NASA may or may not be real. However, this refutation is bad science writing and bad science.

    There are two glaring errors in it:

    • The claim that the "null" experiment should not have a result. The null experiment is badly named; it would create thrust according to one hypothesis as to how this thrust is supposed to be generated, and not create trust according to another hypothesis. There is was a separate reference that should not produce thrust under either hypothesis; and that did not create trust.
    • That it creates thrust without having energy escape. This has not been claimed anywhere else; the emDrive is using energy to create the thrust and the energy is obviously escaping. (It is also, as far as I can read it, using lots more energy than it creates thrust.)

    Apart from these two actual errors in description, the only "evidence" the author has is "This looks sort of similar to cases where science has gone wrong in the past".

    That *is* clearly a warning sign, but it is not actually sufficient to say "This is wrong".

  • by whoever57 (658626) on Thursday August 07, 2014 @07:52PM (#47627275) Journal

    Any 2nd year physics student should be able to laugh this garbage right off a lab bench without even running an experiment.

    Any good science student should be aware that our understanding of physics changes over time. Clearly this device is unlikely because it requires a change to the "laws" of physics.

    The article explains why any good scientist should be able to laugh this off based on the reported experimental results.

  • by fermion (181285) on Thursday August 07, 2014 @08:05PM (#47627329) Homepage Journal
    25 years ago there was desktop cold fusion. A lot of people wanted it, there were conferences on it, probably at least a hundred million was invested in it over a year or two. but it was bogus. The hypothesis was sound, it was no completely unreasonable, but the experiments showing a positive results on the hypothesis were flawed. It is not that cold fusion does not exist as something that might happen, it is that we have not shown it happens. I don't want to muddle the situation, but there is a clear line between what can happen and does happen in the lab. Theoretical people have told me that their models are necessarily not connected with reality. They are math, and the math sometimes tells us what is going on, sometimes fools us, and sometimes is just bonkers. What differentiates all this is good experimental science, which is really hard to do. I mean really hard, and for the most part does not lead to a theory, but only data that can be collected by math. This is why even though Galileo did a lot of good research, it was 100 years before the math caught up and we were able to do what we now classify as as science.It is why electromagnetic, the speed of light, quantum mechanics, and what is to follow is going to drop out the math. Which is to say we have a very complex interactions. Virtual particles drop out the math. The math says that they must exist, but inherently can't do anything useful. This is in the same way that photons can be coupled so they may seem to act faster than the speed of light(maybe, until we get distances longer than the earth-moon system we cannot really know) but no one expects information to be communicated faster than the speed of light. The end result is that if you have an experiment that violates the math, you have to be very sure it is a good experiment, and the consensus is quickly building that this is not. There is a certain responsibility to being an experimentalist. One can't just willy nilly say there are 40 dimensions of energy is created from the aether. On can be sloppy with conclusions, as Einstein was with the photoelectric effect, or Milikin in his oil drop experiment, but one does have a responsibility to do ones best to control systematic errors, and not jump to conclusions when one does not fully understand those errors. Unless, of course, like the two cited authors you are lucky enough to be accidentally correct.
  • by Smallpond (221300) on Thursday August 07, 2014 @08:08PM (#47627339) Homepage Journal

    Sorry. Science doesn't work by votes.

  • Review != accept (Score:5, Insightful)

    by pla (258480) on Thursday August 07, 2014 @08:20PM (#47627395) Journal
    Just over a century ago, N rays were detected by over a hundred researchers and discussed in some three hundred publications

    And just over two hundred years ago, the French Academy of Sciences steadfastly refused to believe that rocks could fall from space, with an abundance of supporting evidence to demonstrate that these "meteorites" had clearly come from weather conditions right here on Earth picking up rocks and flinging them about.

    Funny thing about (good) science - It doesn't simply dismiss new ideas simply because they disagree with existing theories. Oh, but for the first time in human history we have it right? Yeah, about that unified theory of quantum gravity, Doctor...
  • by pr0t0 (216378) on Thursday August 07, 2014 @08:21PM (#47627399)

    It's the reporting. This wasn't a peer reviewed scientific discovery, and it didn't claim to be. It was just a paper that laid out how the experiment was done, and what the results were, nothing more. Just because IFL Science, like every other tech/science site, picks up the story and hints at trips to Mars in a matter of weeks, doesn't mean that's what the experimenters were claiming.

    This is how science works. You do experiments, you post your methods and results. Other scientists may do the same. If there is enough evidence that something may be at work, you do more. If you end up showing that everything we thought we knew about the universe was wrong, THEN YOU START CHANGING THE TEXTBOOKS.

    The law of conservation of momentum, like all scientific laws, comes with the caveat that our understanding of how the universe works is correct. They are not immutable. Given reproduceability, predictability, and strong empirical evidence, it probably is correct; but that doesn't mean it may not need "tweaking" in the face of new evidence. It could also be that no scientific principles are being broken here, it's just there's something else at play we don't understand.

    People who claim otherwise are really just religious zealots in a lab coat.

  • by silfen (3720385) on Thursday August 07, 2014 @09:00PM (#47627567)

    How to fool the world with bad science

    What does that even mean? The Chinese reported some result, NASA tried to reproduce it, and didn't get very convincing results. Any halfway reasonable person looks at what was reported in the press and says "hey, nothing really to see here, they didn't really prove or disprove anything", to which one might add "how nice that people try some new and crazy stuff occasionally".

    Which part of that chain of events is supposed to constitute "bad science"? Who exactly is supposed to have been fooled? Which step along the way does Siegel consider "bad science" and why?

    Instead of making a rational argument for the cost/benefit of this particular experiment, Siegel goes off on some tangent about N-rays, supposedly illustrating the foolishness of some experiments. But there are many other cases where weird observations and experiments that most people thought never could work opened up entirely new areas in physics and biology. If one can learn anything from the history of science, it's that you should sometimes try crazy and foolish experiments because occasionally, they yield a big payoff.

    the impossible space engine that runs off of microwave power reflected inside a cavity

    Nobody knows whether reactionless drives are "impossible" or not; anybody who makes definitive statements one way or the other is a charlatan at this point, including Siegel.

    Sure, it violates the known laws of physics,

    The known laws of physics violate the known laws of physics, because they are not only incomplete but internally inconsistent. Somewhere along the line, you will have to do experiments whose results might violate the known laws of physics if you want to make progress.

    On the contrary, this is bad science because: The results are not robust, in that they are not identically-or-similarly reproducible by different teams.

    I still don't know what that "this" is that Siegel is referring to. How do you know that the results aren't reproducible or robust if you don't try to reproduce them?

    Siegel has the kind of dull mind that we don't want to teach our next generation of scientists or kids, and it is disturbing that guys like him are actually active in science education. Kids: try stupid things that violate known physics. Try things that sticks-in-the-mud like Siegel tell you don't work. And try to reproduce other people's experiments, both the ones that everybody believes and the ones nobody else could get to work.

  • by Anonymous Coward on Thursday August 07, 2014 @09:13PM (#47627613)

    > It does though.

    So the earth really was flat for awhile?

  • by penguinoid (724646) <> on Thursday August 07, 2014 @09:24PM (#47627667) Homepage Journal

    No, it is perfectly possible (and well understood) that you can produce thrust using pure energy with no mass. Just put a lightbulb and a reflector on your ship; as long as you can power the lightbulb you will produce thrust. The problem with this is that it is ridiculously inefficient, and since your power generation is not massless, this is roughly equivalent to using pathetically bad fuel.

    Also, don't confuse energy and momentum. They are separate things, and both are conserved independently of each other.

    The trick to making a good spaceship engine is converting energy efficiently into ship momentum. As far as we know this means creating high-momentum exhaust; conservation of momentum then means your ship gains momentum in the opposite direction. However, the problem is that to create high-momentum exhaust you either need high mass (and this means your ship carries, and has to accelerate, tones of fuel), or you create high-velocity exhaust (which due to the kinetic energy formula means you use a lot more energy).

    If you could find a way to skip the whole exhaust thing and transfer momentum directly into something not on your ship, you would have a space engine far superior to any we know of. The idea with this research was to transfer it into the quantum vacuum something-or-other. This would be analogous to how an airplane transfers momentum to the atmosphere or a boat to the water or a car to the land. In theory this could work and even be more efficient than using light as your exhaust.

  • Re:BullShit (Score:4, Insightful)

    by s.petry (762400) on Thursday August 07, 2014 @09:38PM (#47627697)
    I can't be a sociopath, I don't have a Twitter account.
  • by Anonymous Coward on Thursday August 07, 2014 @11:19PM (#47627991)

    they did pretty much all of the things you would like to see (such as reversing the direction and making sure the thrust reverses).

    * they seem to have done a thoughtful and careful job, including testing in vacuum.

    So, I still think they are likely wrong, but this ups the ante. In my opinion, you can't just say "this is obviously wrong."

    Sure I can. Was the... Was it... Does it...

    That your comment got modded 5:informative is hilarious. How about you RTFM and not phrase your comment in the form of questions? This was NASA. If NASA believed any of those alternate explanations you cited, do you think they'd be stupid enough to damage their reputations by presenting this absent those prominent criticisms? I admit, my respect for the U.S. govt is pretty low, like wow, really low. But still, you're theory of NASA incompetence I find more staggeringly unlikely than a device which superficially breaks the 'law' of conservation of momentum. And of course the stated theory suggests that it is not really a violation of that law, but an effect when dealing with subatomic particles that were not part of the cruder models used when originating the verbiage of such old 'laws'. I want to believe... :)

  • by Anonymous Coward on Thursday August 07, 2014 @11:30PM (#47628035)

    I trust my doctor because I can verify that he knows what he is talking about. Same with my mechanic. If you trust these people blindly because of their "authority", you're a fool.

  • Oh for fuck's sake... Time to debunk this shit, again.
    TFA got it wrong as well, so I suppose I can't blame you people for getting it wrong too, but please try doing a little more research?

    A little background: The EmDrive was invented by a guy named Shawyer. It was tested by NASA, among others, and found to produce about 91 microNewtons. (I'll address the 30-50 that TFA talks about too.) That's way less than the Chinese found, but NASA was also testing it at much lower power and say they are planning to test a higher-power version.

    The article mentions "... and a third person, Guido Fetta, have built three separate versions of the EmDrive". This is wrong, at least according to Fetta. Fetta invented what he calls a "Cannae Drive" ( which resembles an EmDrive but supposedly works on a different principle. In particular, Fetta believes that his drive requires radial slots in the chamber to operate. To test this, two versions of the Cannae Drive were (also, separately from the EmDrive test) tested by NASA: one with and one without the slots. Those tests both produced the same thrust (30-50 microN, about half what the EmDrive produced), which disproves Fetta's theory as to how the Cannae Drive is supposed to work.... and nothing else.

    The null test device that everybody is so dismissedly claiming claiming disproves the EmDrive wasn't even supposed to be an EmDrive! Fetta, inventor of the Cannae Drive, was disproven. Shawyer, inventor of the EmDrive, was actually vindicated because according to his theory, the Cannae Drive (slots or no) is basically an inefficiently-shaped EmDrive.

    I don't know why this is so hard for people to understand.

  • Thank you. The Fucking "Article" in the summary gets it so wrong I want to find the moron who wrote it and force him to actually read the paper that he gets almost completely wrong.

    Error 1) The Cannae Drive and the EmDrive are not the same thing, at least according to the inventor of the Cannae Drive. Every result that the article talks about for the EmDrive was actually from NASA testing the Cannae Drive.

    Error 2) The difference between the test and "null" devices was that one of them had slots on it (believed to be required for the Cannae Drive) and the other did not. According to Fetta (the inventor of the Cannae Drive, not just another person who built an EmDrive to test out), these slots are required. According to Shawyer (the guy who actually invented the EmDrive), they are not required. Looks like the EmDrive guy was right: they weren't required. This is addressed in Q2 of your fine link.

    Error 3) TFA never mentions this, but NASA Eagleworks *ALSO* tested Shawyer's version of the drive. It was over 3 times as efficient, producing about 91 microNewtons of thrust from 17 Watts of power (the Cannae Drive got 40uN from 27W). They didn't have a "null" device for that one, aside from a resistive dummy load... which produced no thrust when energized. Also, the tested drives produced no thrust when *not* energized.

    I really wish people would stop parroting the false claims in TFA.

  • by AthanasiusKircher (1333179) on Friday August 08, 2014 @07:39AM (#47629277)

    It isnt nearly right, its completely wrong and only human ignorance would say it was 'nearly right'

    No it was very much 100% wrong, nearly zero is not zero.

    I suppose you're also absolutely opposed to teaching Newtonian mechanics to intro physics students, too -- right? I mean, after all, we now know it "was very much 100% wrong." Sure, relativistic effects are basically irrelevant at normal speeds, but "nearly zero is not zero." So, we need to revamp our physics curriculum and introduce full-blown Einsteinian space-time to our students immediately on their first day... who cares if they won't have the math to do much with it, or if they'll never be able to do a lab experiment in the physics classroom with sufficient precision to display relativistic effects? We know Newton was "100% wrong," so let's stop teaching our students that ignorant nonsense!

    Of course, I'm being sarcastic here. Why do we still teach Newton when we know it's "wrong"? Because it's a good enough approximation for most purposes. Most of the measurements we take in everyday life will never require us to take relativistic effects into account, because most of our everyday measurement devices don't have sufficient precision to even show those effects at normal speeds. In effect, Newton's theories make sufficiently good predictions for everyday purposes, so we still teach them.

    That's the point of science -- to make good predictions. We can now teach Newton to do physics at normal speeds in everyday life, but we have caveats that say, "If you're going too fast or if you're in an intense gravitational field or... well, you need a better model." Science is not about debates concerning the ultimate nature of reality; it's a tool.

    And we do the exact same thing with the flat-earth model. We use maps all the time, which are generally projected onto a 2-D surface. All 2-D projections of the earth have flaws -- they either screw up distance or distort shapes or area in some way or whatever. And maybe that's an argument to use more globes and computer simulations of globes for students (since many map projections are very misleading)... but we still have little reason to worry about this when we're looking at a map of a small area unless we're firing long-distance projectiles or doing complicated surveying or something.

    Science is a tool, and we use a model that has sufficient accuracy for the task. At one time, we didn't have sufficient tools or applications to worry about the curvature of the earth, so for scientific purposes it was irrelevant. The models weren't "100% wrong" -- they were merely sufficient for the necessary accuracy. Same thing with Newton.

    Science is not concerned with questions about the ultimate nature of reality -- that's something for philosophy or religion or something. It's not really concerned with whether claims about that ultimate reality are "right" or "wrong" or whatever -- what matters is that we can have a sufficiently predictive mathematical model. For some purposes, the flat earth model was reasonably good... which -- as Azimov pointed out in the GP's link -- is why smart people made use of it for so long.

  • by dywolf (2673597) on Friday August 08, 2014 @08:14AM (#47629461)

    you just described what the other guy said about certifications, licenses and education.
    you're saying the same thing, while disagreeing with him.

  • by sycodon (149926) on Friday August 08, 2014 @08:55AM (#47629733)

    Science is a process.

      If you execute the process badly, that's "Bad Science".

    If you execute the process correctly, or at least as best you can given the known parameters, etc. then you can't call it "Bad Science", even though the results are unexpected and controversial. I would expect that even the most cynical observer would be curious as to what is happening here.

    Since his article contained several factual errors related to how the experiment was conducted, it is obvious that he did not even read the full report but just the POS Abstract or even worse, various news accounts.

    So actually, HE is the one engaging in Bad Science Reporting.

  • by WaffleMonster (969671) on Friday August 08, 2014 @11:50AM (#47631143)

    TFA itself has committed the sin of not reading the fucking article. They and their cohort of skeptics read only the summary from NASA without even bothering to get the full paper before drawing exceedingly obvious yet wrong conclusions.

    There is nothing wrong with dismissing something you assume is crap and don't want to waste your time with... as a practical matter there is only so much time we all have to make assumptions to operate. The problem arises when we forget or pretend we didn't make them.

    When you go that extra step of actively debunking you should no longer be able to hide behind your own ignorance and laziness. All those "skeptics" who think they know something simply because they elect to operate under the safety of default position need a good checking from time to time.

    Whatever ultimately happens at least NASA has the guts to go there and actually run experiments which is more than you'll ever get from the armchair skeptics.

  • by dywolf (2673597) on Friday August 08, 2014 @11:56PM (#47635887)

    then you misunderstand fallacies.
    nearly every defined fallacy has a condition or conditions easily satisfied in which the original statement is correct and not fallacious.

    "Correlation is not causation" is hte biggest one you see on slashdot. its a warning, a guide, but its not true 100% of the time. sometimes correlation IS causation. or really whats happening is its saying "hey, there might be something here, we need to look further...but careful, it could be coincidence"

    Appeal to Authority is another that can easily have situations in which its not automatically wrong. and its exactly as I said: the fallacy usually arrives from citing a non-authority as an authority. "I'm not a scientist, but Bobby over there tells me GW is a hoax." Now unless Bobby-over-there is a phd in climatology and current in the field, and not funded by oil companies....he may not qualify. But if Bobby IS that current, established, and well regarded phd, then his opinion on the science and evidence ABSOLUTELY does carry more weight than a non-experts, and its not fallacious in nature, merely potentially so until you know his background and reasoning. we appeal to authority because we are not experts. but if we cant cite experts and automatically wrong for doing so, then either you're doing the work of the ignorant for them, or we must all become experts at everything.

"Right now I feel that I've got my feet on the ground as far as my head is concerned." -- Baseball pitcher Bo Belinsky