Catch up on stories from the past week (and beyond) at the Slashdot story archive

 



Forgot your password?
typodupeerror
×
NASA Space

Why the "NASA Tested Space Drive" Is Bad Science 315

StartsWithABang writes Just over a century ago, N rays were detected by over a hundred researchers and discussed in some three hundred publications, yet there were serious experimental flaws and experimenter biases that were exposed over time. Fast forward to last week, and NASA Tests Microwave Space Drive is front page news. But a quick analysis shows that it isn't theorists who'll need to struggle to explain this phenomenon, but rather the shoddy experimentalists who are making the exact same "bad science" mistakes all over again.
This discussion has been archived. No new comments can be posted.

Why the "NASA Tested Space Drive" Is Bad Science

Comments Filter:
  • by Anonymous Coward on Thursday August 07, 2014 @08:39PM (#47627201)

    Have they accounted for the presence of skeptics during the experiments? That is likely the cause of any anomalies.

  • by Anonymous Coward on Thursday August 07, 2014 @08:46PM (#47627241)

    The NASA science is just fine: http://www.wired.co.uk/news/archive/2014-08/07/10-qs-about-nasa-impossible-drive

    • Thank you. The Fucking "Article" in the summary gets it so wrong I want to find the moron who wrote it and force him to actually read the paper that he gets almost completely wrong.

      Error 1) The Cannae Drive and the EmDrive are not the same thing, at least according to the inventor of the Cannae Drive. Every result that the article talks about for the EmDrive was actually from NASA testing the Cannae Drive.

      Error 2) The difference between the test and "null" devices was that one of them had slots on it (believed to be required for the Cannae Drive) and the other did not. According to Fetta (the inventor of the Cannae Drive, not just another person who built an EmDrive to test out), these slots are required. According to Shawyer (the guy who actually invented the EmDrive), they are not required. Looks like the EmDrive guy was right: they weren't required. This is addressed in Q2 of your fine link.

      Error 3) TFA never mentions this, but NASA Eagleworks *ALSO* tested Shawyer's version of the drive. It was over 3 times as efficient, producing about 91 microNewtons of thrust from 17 Watts of power (the Cannae Drive got 40uN from 27W). They didn't have a "null" device for that one, aside from a resistive dummy load... which produced no thrust when energized. Also, the tested drives produced no thrust when *not* energized.

      I really wish people would stop parroting the false claims in TFA.

  • by Anonymous Coward on Thursday August 07, 2014 @08:51PM (#47627267)

    The result from NASA may or may not be real. However, this refutation is bad science writing and bad science.

    There are two glaring errors in it:

    • The claim that the "null" experiment should not have a result. The null experiment is badly named; it would create thrust according to one hypothesis as to how this thrust is supposed to be generated, and not create trust according to another hypothesis. There is was a separate reference that should not produce thrust under either hypothesis; and that did not create trust.
    • That it creates thrust without having energy escape. This has not been claimed anywhere else; the emDrive is using energy to create the thrust and the energy is obviously escaping. (It is also, as far as I can read it, using lots more energy than it creates thrust.)

    Apart from these two actual errors in description, the only "evidence" the author has is "This looks sort of similar to cases where science has gone wrong in the past".

    That *is* clearly a warning sign, but it is not actually sufficient to say "This is wrong".

    • Bigger stupid one: the "null" device wasn't even supposed to be an EmDrive. It was supposed to be a Cannae Drive, which has a similar design but was invented by a completely different person and (supposedly) operates on different principles. The inventor of the Cannae Drive claimed that the difference between the null and actual test devices would mean there were different results. He was wrong, as shown experimentally.

      The actual inventor of the EmDrive (whose device was also tested by NASA, months ago, and was produced twice the thrust on 60% as much power) says that the Cannae Drive is just an inefficient EmDrive in either null or "real" configuration. http://en.wikipedia.org/wiki/E... [wikipedia.org]

  • by fermion ( 181285 ) on Thursday August 07, 2014 @09:05PM (#47627329) Homepage Journal
    25 years ago there was desktop cold fusion. A lot of people wanted it, there were conferences on it, probably at least a hundred million was invested in it over a year or two. but it was bogus. The hypothesis was sound, it was no completely unreasonable, but the experiments showing a positive results on the hypothesis were flawed. It is not that cold fusion does not exist as something that might happen, it is that we have not shown it happens. I don't want to muddle the situation, but there is a clear line between what can happen and does happen in the lab. Theoretical people have told me that their models are necessarily not connected with reality. They are math, and the math sometimes tells us what is going on, sometimes fools us, and sometimes is just bonkers. What differentiates all this is good experimental science, which is really hard to do. I mean really hard, and for the most part does not lead to a theory, but only data that can be collected by math. This is why even though Galileo did a lot of good research, it was 100 years before the math caught up and we were able to do what we now classify as as science.It is why electromagnetic, the speed of light, quantum mechanics, and what is to follow is going to drop out the math. Which is to say we have a very complex interactions. Virtual particles drop out the math. The math says that they must exist, but inherently can't do anything useful. This is in the same way that photons can be coupled so they may seem to act faster than the speed of light(maybe, until we get distances longer than the earth-moon system we cannot really know) but no one expects information to be communicated faster than the speed of light. The end result is that if you have an experiment that violates the math, you have to be very sure it is a good experiment, and the consensus is quickly building that this is not. There is a certain responsibility to being an experimentalist. One can't just willy nilly say there are 40 dimensions of energy is created from the aether. On can be sloppy with conclusions, as Einstein was with the photoelectric effect, or Milikin in his oil drop experiment, but one does have a responsibility to do ones best to control systematic errors, and not jump to conclusions when one does not fully understand those errors. Unless, of course, like the two cited authors you are lucky enough to be accidentally correct.
    • by 93 Escort Wagon ( 326346 ) on Thursday August 07, 2014 @09:23PM (#47627417)

      Not to mention that ColdFusion tended to exhibit a lot of remotely exploitable bugs.

    • by gl4ss ( 559668 )

      I dunno about this.

      the article makes claims different to the news reports I read about the subject, like about the test environments and that they did use a dummy version which did not show thrust but this tfa claims the dummy version produced thrust, which of course would be a red flag, but as far as I can tell according to nasa from the news the dummy version did not produce thrust and this was brought up as a point about how precarious the experiment had to be... in those news articles most of the articl

  • by mbone ( 558574 ) on Thursday August 07, 2014 @09:06PM (#47627331)

    I thought that this take was pretty appropriate when all we had to go on was the conference abstract. Now, however, the full paper [libertariannews.org] (still not peer reviewed) is out, and it is much better. I still think it is wrong, but I do not think it is bad science, and it will have to be refuted experimentally.

    Comments

    * the "null thruster" is something of a red herring from the abstract. Reading the paper, they have a true "null load," which shows no thrust, while the "null thruster" was a mod of a Cannae drive, and so more of a test of drive theory than the experimental setup, and, in any event, they tested several types of drives.

    * they did pretty much all of the things you would like to see (such as reversing the direction and making sure the thrust reverses).

    * they seem to have done a thoughtful and careful job, including testing in vacuum.

    So, I still think they are likely wrong, but this ups the ante. In my opinion, you can't just say "this is obviously wrong." I bet there will be a bunch of attempts to replicate it in labs all over the place.

    I find the theories here (and I have now read several in some depth) to be bad, either wrong, or handwavy, or both*. I would discount them entirely. In the unlikely event that this effect is real (and I mean, some non-standard physics effect), then the theory is likely to be something different than any of the proposals, The experiment's the thing, and the game now has to be disproving the Eagleworks results. Only once a bunch of people have failed to do that (or one person has done it) is there much else to say.

    * On pushing on virtual particles or quantum spacetime or whatever. These are 1 GHz photons, more or less. Such pushing would cause a _vacuum_ dispersion. Vacuum dispersion limits are set by timing of high energy photons from Gamma ray bursts across cosmic distances [arxiv.org]. These tests use ~ 100 MeV photons over ~10^10 light years, and so are many orders of magnitude tighter than the NASA Eagleworks results. This in my opinion rules out any photon - vacuum interaction as the cause of these anomalous thrusts.

    • In my opinion, you can't just say "this is obviously wrong."

      Yes you can: it's obviously wrong. Read the paper from the inventors [emdrive.com] on how the engine is supposed to work. It's a series of novice-level mistakes about physical principles and mechanisms. The entire idea is completely fucking batshit from the very beginning. The very fact that somebody actually got funding to build one of these absurd snake-oil devices indicates very little except that something is very, very wrong with the funding process. NASA is infamous for this kind of loony bullshit, and they really

      • by mbone ( 558574 ) on Thursday August 07, 2014 @09:59PM (#47627563)

        In my opinion, you can't just say "this is obviously wrong."

        Yes you can: it's obviously wrong. Read the paper from the inventors [emdrive.com] on how the engine is supposed to work. It's a series of novice-level mistakes about physical principles and mechanisms. The entire idea is completely fucking batshit from the very beginning. The very fact that somebody actually got funding to build one of these absurd snake-oil devices indicates very little except that something is very, very wrong with the funding process. NASA is infamous for this kind of loony bullshit, and they really need to stop. It makes them look like morons.

        I agree that the theory (or, at least, that theory) is obviously wrong. Cool, but from experimentalist standpoint, irrelevant. This paper, and the chinese paper, do not appear to the written by charlatans, they claim positive results, and so this will have to confirmed or denied by experiment. I have seen some very bad experimental NASA studies of new physics (*cough*warp drive*cough*), but this one doesn't appear to be so. If you see an obvious flaw in the full paper, please post it and I will publicize it.

        I would advise in general that you don't hyperventilate so much. This process will work out just as it should; I have no doubt that in a year there will be a dozen tests of this and we will likely know for sure one way or the other; in the meantime, I would take a $ 200 bet [xkcd.com] that the standard model will still prevail when this is over.

        • If you see an obvious flaw in the full paper, please post it and I will publicize it.

          I have a better idea: how about I ignore the whole sordid, over-hyped clusterfuck and go do something useful with my time? The Emfdrive company will go under like a submarine in short order, and we won't have to worry about them anyway.

        • by delt0r ( 999393 )
          If you want to test every claim by people that don't appear to be charlatans, you will spend entire lifetimes just showing bunk is bunk (try anitgravity, free energy BS on youtube). Extraordinary claims need extraordinary evidence.
      • I agree. The theory can be rubbished in very simple terms: The inventors assume no new physics, and conclude their device will violate conservation of momentum. All the 'input' physics conserves momentum. Therefore their analysis is wrong.

        If I give you a list of numbers which are all even and ask you to add them up, and you give me a sum which is odd, I know you've messed up. I don't need to check the details of your adding and point to exactly where you went wrong. This situation is analogous.

        So if the dev

      • by delt0r ( 999393 )
        There is another problem with the emDrive. Given the shape of microwave cavities used in communications and powers involved, we would have seen the effect long before now.
    • by RightwingNutjob ( 1302813 ) on Thursday August 07, 2014 @09:48PM (#47627521)

      * they did pretty much all of the things you would like to see (such as reversing the direction and making sure the thrust reverses).

      * they seem to have done a thoughtful and careful job, including testing in vacuum.

      So, I still think they are likely wrong, but this ups the ante. In my opinion, you can't just say "this is obviously wrong."

      Sure I can. Was the apparatus temperature controlled during the vacuum test? Was it tested in all orientations (not just backwards) to remove any gyroscopic weirdness from the rotation of the earth (think Michelson-Morley experiment). Was there EM coupling between the cavity, the torsion balance, and the chamber that could manifest as an anomalous torque, not thrust (that is, did they just make a big brushless motor)? Does the instrument register a thrust when the cavity is radiating but is bolted to the chamber floor and not the balance? Is there no thrust when it's oriented orthogonally? Does it still work if the power supply is electrically isolated from the vacuum chamber without a common return (ie did they build an electron gun)?

      • Re: (Score:2, Insightful)

        by Anonymous Coward

        they did pretty much all of the things you would like to see (such as reversing the direction and making sure the thrust reverses).

        * they seem to have done a thoughtful and careful job, including testing in vacuum.

        So, I still think they are likely wrong, but this ups the ante. In my opinion, you can't just say "this is obviously wrong."

        Sure I can. Was the... Was it... Does it...

        That your comment got modded 5:informative is hilarious. How about you RTFM and not phrase your comment in the form of questions?

    • "* they did pretty much all of the things you would like to see (such as reversing the direction and making sure the thrust reverses).

      * they seem to have done a thoughtful and careful job, including testing in vacuum."

      Read the article carefully.

      They did not actually test in vaccum. They tested at atmospheric pressure, because they did not have suitable vacuum rated amplifiers.
      Spending half a page explaining how the vacuum system worked, only to have a throwaway line later in the paper (search on electrolyti

  • by Loopy ( 41728 )

    It is important and necessary to independently verify and reproduce these results, meaning that if you detail the setup and methods, anyone else can achieve these results for themselves with the proper equipment

    Interesting. I wonder what other popular and controversial "science" is missing this particular step.

    • by pla ( 258480 )
      Interesting. I wonder what other popular and controversial "science" is missing this particular step.

      Well, in this case, NASA's version of this experiment lacks exactly that. They deliberately used a different design from that used (with reproduceable results) by both the British and the Chinese.

      Yes, Kudos to NASA for going to the trouble of setting up an actual "null" hypothesis test - Except, how about they try the experiment as documented before they go tweaking the conditions in unknown ways? Yes
      • Sadly, you're actually wrong even though you're right. Shawyer never said that the "null" device wouldn't produce thrust. That was the claim of a guy named Guido Fetta, who invented something he calls the Cannae Drive (http://en.wikipedia.org/wiki/EmDrive#Cannae_drive). Shawyer just said that the Cannae Drive is an inefficient EmDrive, with or without the slots which distinguished the "null" device from the "real" one.

        Oh, and when NASA tested the actual EmDrive (which was months ago), it actually produced m

  • Review != accept (Score:5, Insightful)

    by pla ( 258480 ) on Thursday August 07, 2014 @09:20PM (#47627395) Journal
    Just over a century ago, N rays were detected by over a hundred researchers and discussed in some three hundred publications

    And just over two hundred years ago, the French Academy of Sciences steadfastly refused to believe that rocks could fall from space, with an abundance of supporting evidence to demonstrate that these "meteorites" had clearly come from weather conditions right here on Earth picking up rocks and flinging them about.

    Funny thing about (good) science - It doesn't simply dismiss new ideas simply because they disagree with existing theories. Oh, but for the first time in human history we have it right? Yeah, about that unified theory of quantum gravity, Doctor...
  • by pr0t0 ( 216378 ) on Thursday August 07, 2014 @09:21PM (#47627399)

    It's the reporting. This wasn't a peer reviewed scientific discovery, and it didn't claim to be. It was just a paper that laid out how the experiment was done, and what the results were, nothing more. Just because IFL Science, like every other tech/science site, picks up the story and hints at trips to Mars in a matter of weeks, doesn't mean that's what the experimenters were claiming.

    This is how science works. You do experiments, you post your methods and results. Other scientists may do the same. If there is enough evidence that something may be at work, you do more. If you end up showing that everything we thought we knew about the universe was wrong, THEN YOU START CHANGING THE TEXTBOOKS.

    The law of conservation of momentum, like all scientific laws, comes with the caveat that our understanding of how the universe works is correct. They are not immutable. Given reproduceability, predictability, and strong empirical evidence, it probably is correct; but that doesn't mean it may not need "tweaking" in the face of new evidence. It could also be that no scientific principles are being broken here, it's just there's something else at play we don't understand.

    People who claim otherwise are really just religious zealots in a lab coat.

    • that Sir is so good!!!! I am so fed up of IFLS websites with their zealots approach... science and expecially physics is _experiment based_ if they send the bloody drive in space and it moves... then as you said we change the goddamn textbooks. I am so fed up of these new armchair crusaders.....if someone is smart or lucky enough to come up with a crazy but working experiment he doesn't necessarily have to be able to explain the whole theory behind it, that could be someone else job. Sorry zealots but real
    • Thank you. The hilarious thing is that this time, the zealots aren't even reading the report before "debunking" it. TFA (and, to be fair, lots of other sources) confused the recent NASA experiments on the Cannae Drive for experiments on the EmDrive. These are similar devices, but are invented by different people and their inventors claim different explanations for how they work. The actual inventor of the EmDrive (whose device was also tested, and produced more than twice as much thrust as the ~40 from the

  • by silfen ( 3720385 ) on Thursday August 07, 2014 @10:00PM (#47627567)

    How to fool the world with bad science

    What does that even mean? The Chinese reported some result, NASA tried to reproduce it, and didn't get very convincing results. Any halfway reasonable person looks at what was reported in the press and says "hey, nothing really to see here, they didn't really prove or disprove anything", to which one might add "how nice that people try some new and crazy stuff occasionally".

    Which part of that chain of events is supposed to constitute "bad science"? Who exactly is supposed to have been fooled? Which step along the way does Siegel consider "bad science" and why?

    Instead of making a rational argument for the cost/benefit of this particular experiment, Siegel goes off on some tangent about N-rays, supposedly illustrating the foolishness of some experiments. But there are many other cases where weird observations and experiments that most people thought never could work opened up entirely new areas in physics and biology. If one can learn anything from the history of science, it's that you should sometimes try crazy and foolish experiments because occasionally, they yield a big payoff.

    the impossible space engine that runs off of microwave power reflected inside a cavity

    Nobody knows whether reactionless drives are "impossible" or not; anybody who makes definitive statements one way or the other is a charlatan at this point, including Siegel.

    Sure, it violates the known laws of physics,

    The known laws of physics violate the known laws of physics, because they are not only incomplete but internally inconsistent. Somewhere along the line, you will have to do experiments whose results might violate the known laws of physics if you want to make progress.

    On the contrary, this is bad science because: The results are not robust, in that they are not identically-or-similarly reproducible by different teams.

    I still don't know what that "this" is that Siegel is referring to. How do you know that the results aren't reproducible or robust if you don't try to reproduce them?

    Siegel has the kind of dull mind that we don't want to teach our next generation of scientists or kids, and it is disturbing that guys like him are actually active in science education. Kids: try stupid things that violate known physics. Try things that sticks-in-the-mud like Siegel tell you don't work. And try to reproduce other people's experiments, both the ones that everybody believes and the ones nobody else could get to work.

  • Just put a payload experiment in orbit and see if you can drive it around. If the scientists running the experiments accounted for the motion of waves on a beach five miles away, I'm pretty certain that makes it worth a payload slot. We could dick around down here for years arguing about whether the results are valid or not, or we could put one up there and try it.

    Sounds like the perfect cubesat experiment.

    • The problem with that idea (much though I'd love to see it) is that the current experimental apparatus - the one that produce only-barely-detectable thrust - is already much too large for a cubesat, and that's ignoring the need to power it. The Chinese tested a much more powerful version, but they had to use kilowatts of power to get that much thrust; again, the whole thing is way too big for a cubesat. You're talking a moderate-sized (and priced, by satellite price standard) satellite here, and you'd need

  • by Sir Holo ( 531007 ) on Thursday August 07, 2014 @10:53PM (#47627757)
    Yes, N-rays were a false pursuit. (See book "Diamond Dealers and Feather Merchants")

    Cold fusion also. The palladium was soaking up hydrogen, which the original experimenters (Pons & Fleischmann?) misinterpreted as demonstrating room-temperature cold fusion.

    The public needs understand that un-refereed reports are not fact. Further, even refereed journal articles are not fact. It is only after others reproduce experiments and find confirming results that we get closer to "fact." Even then, it's just "confirmed theory."

    Why the popular press loves to breathlessly report on recent journal articles as "fact" only confuses the matter.
    • by Horshu ( 2754893 )
      "Why the popular press loves to breathlessly report on recent journal articles as "fact" only confuses the matter." I take it you don't believe that a 13 year old girl will revolutionize battery power with her science fair project?
  • To scientifically prove/disprove something you need experimental data, which is exactly what NASA seems to be doing in this case. The results are less than stellar but as long as they are not burning any significant amount of money whats the issue? If we really want to know if this thing works or not just put it in a small cubesat and piggyback it on another satellite launch, if it can change its orbit then we know it works, if it can't we know its useless. Again no significant amount of resources shoul

  • by Trogre ( 513942 ) on Friday August 08, 2014 @06:29AM (#47628799) Homepage

    I haven't seen any of the articles, but I'm guessing the news about it has been spread in this fashion:

    Rocket companies hate this. NASA has built a new type of engine using a simple trick. You will NEVER believe what happened when they switched it on. When I saw it, my mind was blown!
    (Photo with a couple of red circles around guys in lab coats)

  • by Minwee ( 522556 ) <dcr@neverwhen.org> on Friday August 08, 2014 @11:24AM (#47630313) Homepage

    The term that applies here is either "Bad Journalism" or "Bad Science Reporting". Calling it "Bad Science" and leaving it at that is giving the real charlatans a free pass.

    Scientist: "Hey, this is weird... We just put together something that shouldn't work but it sort of looks like it did."

    Headline: "NEW EXPERIMENT PROVES THAT EVERYTHING WE KNOW ABOUT PHYSICS IS WRONG!"

    Scientist: "It's not that... Look, here's a copy of a presentation we just gave to the rest of the department. There's a tiny and barely measurable bias in our results that we should be able to explain away but can't."

    Headline: "SCIENTIST WITH TWO ASSES HAS RESEARCHERS BAFFLED!"

    Scientist: "Um, that wasn't a press release, just a little paper we threw together to discuss our results. It's for other people familiar with what we're doing, and who know what words like 'bias' mean."

    Headline: "LEAKED INTERNAL DOCUMENTS REVEAL NEW LAWS OF PHYSICS!"

    Scientist: "I'm just going to back away slowly now and call some nice friends of mine who can show you out of the building. Try not to make any sudden moves..."

    Headline: "SCIENTISTS INVOLVED IN COVERUP OF REVOLUTIONARY NEW SPACE DRIVE!"

    Scientist: "Well, look at that. I just put a minus sign instead of a plus sign in one of the equations. If you do the math over again the results make a little bit more sense this way."

    Headline: "REVOLUTIONARY SPACE DRIVE SCIENTIST WITH TWO ASSES IS A FRAUD! HOW WERE WE ALL FOOLED?"

  • by WaffleMonster ( 969671 ) on Friday August 08, 2014 @12:50PM (#47631143)

    TFA itself has committed the sin of not reading the fucking article. They and their cohort of skeptics read only the summary from NASA without even bothering to get the full paper before drawing exceedingly obvious yet wrong conclusions.

    There is nothing wrong with dismissing something you assume is crap and don't want to waste your time with... as a practical matter there is only so much time we all have to make assumptions to operate. The problem arises when we forget or pretend we didn't make them.

    When you go that extra step of actively debunking you should no longer be able to hide behind your own ignorance and laziness. All those "skeptics" who think they know something simply because they elect to operate under the safety of default position need a good checking from time to time.

    Whatever ultimately happens at least NASA has the guts to go there and actually run experiments which is more than you'll ever get from the armchair skeptics.

It is easier to write an incorrect program than understand a correct one.

Working...