Follow Slashdot stories on Twitter

 


Forgot your password?
Close
typodupeerror
×
Science

Journal Devoted to the Null Hypothesis 31

Posted by michael from the office-decorated-with-demotivator-posters dept.
Xcott R13, 3(0,R4) writes "It may sound dull even for academia, but I personally am thrilled that someone is starting a journal devoted entirely to scientific research that fails to produce significant results. Researchers tend to publish successes, so we rarely ever read about experiments or approaches that didn't pan out, leaving future researchers to reinvent the square wheel. The "Journal of Articles in Support of the Null Hypothesis" intends to make some of this valuable boring information available. And such a wonderful title: too bad it's an online journal, else I could put it on the bookshelf next to the Annals of Improbable Research. Causing an explosion that would destroy the Universe."
This discussion has been archived. No new comments can be posted.

Journal Devoted to the Null Hypothesis More

Journal Devoted to the Null Hypothesis

Comments Filter:
  • I would be interested in hearing a response to this from someone in the field. Is there someone in the slashdot community who is distinguished in one of the natural sciences, who would care to comment? Thanks.
    • The null hypothesis was proven false just a few years ago. Why can't slashdot get a clue?! Sorry I only know about unnatural sciences.

    • Journal System considered harmful (Score:3, Interesting)

      by Anonymous Coward
      The whole `journal system' is flawed. It's goal is to *prevent* the acceptance of `ridiculous' ideas presented by `upstarts', i.e. anything the Big `Profs'---who had not a single original idea in the past two decades---will not like to see published, specially because it would (1) show that the `Profs' themselves are not doing much useful science, and (2) it would force the `Profs' to `upgrade' (or get dumped). NO NO NO! If the Profs don't like your ideas, you are NOT to be published, ever. Period. No publications means no funds, no appointments, you get the picture.
      The ScientificCommunity(TM)---which is neither---wants no `upstarts' making ripples in the water. They impede progress. As the saying goes: academic politics are the worst kind, because the stakes are so low.

      In the world of ModernScience(TM) you either conform or are crushed. And the best way for the `Profs' to have this power is the closed journal system. Why do you think scientists are the very LAST ones to use the Web, the very LEAST inclined to make the proccess transparent for the public to see? When was the last time you knew which papers were rejected and why? Never?---I thought so? Ever wonder *why*?

      That's why they insist in what they call `peer review', which again is neither. The `reviewers' are NOT your peers, but the editor's choice, IF it is the all-powerful editor's whim to even grant that to the author. There is no accountability.

      REAL science is pushed to be published in what (yet again) the Profs call ``grey literature'' (meaning what the Profs do not endorse) or forever keep your peace.

      Sounds too harsh. Think again, check the history of science. All good ideas are rejected for 30 years, their authors ridiculed and their careers (an sometimes their very lives) wrecked, and *then* some of the Profs *discovers* your idea and mekes it his own, after you have been crushed. Think Wegener and plate tectonics. Think Semmelweiss and aseptical surgical practice. Think Plank and quantum physics. Think Mendel and genetics. (And try to find out about the ones who are being crushed this very moment, and of their much-needed ideas).

      Don't like it. Well, do *something* about it.
      • This is interesting (plz note, moderators). While I am very skeptical of academia as a whole, my political views very strongly support the ideas of reputation and an "aristocracy of ideas" in any field.

        As important as it is to allow truly great breakthroughs to surface, the "barriers to entry" play a key role in preventing crackpot ideas from wasting resources.

        A lot of the theories and experiments that don't survive peer review really are crazy or fraudulant. Witness the guy in the Village Voice who claimed to have discovered a new particle, or the recent story on slashdot about the Magic Box.

        As for the examples you give of valid theories that were squelched, well, they all did eventually become accepted, evidence that the system, at worse, seems to slow things down. Good ideas will stand on their own merit.

      • Re:Journal System considered harmful (Score:4, Interesting)

        by krlynch ( 158571 ) on Monday May 06, 2002 @03:41PM (#3471983) Homepage

        May I ask what horrible experience you have had to so warp your view of reality? I don't disagree that there have been SOME ideas/theories/experiments that have been improperly rejected from publication, but you would be hard pressed, I think, to find an example of truly revolutionary and fundamentally correct work being rejected by the "scientific establishment" (clue for the clueless: there is no such thing as an organized, coherent scientific establishment...). In fact, your whole argument makes little to no sense, since it is REVOLUTIONARY, FUNDAMENTAL work that gets people recognition and promotion, NOT "sticking to the party line". This DOESN'T mean that all theories that disagree with the established theory will be accepted as "plausible" .... most theories by people not trained in the field are notably ridiculous, and worthy of begin ignored, simply because they don't indicate any understanding of the material they claim to be discussing.

        My experience is in theoretical particle physics, and in just the last six years, every talk I've been to starts out: "The current theory is wonderful, but deficient in the following ways ... I'm going to tell you about work I've done to throw out this part of the standard theory." Standard Theories become standard, not because we want them to, but because they match the data better than any other theory that has come along, are internally consistent, are predictive, and are falsifiable.

        As for your post, there are so many incorrect points that I don't have the time to correct them all, but I'll pick a sample:

        1. Why do you think scientists are the very LAST ones to use the Web?: Huh? Scientists INVENTED the network, invented the protocols, and invented teh Web itself, for the dissemination of scientific data. The need for higher speed backbones (Internet2 and its kin) is being DRIVEN by the ever increasing needs of the scientific community for bandwidth. Scientific researchers are on the forefront of the open source and free software movements, and certainly not "the last ones to use the web".
        2. `peer review', which again is neither: articles are reviewed by members of the scientific field that the journal caters to; that is, multiple practicing scientists read and comment on all aspects of the research and reporting in the submitted articles, and authors are given the chance to dispute or address the conclusions of the reviewer. Seems pretty much like "peers" "reviewing" the work to me.
        3. When was the last time you knew which papers were rejected and why?: Whenever the authors whose papers are rejected make available the paper and the referees' comments. It is generally considered poor manners to make public that you have rejected someones work...but if they want to do it themselves, that's up to them. Personally, I've talked to a dozen people in the last year alone, and been to three talks where people started out saying "This work was rejected by Journal X; I'll tell you why I think they were wrong."
        4. All good ideas are rejected for 30 years: this is truly an silly statement, with its all encompassing conclusion. From my own field, I can think of any number of advances that were accepted exceedingly rapidly, or were deemed interesting enough to be worth extended study and discussion: quantum mechanics, special and general relativity, the experimental confirmation of the frame independence of hte speed of light, the necessity of antiparticles, the prediction of the neutrino, the path integral formulation of quantum theory, quantum electrodynamics, the quark model, the discovery of the muon, the discovery of parity violation, quantum chromodynamics, etc. etc. etc. All of these papers demolished the status quo, and yet were accepted very early on by the journals they were submitted to, and the community who read them.


        So, to summarize, I don't think you have a clue as to what you are talking about, and suspect that you have some personal agenda or vendetta that you feel needs to be made public. I wish you had let us know WHY you feel "rejected" by the scientific community. (remember that "made public" complaint you had?) I certainly don't claim that the process is perfect, but it is the best, most reliable one that we have found so far. Perhaps you could let us know what alternatives you had in mind?

        • Perhaps it is the state of your field... (Score:2, Insightful)

          by Anonymous Coward
          Alas, I am in psychology, and while I resent the notion that it is not a "hard" science (we have to wrestle with computationally intensive [read: supercomputers] integration of complex multivariate statistical distributions, genomics, imaging physics, etc.), I will propose that by its very nature psychology is more susceptible to emotional or political bias than other fields, such as theoretical physics.

          I cannot tell you how many times I have seen articles rejected from journals because of trite reasons that have nothing to do with anything other than the reviewer's ego. Granted, many of those articles were later accepted in fine journals, but some weren't, and the sheer number of articles rejected for unjustifiable reasons is ridiculous.

          The problem is that in a field that can potentially be as fuzzy as psychology, someone can raise an objection that is theoretically problematic, but pragmatically in all likelihood makes no difference. There is often no way of quantifying the magnitude of effect of a possible nuisance variable--or at least, no one does--so in effect, every possible problem can be and are treated as a real problem. This allows someone with a personal agenda to easily prevent papers from being published just by raising a possibility, of which there are an infinite number. You are left with relying on the good will and friendliness of the reviewer.

          Eysenck, one of the most prominent psychologists of the last century, for example, founded his own journal out of frustration regarding this fact. Think about the issues dealt with in psychology: free will, nature vs. nurture, perceptual quality, decision making, intellectual abilities, etc. My colleague and I were just discussing yesterday how frequently incredibly rational, intelligent individuals become incoherent and insensible when discussing psychology (how many posts on Slashdot irrationally start quoting science fiction authors when discussing psych?)

          There is empirical evidence to suggest the peer-review system is in trouble as well. I recently read of a meta-analysis presented at a conference (the National Academy of Sciences?) suggesting that peer-review did not improve the quality of articles eventually accepted.

          About your question of how many major ideas never made it into journals: if they never made it into the literature, we would never know, would we? I'm sure we could all identify cases of famous theories being forgotten in obscure journals or manuscripts, only to be rediscovered later. How many times have we learned that so-and-so was not the original discoverer of X, because it was relegated to obscurity because of the review process?

          Kuhn, a prominent philosopher of science, suggested that a prime determinant of the acceptance of a scientific theory is what he called "The Big Mouth Factor". Guess what he was talking about.
      • That's a brilliant post.

        Some additional examples of the tyranny of the orthodox:

        Jenner with the Smallpox vaccine.

        TG Morton and anasthesia

        Pasteur (he's not a doctor, we don't have to listen to him)

        de Broglie (was about to be denied a PhD when Einstein commented how insightful his work was)
      • They laughed at Einstein. They laughed at the Wright Brothers. But they
        also laughed at Bozo the Clown.
        -- Carl Sagan

        Yes, most of the radical ideas in science have been ridiculed at one point in time. And scientists have regrettably supressed legitimate research because of their personal egos. A good (and unfortunate) relatively recent example is Brian Jospephson. It is unfortunate that this happens, and can really be hell on the people involved. Scientists are humans too, and sometime that gets in the way -- just like in every other field of human endevour.

        But the thing in science is that if your theory is right, eventually people gather evidence for it, and scientists *do* accept it. And for every one misunderstood genius, there are hundreds of genuine crackpots who really get ridiculed more than they deserve, but still shouldn't be published alongside good research. Which is the whole purpose of peer review: to collect the small fraction of material that is interesting to the readers of the journal.

        If you want to see what happens without peer review, go visit the lanl archives [lanl.gov], which are a very valuable resource, but only if you realize that a large fraction of what is there is, while often written by respectable "Profs", wrong.

        Propose another system that works better, and, if it does, it will evetually be adopted.
      • Thanks a lot, you just broke my kook-o-meter.
    • This is actually very interesting. There is a place for this, and it has the potential to be a great resource. There are a couple of problems, though:
      • When you list your publications, an article in this journal will not impress people. The stigma of the name may be enough to repel researchers from having it including in their curriculum vitae. (I'm sad to say...)

      • The editorial board looked really weak. A journal of this nature would have to have a stellar editorial board for it to achieve the stated goals. That isn't the case here.
    • > Is there someone in the slashdot community who is distinguished in one of the natural sciences

      Try here [extremefineart.com] - very natural to say the least...

  • "I could put it on the bookshelf next to the Annals of Improbable Research. Causing an explosion that would destroy the Universe."

    No, I tried this and the universe wasn't destroyed. See last month's issue for details.

  • Will this be the "ugly fat girl" of the world of science?
    "Sorry to hear it didn't pan out, there's always that null hypothesis journal"

  • Good idea...in theory... (Score:3, Interesting)

    by kalyptein ( 313110 ) on Monday May 06, 2002 @11:05AM (#3469661)
    I am a grad student in molecular biology, and I've learned the hard way how nice this kind of journal would have been. I wrote a research proposal about something that, when it showed up in front of my committee, whas dismissed out of hand. Turns out that line of research ran into a dead end five years ago. People don't go back and publish "oh, btw, we were wrong". I, and apparently my advisor who gave me the topic, were too dumb to pick up that this was a discontinued line of research.

    On the other hand, it would be a terminally dull read. And people would probably be afraid to publish in it, thinking it would hurt their career to be openly associated with failure.
    • I, and apparently my advisor who gave me the topic, were too dumb to pick up that this was a discontinued line of research.

      Maybe you should consider a new thesis advisor (seriously). As a student, you would not be expected to know about this sort of thing, but your advisor seems way out of touch. This has a bigger impact on your future than you might think.

    • Re:Good idea...in theory... (Score:5, Interesting)

      by 4of12 ( 97621 ) on Monday May 06, 2002 @02:30PM (#3471347) Homepage Journal

      Turns out that line of research ran into a dead end five years ago. People don't go back and publish "oh, btw, we were wrong".

      For a long time, too, I've thought that careful documentation of what does not work and why is just as important as what does work and why.

      Publication of "How Great Idea Blah Doesn't Really Work" is vitally important to preventing other creative, thinking-outside-the-box people from chasing down the same dead end in the maze of symptoms of the truth.

      However, if you expend a little effort you can find out many of these things in two ways:

      1. At the leading edge, talk to people at conferences. Let them know what you're thinking and see what they say. Talk to several people, too. Of course you need to take debunking of your idea on a professional and not personal level. Also you have to be reasonably trusting that not everyone is out to scoop your great ideas.
      2. Look carefully at the written articles in your field of interest. Many times in the introductions and conclusions there are little sentences that hint at why some things are not practical; often these sentences help to round out arguments that would other suggest that some harebrained scheme would cure the world's problems.
    • Wow, reminds me of this apropos joke [uci.edu]...

  • Sometimes.... (Score:3, Informative)

    by dalassa ( 204012 ) on Monday May 06, 2002 @11:13AM (#3469726) Journal
    Not getting an answer is as important as getting one.

    "Well this expirement/proof did nothing, so lets try something else."
    And scientific progress moves on. Just like all the failed proofs of Fermat's Last Therom.
    • Exactly. For example, a search for the Higgs Boson that doesn't find it would qualify for this journal, and might force us to rewrite large portions of current physics theory. (Basically, we'd have to redo just about everything fundamental that involves mass.)

  • Not a new idea (Score:5, Informative)

    by Spurion ( 412996 ) on Monday May 06, 2002 @11:43AM (#3469963)
    Richard Feynman was a very strong advocate of having somewhere to publish non-positive results. He was distressed by the credibility added to the existence of psychic powers, influence of star signs, alien abductions and so on, by scientific research. It arises from the fact that if you test your hypothesis to, say, the 5% level, then on average one in twenty researchers is going to get a false positive. People tend to publish the positive results, and the nineteen negative results get forgotten because they're boring, giving a misleading picture. Of course, all the research could be completely accurate, but the skewed statistics make that irrelevant.

    So Feynman dearly wanted a Journal of the Null Hypothesis. I think I found that in "Surely You're Joking, Mr. Feynman".
  • Aside from the efficiency gain of not trying something that won't work there is a much more important reason for a journal of this nature. Consider that we are all to happy to accept that what we expected to happen did but it can be more informative to try to figure out why what we expected to happen didn't. This latter is much more important in the long run and can be seen as the very essence of science. Suppose you thought the earth was flat and ran an experiment to "prove" that. Or you thought the parts of an atom were put together like a raisin pudding. Our greatest advancements and development of new fields have occurred when the conventional wisdoms were proven incorrect.

    So such a journal needs to have two aspects. One a list of things that won't work and why the person that tried them thought they would to reduce reinventing square wheels and the other is a critical examination/explanation/"proof" of why it wouldn't work. Those experiments that don't have the latter stand as possible areas for great gain to humanity.

  • This is Psych, folks, not 'Hard' Science (Score:5, Insightful)

    by sailordave ( 199879 ) on Monday May 06, 2002 @01:18PM (#3470803)
    Read the Journal's website, folks -- it's a psych journal, not a mol-bio/physics/hard-science journal. Most of the posts here are getting this wrong.

    This is in fact the reason why this journal is such a great idea. As a social science, the field of psychology has a much greater problem than fields like physics with dubious positive experiments getting overhyped -- the media will hype the one study that says the Internet turns kids into axe murders, but it doesn't mention the 99 other studies that found no relation.

    Feynmann, in fact, wrote an article called 'Cargo-Cult Science' [brocku.ca], in which he attacked the discipline of psychology for not repeating experiments to check old results. Yes, he would 100% approve of this new journal.
  • ...consist of someone finding a correlation between some variables. They'll publish the results if they get something 95% or 99% significant. This means that if you do 20 experiments, even if there's no underlying phenomenon you still expect to get one publishable paper by chance. And of course we only see the published papers so we get a very skewed view. By publishing failed experiments we get to see the other 95%. (Of course no experiment truly fails...they all give some information.)
  • I have to read normal journal articles all the time, and let me tell you, sometimes its the most tedious and boring part of being a scientist. Wading through reams of data and trying to figure out how their results are supposed to support their hypotheses is bad enough. Wading through reams of data and trying figure out what conclusions to draw from unexpected or inconclusive data... Holy cow, this will be a boring journal.

  • One of the most important null resultss in physics was the Michelson Inferometer experiment. It consisted of two mirrors and a beam splitter and displayed interference fringe when the 2 beams crossed again. The idea was that by situating it in different directions one could measure the speed of the earth through the ether by the diffenence in the interference patterns. The results came back negligable and eventually they realized that the reason that the results were null was because there was no ether (I'm not sure if the discovery was a result of the experiment though).

Slashdot Top Deals

HELP!!!! I'm being held prisoner in /usr/games/lib!

Close