Become a fan of Slashdot on Facebook


Forgot your password?

Gambling Could Reveal Which Scientific Studies Are Worth Their Salt ( 63

sciencehabit writes with this interesting story about using prediction markets to weed out false positives in scientific experiments. From the article: "Gambling is frowned upon in many circles. But what if the gamblers are researchers betting on how each other's experiments will turn out, and the results are used to improve science itself? A group of psychologists has found that their collective gambling—with real money—predicted the outcome of attempts at replicating experimental results better than their own expert guesses. They propose that this type of gambling setup, known as a prediction market, could become part of how science gets done."
This discussion has been archived. No new comments can be posted.

Gambling Could Reveal Which Scientific Studies Are Worth Their Salt

Comments Filter:
  • the interesting part (Score:5, Interesting)

    by gtall ( 79522 ) on Tuesday November 10, 2015 @09:17AM (#50899945)

    The interesting part seems to be that at least scientists use different weights between asking their opinion (such as a poll) and gambling on outcomes with real money.

    • Re: (Score:2, Interesting)

      by Anonymous Coward

      I wonder if... these psychologists can replicate experimental results of *this* experiment... or are their experiments as good as gambling?

    • Re: (Score:3, Interesting)

      Surprisingly and counter to intuition, it doesn't seem like artificial markets need to use real money to work []. Play money produces comparable results in many cases.

      It might be that all that is needed is some kind of limited resource so that people who make consistently bad predictions have their impact weighted down until they no longer affect the combined prediction, and that the wisdom of (thus suitably selected) crowds does the rest.
      • by Anonymous Coward on Tuesday November 10, 2015 @09:52AM (#50900139)

        Not really. Gambling is seldom limited by if the gamblers can afford it or not.
        The big difference is that usually when it comes to making predictions people have a situation where they one outcome is beneficial for them even if it was just a case of "It would be nice if this was the case."
        There simply is nothing to gain from having a correct prediction and nothing to lose from having a wrong prediction so people pick the outcome based on what they would like to happen.
        Bringing in money, real or fake, is sufficient for people to override this mental block and go with what they actually think, not what they want to think.

    • The interesting part seems to be that at least scientists use different weights between asking their opinion (such as a poll) and gambling on outcomes with real money.

      Nope. Spot the difference:

      " their collective gambling —with real money—predicted the outcome of attempts at replicating experimental results better than their own expert guesses. "

      • Proposals are gambling. When you write a white paper at a company it goes through layers of managerial review if it would be worth the companies time to have you complete a full proposal. Then the Government review panel looks at and decides if they want to invest in that. Peer review informs on the odds foliding in many predictors of success, such as past performance.

        Finally the reviews are parimutual gambling. Your odds of winning are not strictly the value of your proposal but the relative odds amongs

    • I don't find it surprising. You could ask anyone their opinion about something and they'll probably give you some amount of truth mixed with a little bit of what they want to believe is true. On the other hand, if asked to wager on something, there's potential loss involved and no matter how much you might like to lie to yourself, it cannot affect reality.

      Even the most devoutly religious would not actively wager money to put their faith to the test. The clever ones would probably quote some scripture abo
      • by moeinvt ( 851793 )

        Well said. There's definitely a subtle alteration of perspective going on when you put a monetary loss into the equation. I think you've nailed it by suggesting that wagering diminishes the "wishful thinking" influence on a belief.

      • > Even the most devoutly religious would not actively wager money to put their faith to the test.

        People do this kind of stuff all the time.

        Many people wager their lives on their belief in a God, and... usually die.

        George R. Price famously gave all his possessions to the poor; got evicted, fell into depression and then killed himself.

  • by Anonymous Coward

    WTF are all these MBA's doing in my lab?

    • by khallow ( 566160 )
      Do you really think having an MBA indicates someone is good at trading in a prediction market?
      • by Anonymous Coward

        No, but I do think MBA's will start to show up anywhere they smell money.

        They'd probably start offering derivative products to the scientists to mange their exposure.

        • MBA /= traders. And, there is nothing wrong with derivatives. The problem is forcing an industry to sell a product they don't want (sub-primes) and then being surprised that they sold off the paper as quickly as they could.
          • by moeinvt ( 851793 )

            "there is nothing wrong with derivatives."

            There are plenty of problems with the derivatives that were being cooked up in the early '2000's.

            A "Credit Default Swap(CDS)" was basically a fraud perpetrated by companies like AIG. It was a way of selling "insurance" without calling it "insurance" and thus evading the capital requirements for traditional insurance policies. You bought bonds from XYZ Inc and want "insurance"(a CDS) in case they don't pay you back. AIG is happy to oblige, even though AIG doesn't

    • The scientists become less attached to the MBAs than to lab rats, which should eliminate some experiment bias.
  • Las Vegas casino overs will have something to say about this. But better than mere gambling, let's put these guys in the ring. Make it into a Gladiator contest. Then you'll have a market.

  • by ArsenneLupin ( 766289 ) on Tuesday November 10, 2015 @09:55AM (#50900163)
    ... then why not also for science?
  • if it's actually science, then you can repeat the experiment. we solved this problem, it's called the scientific method.

  • by the_skywise ( 189793 ) on Tuesday November 10, 2015 @10:30AM (#50900413)

    It's just enforcing consensus - It doesn't mean the science will be right or wrong anymore than which horse wins the race.

    Yeah, the odds increase the probability of a success but that's not a guarantee.

    Geez, is this what's become of "science" in the world today?!

    • by ganv ( 881057 )

      Of course, empirical verification has to be the deciding factor. But right now there is very little way to communicate the insights that many scientists have about which studies are reliable. We could be much more efficient in doing future studies if there were a way to collect assessments of those who work in closely related fields without taking years to get to know people well enough to get them to tell you their actual opinions even though they haven't been able to focus on that question to publish a

    • To me, this seems like an important development. Keep in mind that there's huge biases in scientific publishing in favor of interesting studies (translation, in favor of studies with "surprising", aka unlikely, results; also against repeating and confirming experiments). Furthermore, once a journalist gets ahold of it, every piece of research is likely to produce a cure for cancer, fusion reactors, or revolutionize some aspect of computing. This based on the part where the scientist said, "hint, hint, need

    • It's called a "prediction market". It's been experimentally found to have better success in getting close to the truth than other techniques we have.

  • I have an idea - fantasy science leagues!

    Model it after fantasy football, you pick your team of scientists and win or lose based on how they do.

    Draft kings is on the job as we speak.

  • We already have institutionalized team based gambling to help decide which science to pursue. It's called "grant review committees."

    • Grant review committees usually don't have much at stake when making their decisions.

      • Really? Grant review committees I've been on handed out a couple million dollars worth of bets... I mean, grants.

        • Yes, they hand out millions of dollars worth of bets... with other people's money.

          And what happens to the grant committee members if the projects don't deliver anything useful? Basically nothing.

          Of course, scientific research projects rarely "fail" anyway because grant recipients have an interest in representing their work in the best possible light, and neither granting agencies nor grant committees have an incentive for challenging them.

          • Scientific research projects rarely fail because even negative results can be useful scientific information.

  • Gambling is a losers game. The idea of tying a losers game to science is a bit disturbing at best.

    I see they have been having difficulty getting participants.


    • A group of psychologists has found that their collective gambling—with real money—predicted the outcome of attempts at replicating experimental results better than their own expert guesses.

      That might work for something like psych, where few results are reproducible - or even where few "experiments" are re-tried. However, I doubt that the real world reflects the wishes of some profs with a penchant for a flutter and too much free time.

      Now if they could apply quantum mechanics to the problem, instead of gambling, there might be some interesting results.

    • by moeinvt ( 851793 )

      "Gambling is a losers game."

      In Vegas, yes, because the odds always favor the house. In the real world, the odds might be in your favor. You read a paper describing an experiment and the results. It's up to you to determine if it makes sense and if you think the results can be replicated. Asking yourself "Would you bet money on it?" just puts it in a different perspective. In our culture, bringing money into the equation puts a new twist on things. I think that asking that question is simply analyzing

  • God/nature does not play dice, but apparently scientists do or should?
  • by bfwebster ( 90513 ) on Tuesday November 10, 2015 @10:49AM (#50900641) Homepage

    Estimating confidence in the completion date for an IT project can be hard. When I've been asked to review a troubled IT project, I've found that a very useful technique to determine how far along the project actually is -- and when it's actually likely to be done -- is to ask those involved at different levels what their "$1,000 confidence level" is -- that is, what date they would be willing to bet $1,000 out of their own pocket that the project will actually be done by. I find that even the most optimistic engineer suddenly turns cautious and starts thinking of all the things that could go wrong.

    I then ask what their $10,000 confidence level is. Those answers tend to be the most accurate. ..bruce..

  • In the Palormares incident in 1966, a B-52 collided with a refueling tanker and both planes were lost. The plane was carrying 4 nuclear weapons, 3 of which were found on land (and two had their conventional explosives go off), but one fell into the Med. Dr. John P. Craven used a Bayesian Search theory to locate the lost bomb. The first step in the Bayesian Search is to formulate hypotheses for where the bomb might have gone down. Craven put a bottle of Scotch up as a prize for the person who came closes
  • "Would I bet money on it and with what odds?"

  • One is gambling ones reputation when making a public prediction. But the estimating a value of one public statement (correct or incorrect) is unenumerated. Money is a token for enumerating exchange. Currency doesn't have to exist in the form issued by a government entity. Stackexchange, for example, is pretty good at enumerating reputation gains and losses (and has methods for staking ones reputation gains on answers to certain questions). It's hardly surprising that enumerated exchange of knowledge is
  • I have a feeling they're doing better science because if they bet against one of their peers, they know if they have skin in the game that they're going to be a lot more critical and call bullshit easier. When it's a dolt/dumbass that's handing out public money, he doesn't care. Let's see, followed outline, has some good looking graphs, isn't disproving Man Made Global Warming - don't care, give 'em money.

Torque is cheap.