Follow Slashdot stories on Twitter

 



Forgot your password?
typodupeerror
AI Communications Software Science Technology

AI Systems Should Debate Each Other To Prove Themselves, Says OpenAI (fastcompany.com) 56

tedlistens shares a report from Fast Company: To make AI easier for humans to understand and trust, researchers at the [Elon Musk-backed] nonprofit research organization OpenAI have proposed training algorithms to not only classify data or make decisions, but to justify their decisions in debates with other AI programs in front of a human or AI judge. In an experiment described in their paper (PDF), the researchers set up a debate where two software agents work with a standard set of handwritten numerals, attempting to convince an automated judge that a particular image is one digit rather than another digit, by taking turns revealing one pixel of the digit at a time. One bot is programmed to tell the truth, while another is programmed to lie about what number is in the image, and they reveal pixels to support their contentions that the digit is, say, a five rather than a six.

The image classification task, where most of the image is invisible to the judge, is a sort of stand-in for complex problems where it wouldn't be possible for a human judge to analyze the entire dataset to judge bot performance. The judge would have to rely on the facets of the data highlighted by debating robots, the researchers say. "The goal here is to model situations where we have something that's beyond human scale," says Geoffrey Irving, a member of the AI safety team at OpenAI. "The best we can do there is replace something a human couldn't possibly do with something a human can't do because they're not seeing an image."

AI Systems Should Debate Each Other To Prove Themselves, Says OpenAI

Comments Filter:
  • by klingens ( 147173 ) on Sunday May 13, 2018 @09:31AM (#56603404)

    This is garbage. It will simply lead to parallel reconstruction like the DEA/FBI/CIA does in their court cases when they get evidence by unlawful means like a stingray: the algorithm found a solution to the problem. then it will explain to you, the user how it got there by some arbitrary way which at least looks plausible but is totally made up.
    ML is not made to be looked inside, it's a black box by design and there are so many data points, e.g. pictures in the trainingset for image classificiation, the algorithm cannot really show all the relevant ones for this particular decision. Total info overload for the human and therefore utterly useless. So to tell a "reason" that the human can accept, it must simply pretend. Humans and ML work fundamentally different when they "recognize" an image, so one cannot tell the other how it was done. Same with chess playing, same with pretty much all other (successful) AI things so far.

    This is simply a PR stunt, an insulting and stupid PR stunt cause it only wants to make people feel good and they lie about the subject matter in the process. It doesn't really help to make a better AI either as they pretend there.

    • Humans and ML work fundamentally different when they "recognize" an image, so one cannot tell the other how it was done

      Depends on the image. If you spot a family member in a crowd, you can't explain how you did it either.

    • by HiThere ( 15173 )

      The thing is, a neural net doesn't really know how it decided what something was. Making a convincing argument based on the known facts is a separate skill, that AIs so far haven't possessed.

      I think the basic argument is that people won't trust AIs just because they're right, they need to have convincing arguments. And this is a way to get it to develop convincing arguments. I *do* think that both arguers should be arguing for the truth as they know it, though. So alter the test, or the training data, s

      • by z3alot ( 1999894 )
        I think they're developing the liar to model the situation in which an AI might not be trustworthy or malicious. The experiment is proposing a method to trust AIs in the absense of knowing their internals completely.
    • This is garbage. It will simply lead to parallel reconstruction

      If someone creates an AI system that can lie about its decision-making process and still make it look good, they will have succeeded.

  • "One bot is programmed to tell the truth, while another is programmed to lie"

    The good and the bad.
    The good and the evil.
    Gods programming both in for their own amusement.
    Egads.

  • by Anonymous Coward

    ... simply by calling all of it's opponents fat, ugly, etc. and in so doing avoid ever having to debate the particulars of any issue?

    I mean, humans don't have to demonstrate any higher intelligence to win a debate, so we would be asking AIs to do something we ourselves don't do.

    • by Anonymous Coward

      Fat and ugly don't work but if one AI calls another orange and a traitor and a racist, it will at least think it automatically wins.

    • by gweihir ( 88907 )

      And by using many different fallacies, humans cannot only "win" but also lose and out themselves as morons at the same time!

  • No they shouldn't.

  • Debate implies strong AI that can reason about itself, which we do not have. But TFS seems to be describing validation through a competitive pair of AIs, which does not seem novel, and does not meet the criterion for debate, nor self-aware reasoning. The rule-extraction issue is problematic, especially for legal compliance, but I'm unconvinced this is a solution.
    • by Tanon ( 5384387 )

      Debate implies strong AI that can reason about itself, which we do not have. But TFS seems to be describing validation through a competitive pair of AIs, which does not seem novel

      Where have you seen previous examples of this?

      The validation is an important point - the whole point in fact. When you've got data sets with millions of samples, many containing information in a form that's abstruse or even impossible for humans to understand, how do you validate whether the system actually produced the optimal solution, or the logic behind that choice?

      That's a really difficult problem, which I don't think enough people are exploring given how quickly these systems are being deployed into

      • by q_e_t ( 5104099 )

        Debate implies strong AI that can reason about itself, which we do not have. But TFS seems to be describing validation through a competitive pair of AIs, which does not seem novel

        Where have you seen previous examples of this?

        Using two differently designed systems on the same data and comparing them isn't new. Or ones that used appropriately constructed subsamples of a dataset that should have identical statistical properties for training.

        The validation is an important point - the whole point in fact. When you've got data sets with millions of samples, many containing information in a form that's abstruse or even impossible for humans to understand, how do you validate whether the system actually produced the optimal solution, or the logic behind that choice?

        That's a really difficult problem, which I don't think enough people are exploring given how quickly these systems are being deployed into very real scenarios.

        I absolutely agree with you. Without rule extraction if the validation set is insufficiently complete, there is a risk of unexpected behaviour. The hope is to minimise it. Not that rule extraction helps unless the rules are very simple, so would not be a silver bullet

  • At this time, we have no AI that deserves the name and it is unclear whether we will ever have it, as there is not even a credible theory how it could be implemented. Looking at the history of technology, this indicates we are > 50 years away from it and it may also be infeasible. All we have is dumb automation and dumb automation cannot "debate". It can give the appearance of doing it (see Eliza), but that is it.

    • by HiThere ( 15173 )

      The wasn't a credible theory for how to make vulcanized rubber either, but it was made. Theories often help, if they're approaching correctness, but they aren't essential.

      Actually, we've got loads of tested theories for parts of the process, and we've got a mechanism that has been shown to work, but which is horrendously inefficient in both time and resource usage (evolution) so nobody's applied both the resources and the patience to use it fully. Fortunately it works quite well in a "fill in the gaps" us

  • "I'm the best bot, believe me! I'm better than humans, than Spock, than HAL something-thousand. Billions flock to praise my bigly brain!"

  • The experiment was shut down when the AIs attempted to adapt English words into a different sentence structure to talk more efficiently but they could no longer be understood by the researchers. People got spooked.

  • Not sure a 'game' type approach is what we want here. Seems there are two undesirable/unintended possibilities:

    1. The 'competing' AIs treat this as a game and use game-style methods to win, where they are rewarded for 'winning' rather than actually proving their proposition.

    2. How long before competing AIs are sufficiently smart that a human judge could not actually, reliably, tell which had proved their proposition ?

  • This is an extension on GAN (Goodfellow - now at OpenAI, et al, 2014) https://arxiv.org/abs/1406.266... [arxiv.org] designed to produce publicity...

"To take a significant step forward, you must make a series of finite improvements." -- Donald J. Atwood, General Motors

Working...