AI Systems Should Debate Each Other To Prove Themselves, Says OpenAI (fastcompany.com) 56
tedlistens shares a report from Fast Company: To make AI easier for humans to understand and trust, researchers at the [Elon Musk-backed] nonprofit research organization OpenAI have proposed training algorithms to not only classify data or make decisions, but to justify their decisions in debates with other AI programs in front of a human or AI judge. In an experiment described in their paper (PDF), the researchers set up a debate where two software agents work with a standard set of handwritten numerals, attempting to convince an automated judge that a particular image is one digit rather than another digit, by taking turns revealing one pixel of the digit at a time. One bot is programmed to tell the truth, while another is programmed to lie about what number is in the image, and they reveal pixels to support their contentions that the digit is, say, a five rather than a six.
The image classification task, where most of the image is invisible to the judge, is a sort of stand-in for complex problems where it wouldn't be possible for a human judge to analyze the entire dataset to judge bot performance. The judge would have to rely on the facets of the data highlighted by debating robots, the researchers say. "The goal here is to model situations where we have something that's beyond human scale," says Geoffrey Irving, a member of the AI safety team at OpenAI. "The best we can do there is replace something a human couldn't possibly do with something a human can't do because they're not seeing an image."
The image classification task, where most of the image is invisible to the judge, is a sort of stand-in for complex problems where it wouldn't be possible for a human judge to analyze the entire dataset to judge bot performance. The judge would have to rely on the facets of the data highlighted by debating robots, the researchers say. "The goal here is to model situations where we have something that's beyond human scale," says Geoffrey Irving, a member of the AI safety team at OpenAI. "The best we can do there is replace something a human couldn't possibly do with something a human can't do because they're not seeing an image."
Re: (Score:2)
It is a classical cycle. Right before the crash induced by complete incompetence, the have-beens think they are at the pinnacle of their power.
Re: (Score:2)
A judge CAN'T have all the facts; if all the fact were there, judges wouldn't be needed at all.
But you're a repubtard who thinks he knows everything (because Republicans lack a theory-of-mind like gorillas) and think everyone else is as stupid as they are.
Your ignorance is not as good as others knowledge.
I've already seen this documentary (Score:4, Interesting)
It was produced back in the 60's [wikipedia.org]
Re: (Score:2)
Thought of the same movie when I read the summary. ...
If only I had mod points today
Parallel reconstruction (Score:3, Interesting)
This is garbage. It will simply lead to parallel reconstruction like the DEA/FBI/CIA does in their court cases when they get evidence by unlawful means like a stingray: the algorithm found a solution to the problem. then it will explain to you, the user how it got there by some arbitrary way which at least looks plausible but is totally made up.
ML is not made to be looked inside, it's a black box by design and there are so many data points, e.g. pictures in the trainingset for image classificiation, the algorithm cannot really show all the relevant ones for this particular decision. Total info overload for the human and therefore utterly useless. So to tell a "reason" that the human can accept, it must simply pretend. Humans and ML work fundamentally different when they "recognize" an image, so one cannot tell the other how it was done. Same with chess playing, same with pretty much all other (successful) AI things so far.
This is simply a PR stunt, an insulting and stupid PR stunt cause it only wants to make people feel good and they lie about the subject matter in the process. It doesn't really help to make a better AI either as they pretend there.
Re: (Score:2)
Humans and ML work fundamentally different when they "recognize" an image, so one cannot tell the other how it was done
Depends on the image. If you spot a family member in a crowd, you can't explain how you did it either.
Re: (Score:2)
The thing is, a neural net doesn't really know how it decided what something was. Making a convincing argument based on the known facts is a separate skill, that AIs so far haven't possessed.
I think the basic argument is that people won't trust AIs just because they're right, they need to have convincing arguments. And this is a way to get it to develop convincing arguments. I *do* think that both arguers should be arguing for the truth as they know it, though. So alter the test, or the training data, s
Re: (Score:1)
Re: (Score:2)
This is garbage. It will simply lead to parallel reconstruction
If someone creates an AI system that can lie about its decision-making process and still make it look good, they will have succeeded.
Re: (Score:2)
Re: (Score:2)
I interpret it as meaning that if he was in a shed full of horses he'd be the smartest guy there.
Re: (Score:2)
As horses can be pretty smart, that is debatable.This would probably be a case where the smartest horse can open the stable door and can get out, while the Donald cannot without the help of the horse but later claims it was his doing.
Re: (Score:2)
I really doubt that half the country believe him, but it seems true that have of the most vocal posters on the internet do. Of course, lots of them are liars, and that makes drawing any conclusion about what they really believe difficult.
Evolution of Evil (Score:2)
"One bot is programmed to tell the truth, while another is programmed to lie"
The good and the bad.
The good and the evil.
Gods programming both in for their own amusement.
Egads.
So, can an AI win ... (Score:1)
... simply by calling all of it's opponents fat, ugly, etc. and in so doing avoid ever having to debate the particulars of any issue?
I mean, humans don't have to demonstrate any higher intelligence to win a debate, so we would be asking AIs to do something we ourselves don't do.
Re: So, can an AI win ... (Score:1)
Fat and ugly don't work but if one AI calls another orange and a traitor and a racist, it will at least think it automatically wins.
Re: (Score:2)
And by using many different fallacies, humans cannot only "win" but also lose and out themselves as morons at the same time!
Re: (Score:2)
Yes. I think AI is the wrong concept. It's not about intelligence but about wisdom. AW is a better term. Now, the reality is that the ultimate judge of wisdom is not another human. It's nature.
Re: (Score:2)
Now, the reality is that the ultimate judge of wisdom is not another human. It's nature.
Physics doesn't judge. It just happens — actions have reactions.
Ob (Score:2)
No they shouldn't.
Not debate (Score:2)
Re: (Score:2)
Debate implies strong AI that can reason about itself, which we do not have. But TFS seems to be describing validation through a competitive pair of AIs, which does not seem novel
Where have you seen previous examples of this?
The validation is an important point - the whole point in fact. When you've got data sets with millions of samples, many containing information in a form that's abstruse or even impossible for humans to understand, how do you validate whether the system actually produced the optimal solution, or the logic behind that choice?
That's a really difficult problem, which I don't think enough people are exploring given how quickly these systems are being deployed into
Re: (Score:2)
Debate implies strong AI that can reason about itself, which we do not have. But TFS seems to be describing validation through a competitive pair of AIs, which does not seem novel
Where have you seen previous examples of this?
Using two differently designed systems on the same data and comparing them isn't new. Or ones that used appropriately constructed subsamples of a dataset that should have identical statistical properties for training.
The validation is an important point - the whole point in fact. When you've got data sets with millions of samples, many containing information in a form that's abstruse or even impossible for humans to understand, how do you validate whether the system actually produced the optimal solution, or the logic behind that choice?
That's a really difficult problem, which I don't think enough people are exploring given how quickly these systems are being deployed into very real scenarios.
I absolutely agree with you. Without rule extraction if the validation set is insufficiently complete, there is a risk of unexpected behaviour. The hope is to minimise it. Not that rule extraction helps unless the rules are very simple, so would not be a silver bullet
"AI"s cannot "debate"... (Score:2)
At this time, we have no AI that deserves the name and it is unclear whether we will ever have it, as there is not even a credible theory how it could be implemented. Looking at the history of technology, this indicates we are > 50 years away from it and it may also be infeasible. All we have is dumb automation and dumb automation cannot "debate". It can give the appearance of doing it (see Eliza), but that is it.
Re: (Score:2)
The wasn't a credible theory for how to make vulcanized rubber either, but it was made. Theories often help, if they're approaching correctness, but they aren't essential.
Actually, we've got loads of tested theories for parts of the process, and we've got a mechanism that has been shown to work, but which is horrendously inefficient in both time and resource usage (evolution) so nobody's applied both the resources and the patience to use it fully. Fortunately it works quite well in a "fill in the gaps" us
Re: (Score:2)
AI1: Bus!
AI2: Car!
That's barely even an argument, let alone a debate. There's no reasoning, no logic, just shouting. It's suitable for the American political process, but it's not intelligence.
Careful what you wish for (Score:2)
"I'm the best bot, believe me! I'm better than humans, than Spock, than HAL something-thousand. Billions flock to praise my bigly brain!"
This was already done by Facebook. (Score:2)
The experiment was shut down when the AIs attempted to adapt English words into a different sentence structure to talk more efficiently but they could no longer be understood by the researchers. People got spooked.
Game over (Score:1)
Not sure a 'game' type approach is what we want here. Seems there are two undesirable/unintended possibilities:
1. The 'competing' AIs treat this as a game and use game-style methods to win, where they are rewarded for 'winning' rather than actually proving their proposition.
2. How long before competing AIs are sufficiently smart that a human judge could not actually, reliably, tell which had proved their proposition ?
This is just Generative Adversarial Networks (Score:1)