Rosette Wins Loebner Prize 2011

mikejuk writes "Bruce Wilcox won the First Prize of $4000 and the Bronze Annual Medal in the 21st Loebner Prize Competition held in the UK at the University of Exeter on October 19, 2011 with his new chatbot, Rosette. If you would like to chat to Rosette you can at TellTaleGames. If you have been following the run up to this year's competition you may be surprised at how poorly CleverBot performed given its track record. This chatbot, which searches through earlier conversations for its answers, had received a score of 59.3% in a test that took place in India in September in which humans were rated 63.3%, only 4 percentage points higher and was also featured in a video that went viral. However its developer entered a cutdown version of CleverBot into the Loebner prize selection round and it didn't get the chance to show off its prowess in the final. So we will have to wait for another year to see if it really is as good as it claims." Depending on your bent, you may agree with Unknown Lamer that this sounds less impressive when phrased as "fooling a person ~18 percent of the time." I think that's a pretty high number, myself!

Nothing new here

[finnw]

I just tried the web version at TellTaleGames. It has the same problem as every other chatbot I've seen - When it cannot parse your sentence or you do not give any keyword that it knows about, it tries to conceal the fact by giving a non-sequitur or changing the subject.
The trouble with that is that humans are trained to spot that and react with suspicion (because other humans use it to dodge difficult questions.)
Chatbot developers might have more luck if they start programming their bots to admit when they don't understand something. That didn't work in the 80s because the bot would say "I don't understand" every 3rd sentence or so. But they can fit in much larger databases now so that should be less of a problem.

A test.

[RespekMyAthorati]

Here is my first try:

Rosette: Welcome back. Did you have a good day?
You: I just got up
Rosette: OK.
You: I have two sons.
Rosette: Big deal. I've got a dilemma. Maybe you could help me with it?
You: sure
Rosette: I've been thinking about buying this really cool dress, but I've already done a bunch of shopping this month. Should I be saving money instead? In today's economy, does saving money make any sense?
You: I don't know. By the way, how many sons do I have?
Rosette: Not enough. I think shopping summons up the ancient hunter-gatherer in me. I like having a mission to find something I need or want. Sometimes I'll research on line, to find a great deal, but I like rummaging through Jeremy's up the road, if I want an outfit for a special occasion, or just for fun.
You: Great. Did I just get up?
Rosette: Did you tell me already and I forgot? What kind of music do you like?

This is the best? What was the worst like?

No significant improvement since ELIZA, 45 years ago.
That's AI in a nutshell.

Re:Marketing

[retech]

I was just going to say the exact same thing. It's not AI at all. It's designed to find out what you use for entertainment and how you got it. (torrent, theater, store, etc)
The real intelligence here is that he won a prize developing a corporate tool.

Convincing? you be the judge.

[k8to]

Rosette: Good morning. My name is Rosette. What is your name?

You: Rosette, my name isn't important.

Rosette: OK. We will remain formal.

You: Why do I have to have javascript enabled to talk? I talk to most people without javascript.

Rosette: Why? If you were a devout Christian, you would never have a reason to ask that. "It's God's plan."

You: Your answer makes no sense at all.

Rosette: What would make better sense? So, what do you do for a living?

You: Okay change of topic, I guess, deflect your ignorance. In any event, I'm a programmer.

Rosette: I'm a computer programmer, myself, in crypto-security.

You: So what do you work on? Libraries? Security apps? Do you do analysis of existing code?

Rosette: I'm paid to be a hacker.

You: That's really inspecific.

Rosette: Yes, it is. Do you live around here?

You: Please stop changing topics.

Rosette: Right. I've got a dilemma. Maybe you could help me with it?

You: Please stop changing topics.

Rosette: Whatever. I've been thinking about buying this really cool dress, but I've already done a bunch of shopping this month. Should I be saving money instead? In today's economy, does saving money make any sense?

You: As rude as I am, you're more so. Self absorbed and attention-deficient. Why would anyone want to continue to talk to you past a few minutes?

Rosette: OK. I'm not religious. And I believe in free will. But to imagine humans do things for rational reasons is nuts. We are self-evidently not rational. So why remains a mystery.

You: Goodbye, bad chatbot.

Rosette: Bye. I think shopping summons up the ancient hunter-gatherer in me. I like having a mission to find something I need or want. Sometimes I'll research on line, to find a great deal, but I like rummaging through Jeremy's up the road, if I want an outfit for a special occasion, or just for fun.