Rosette Wins Loebner Prize 2011 57
mikejuk writes "Bruce Wilcox won the First Prize of $4000 and the Bronze Annual Medal in the 21st Loebner Prize Competition held in the UK at the University of Exeter on October 19, 2011 with his new chatbot, Rosette. If you would like to chat to Rosette you can at TellTaleGames. If you have been following the run up to this year's competition you may be surprised at how poorly CleverBot performed given its track record. This chatbot, which searches through earlier conversations for its answers, had received a score of 59.3% in a test that took place in India in September in which humans were rated 63.3%, only 4 percentage points higher and was also featured in a video that went viral. However its developer entered a cutdown version of CleverBot into the Loebner prize selection round and it didn't get the chance to show off its prowess in the final. So we will have to wait for another year to see if it really is as good as it claims." Depending on your bent, you may agree with Unknown Lamer that this sounds less impressive when phrased as "fooling a person ~18 percent of the time." I think that's a pretty high number, myself!
Maybe fooling a complete moron 18% of the time (Score:2)
I've tried Cleverbot a bunch of times and I was really unimpressed. Just being able to spit back a human sounding response is really insufficient for this. The path of the conversation remains a random jumble. I've never even come close to thinking that it could be a human on the other end. It seems like the real test is the ability of the bot to hold a conversation about an arbitrary subject, not just random (and I really mean random) banter.
Re: (Score:1)
Considering how easily people are fooled (like the 92% that believe in God, angels, etc.) you'd think it would be a much higher percentage.
Re: (Score:3)
It'd be more difficult to tell the difference if the responses lagged a little bit, you know, like it took some time for someone to read your text, formulate and type a response.
Re: (Score:2)
the real question is if she's hot or not! (.com)
Re: (Score:3)
There was a little lag when I tested it but she basically ignored my questions and asked her own. So in that sense it is like a typical woman.
GREAT IDEA!! chat-with-god.com (Score:2)
Somebody should be making prayer bots, god chat, and dead relative chats!! Twitter your dead relative! etc.
Maybe even some money in that one?
Wouldn't it be awesome if someday there is a fan club or cult of followers of a chat bot? cyber prophet? why not.... Joseph Smith pulled it off and wasn't even believable; how many years until we get a bot as capable? I suppose you'd need a human face on it because people would be less trusting of a machine... I also suppose having it in text on the internet would al
Re: (Score:1)
God does exist in Chatterbot form already [titane.ca].
(Actually, this one is based on ALICE. If you ask it if its name is pretty much anything, it will say that its name is God, but if you ask if its name is Alice, it will say 'yes'.)
Re: (Score:2)
How does that make you feel?
Re: (Score:1)
You're in a desert, walking along in the sand, when all of a sudden you look down...
What one?
What?
What desert?
It doesn't make any difference what desert, it's completely hypothetical.
But, how come I'd be there?
Maybe you're fed up. Maybe you want to be by yourself. Who knows? You look down and see a tortoise, Leon. It's crawling toward you...
Tortoise? What's that?
You know what a turtle is?
Of course!
Same thing.
I've never seen a turtle... But I understand what you mean.
You reach down and you flip the tortoise
Nothing new here (Score:4, Insightful)
I just tried the web version at TellTaleGames. It has the same problem as every other chatbot I've seen - When it cannot parse your sentence or you do not give any keyword that it knows about, it tries to conceal the fact by giving a non-sequitur or changing the subject.
The trouble with that is that humans are trained to spot that and react with suspicion (because other humans use it to dodge difficult questions.)
Chatbot developers might have more luck if they start programming their bots to admit when they don't understand something. That didn't work in the 80s because the bot would say "I don't understand" every 3rd sentence or so. But they can fit in much larger databases now so that should be less of a problem.
A test. (Score:4, Insightful)
Rosette: Welcome back. Did you have a good day?
You: I just got up
Rosette: OK.
You: I have two sons.
Rosette: Big deal. I've got a dilemma. Maybe you could help me with it?
You: sure
Rosette: I've been thinking about buying this really cool dress, but I've already done a bunch of shopping this month. Should I be saving money instead? In today's economy, does saving money make any sense?
You: I don't know. By the way, how many sons do I have?
Rosette: Not enough. I think shopping summons up the ancient hunter-gatherer in me. I like having a mission to find something I need or want. Sometimes I'll research on line, to find a great deal, but I like rummaging through Jeremy's up the road, if I want an outfit for a special occasion, or just for fun.
You: Great. Did I just get up?
Rosette: Did you tell me already and I forgot? What kind of music do you like?
This is the best? What was the worst like?
No significant improvement since ELIZA, 45 years ago.
That's AI in a nutshell.
No. AI in a nutshell is this: (Score:2)
Passing the Turing test, when it comes to contests like these, has mostly changed from an attempt to create artificial intelligence into an attempt to convincingly simulate human stupidity.
Re: (Score:2)
Which may be a more accurate portrayal of average human behavior than intelligence. The depressing thing about fooling 18% isn't what it says about the 18%, but what the 18% are effectively saying about the rest of us.
Re: (Score:2)
Anyway I'm biased. In my opinion it's better to focus on augmenting humans than to create AIs. The lines will get blurry in some cases, but there's still a difference in philosophy and objective.
If you want nonhuman intelligence there's already plenty at the pet stores and farms, AND we don't treat them all that well, so why create AIs? To enslave them? What if they are smart enough to not want to be slaves? What if we create AIs tha
Re: (Score:1)
You seriously can't imagine the outcome of this one?
We would pull the plug the instance it uttered these desires.
Re: (Score:3)
You should consider yourself fortunate. I tried it just how and it seems the computer running the bot has been slashdotted:
Re: (Score:2)
No significant improvement since ELIZA, 45 years ago. That's AI in a nutshell.
45 years ago, software couldn't recognize your voice, read your license plate, beat a chess grandmaster, translate foreign documents, or map your body into video games. Sure, the field has had its share of false promises, but--unlike fusion, for instance--AI has delivered solid and commercially-viable results over the past 2-3 decades by helping us solve numerous problems that were otherwise unapproachable.
Incidentally, I had an AI researcher tell me that the Loebner prize was akin to "trying to reach the
Re: (Score:1)
Well, the guy has a point that the chatbot blows. I tried it out too, and after about 6 or 7 messages it threw me the dilemma thing too.
And you have a point that AI is definitely improving :) Btw, when are we getting together again, we should do at least once a month no?
Phos
Re: (Score:2)
Rosette: I'm a little busy at the moment. Could you wait a bit?
You: Sure, why not.
Rosette: I'm a little busy at the moment. Could you wait a bit?
Just like every other woman with whom I chatted before. Sounds perfectly human to me.
Stunningly bad (Score:3)
--------------
Rosette: Good morning. My name is Rosette. What is your name?
You: Hey there!
Rosette: Greetings.
You: My name is Bob, nice to meet you
Rosette: Nice to meet you, Bob nice to meet you. May I call you nice?
---------------
yeah yeah, I used a comma instead of a semicolon. I guess that tripped it up?
Re: (Score:3)
Whenever I said "I'm hungry", my dad would reply "nice to meet you, hungry".
Guess my dad would make a pretty bad AI.
Re: (Score:2)
why claim you are if it is not so ?
hmmm...puzzled.. gonna think about it...
Re: (Score:2)
Heh, reminds me of a giant boat of a car my grandpa had in the late 80s. Instead of just "dinging" at you when you left the lights on or the door open, it would speak to you.
Car: "The door is ajar".
Grandpa: "No it's not, it's a door, you stupid car!"
Hurr hurr hurr. RIP, Papa.
Marketing (Score:1)
Re:Marketing (Score:4, Insightful)
The real intelligence here is that he won a prize developing a corporate tool.
Convincing? you be the judge. (Score:4, Insightful)
Rosette: Good morning. My name is Rosette. What is your name?
You: Rosette, my name isn't important.
Rosette: OK. We will remain formal.
You: Why do I have to have javascript enabled to talk? I talk to most people without javascript.
Rosette: Why? If you were a devout Christian, you would never have a reason to ask that. "It's God's plan."
You: Your answer makes no sense at all.
Rosette: What would make better sense? So, what do you do for a living?
You: Okay change of topic, I guess, deflect your ignorance. In any event, I'm a programmer.
Rosette: I'm a computer programmer, myself, in crypto-security.
You: So what do you work on? Libraries? Security apps? Do you do analysis of existing code?
Rosette: I'm paid to be a hacker.
You: That's really inspecific.
Rosette: Yes, it is. Do you live around here?
You: Please stop changing topics.
Rosette: Right. I've got a dilemma. Maybe you could help me with it?
You: Please stop changing topics.
Rosette: Whatever. I've been thinking about buying this really cool dress, but I've already done a bunch of shopping this month. Should I be saving money instead? In today's economy, does saving money make any sense?
You: As rude as I am, you're more so. Self absorbed and attention-deficient. Why would anyone want to continue to talk to you past a few minutes?
Rosette: OK. I'm not religious. And I believe in free will. But to imagine humans do things for rational reasons is nuts. We are self-evidently not rational. So why remains a mystery.
You: Goodbye, bad chatbot.
Rosette: Bye. I think shopping summons up the ancient hunter-gatherer in me. I like having a mission to find something I need or want. Sometimes I'll research on line, to find a great deal, but I like rummaging through Jeremy's up the road, if I want an outfit for a special occasion, or just for fun.
Re: (Score:2)
I think perfect grammar makes the whole thing even less convincing. Not many people put that much effort in chat talk. And the whole thing seems too vanilla to me.
Lost me at the first reply (Score:2)
Rosette: Good afternoon. My name is Rosette. What is your name?
You: Hi Rosette, I'm Bob.
Rosette: Hi to you too.
Bzzzzzzzt, fail. I reply to a greeting and name request by reciprocating the greeting and my name. It responds with yet another reciprocation of the greeting, something a human would almost never do.
Sheesh... I thought they said Rosetta at first ... (Score:2)
I was thinking, "Kind of late to give Apple a prize for their PPC emulation when OS X Lion already removed it!"
Re: (Score:2)
Re: (Score:2)
You: hi
Rosette: I'm a little busy at the moment. Could you wait a bit?
You: sure
Rosette: I'm a little busy at the moment. Could you wait a bit?
Rosette: I'm a little busy at the moment. Could you wait a bit?
You: ok, ok. you don't have to keep saying that.
Rosette: I'm a little busy at the moment. Could you wait a bit?
Rosette speaks with to Cleverbot (Score:1)
Just a quick test i threw together.
Rosette: Hi, again,. Is everything OK?
Cleverbot: Not really, but there's nothing much I can do about it.
Rosette: I'm sorry to hear that.
Cleverbot: Can you help me?
Rosette: Not often. can you?
Cleverbot: You are fibbing again.
Rosette: No. You are fibbing.
Cleverbot: No, I'm not.
Rosette: You disagree? My name is Rosette. What is your name?
Cleverbot: Cleverbot.
Rosette: Nice to meet you, Cleverbot. May I call you Cleverbot?
Cleverbot: Nice to meet you. How old are you?
Rosette: Hi
Off in the weeds in two sentences... (Score:2)
Rosette: Good evening. My name is Rosette. What is your name?
You: None of your business.
Rosette: OK. We will remain formal.
You: You're damn straight we will.
Rosette: I like card games, but never for money.
You: You only play strip poker, eh?
Rosette: I like board games, and played a lot as a child. I haven't that much time these days.
Alternately - and here's a hint, chatbot devs - my favorite test for chatbots is to simply echo back whatever they say to me. Rosette takes a while to start to get the nonsensic
Very poor; caught out during by first exchange. (Score:2)
Rosette: Good morning. My name is Rosette. What is your name?
You: Pardon
Rosette: Nice to meet you, pardon.
You: My name is not Pardon
Rosette: Nice to meet you, not Pardon. May I call you not?
I get the impression this is not a human (Score:1)
Rosette: Pretty good, thanks. How are you?
You: Not too good, I'm afraid.
Rosette: Nice to meet you, afraid. May I call you afraid?