IQ Test Pegs ConceptNet 4 AI About As Smart As a 4-Year-Old 121
An anonymous reader writes "Artificial and natural knowledge researchers at the University of Illinois at Chicago have IQ-tested one of the best available artificial intelligence systems to see how intelligent it really is. Turns out–it's about as smart as the average 4-year-old. The team put ConceptNet 4, an artificial intelligence system developed at M.I.T., through the verbal portions of the Wechsler Preschool and Primary Scale of Intelligence Test, a standard IQ assessment for young children. They found ConceptNet 4 has the average IQ of a young child. But unlike most children, the machine's scores were very uneven across different portions of the test." If you'd like to play with the AI system described here, take note of the ConceptNet API documentation, and this Ubuntu-centric installation guide.
More like autistic-savant 4 year old (Score:5, Insightful)
Re:More like autistic-savant 4 year old (Score:5, Funny)
Re: (Score:3, Funny)
Damn it, I'm not even good at slashdot reading - I score bad at math and don't get laid! :(
Re:More like autistic-savant 4 year old (Score:5, Funny)
Re:More like autistic-savant 4 year old (Score:5, Funny)
Re: (Score:3)
He said IQ, not emotional maturity. So 5 1/2.
Re: (Score:2)
But.. But.. Its Mentally Unstable and its got the intelligence of a 4 yr old. Surely its already *passed* the level of your typical CEO?
Re: (Score:1)
Isn't it already past that point then?
Re: (Score:2, Funny)
Of course you have to compare it to a child. How else will it ever ascend to becoming a contestant on "Are You Smarter Than A 5th Grader?"
Re:More like autistic-savant 4 year old (Score:4, Interesting)
Re: (Score:3, Insightful)
Being as intelligent as a 4 year old human on an IQ test is not even remotely related to having the learning abilities of a 4 year old human.
Re: (Score:2)
First thing I thought of was little Anthony Fremont.
(For those that don't get the reference: http://en.wikipedia.org/wiki/It's_a_Good_Life_(The_Twilight_Zone) [wikipedia.org] )
Re: (Score:2)
Re: (Score:3)
Except for the small fact that current AIs have no emotions whatsoever.
Re: (Score:2)
"From the article: âoeIf a child had scores that varied this much, it might be a symptom that something was wrong,â said Robert Sloan, professor and head of computer science at UIC, and lead author on the study."
Yes, exactly. A 4 year old, maybe. But a severely mentally damaged 4 year old.
Re: (Score:2)
Re: (Score:2)
Bullshite.
It just goes to show you what a pile of crap Weschler and Stanford-Binet are. And this is from someone who has a bigger estimated score than my UID. Its bollocks.
Any unimpaired four-year-old knows not to eat feces. That's not measurable through the bias of IQ, and I doubt these "nets" know the difference between that... and Shinola.
Sample output when tested (Score:5, Funny)
No!
Re: (Score:1)
Why?
But (Score:3)
Does the AI use contractions?
Re: (Score:2)
Re: (Score:2)
He does use them in the series finale, in the alternate future timeline. Really, I think it was more a plot device to make sure the crew could distinguish him from Lore, and to be brought up on occasion. Like the episode where Riker is taken hostage by an alien on the surface that wants him to be his Dad, and during a hallucination Data uses a contraction, which served as the final piece of evidence that the events going on were not real.
Re: (Score:2)
Re: (Score:3)
The whistling thing I could understand, since he presumably had no respiration.
Re: (Score:2)
Re: (Score:1)
In the TNG episode Birthright Part 1, Dr Bashir remarks how Data is "breathing" ("..Yes. I do have a functional respiration system....to maintain the thermal control of my internal systems.." and also has a "pulse" ("..My circulatory system not only produces bio-chemical lubricants, but regulates micro-hydraulic power..").
I don't think they explained why we was unable to whistle (in tune), my guess is it was part of the story arc about his quest to become more human.
Re: (Score:3)
Re: (Score:2)
Re: (Score:2, Funny)
Not that it matters, but your XO emoticon means "hug and kiss" in straight culture.
Re: (Score:2)
I never really understood Data's issue with using contractions.
I thought it was a deliberate inability programmed in by his creator to make Data less like Lor and to make him feel less threatening to the other colonists..
Re: (Score:3)
Re: (Score:1)
No, no, you're definitely on to something there.
Re: (Score:2)
No, but at least it doesn't hang when you ask it the value of pi.
Re:But (Score:5, Funny)
for a 4 year old i am pretty sure the value of pi is "more" and possibly "with ice cream"
Re: (Score:3, Funny)
Re:But (Score:5, Insightful)
Re: (Score:2)
Re: (Score:3)
To be accurate, it would need
the same input inaccuracies (pi and pie verbally being the same), combined with
the order of learned experiences which influences weighting (pie before pi), combined with
a 4-year-old's limited capacity for context, combined with
need (eat) & desire (sweet foods).
We have a ways to go.
Re: (Score:2)
Re:But (Score:4, Funny)
Re: (Score:1)
Misleading crap (Score:5, Insightful)
We are nowhere near getting an AI that can navigate the world at the level of a 4 year old. All the program can do is simple tasks in vocabulary and such with no real understanding of those words. Nothing to see here.
Re:Misleading crap (Score:5, Funny)
Re: (Score:1, Insightful)
And you married her. How smart does that make you? About as smart as someone linking his FaceBook account to his Slashdot account.
Re: (Score:1)
Why do you give a shit that he gives a shit that I give a shit that he gives a.... oh never mind.
Re: (Score:2)
A 4 year old is so far ahead of any current AI they shouldn't even think of making this kind of comparison. It would be a Nobel prize winning feat to produce AI that can operate at the level of a 6 month old, let alone something that can walk around, talk, learn, imagine, and play.
Re: (Score:2)
Re: (Score:2)
The ability to look around, recognize faces, react to and locate sounds, control the behaviour of parents with crying, and take objects and put them in its mouth as good as a six month old would be Nobel prize winning AI.
Re: (Score:3)
Re: (Score:1)
Re: (Score:2)
Re: (Score:3)
I never understood why people think a A.I. should learn any faster than a real child could. It's like people think because it's a computer it automagically knows everything there is ever to know, but in reality A.I. still requires training and positive/negative reinforcement just like really children do.
Re: (Score:2)
I agree, but after a program has learned for a while, you can make a copy of it. So if it takes 4 years to teach the program, it doesn't mean it takes 4 years for every copy of the program.
Re: (Score:2)
Re: (Score:2)
It makes sense that you would want M versions of the program, but once you have some that work well enough for something you want to do, you can still create copies of those versions.
Want more diversity? Get real 4 year olds. Don't forget snacks.
Re: (Score:2)
You say that like we have anything remotely like the kind of AI implied by the summary.
We're not even close. Hell, we don't even understand the problem at the simplest level. (Hint: The GP's 6-month-old comment was spot on.)
Re: (Score:1)
Re: (Score:2)
I never understood why people think a A.I. should learn any faster than a real child could. It's like people think because it's a computer it automagically knows everything there is ever to know, but in reality A.I. still requires training and positive/negative reinforcement just like really children do.
Because computers typically is much faster than me at doing something, they can read Wikipedia faster than I could read an A4 page, my math speed is measured in seconds per floating point operation not the other way around and it could query huge database much faster than I could find the index cards at the library, much less find anything. What they're short on is the ability to comprehend and learn, not process. If they're so slow it's because they're waiting for humans to give them feedback or tweak thei
Re:Misleading crap (Score:5, Insightful)
We are nowhere near getting an AI that can navigate the world at the level of a 4 year old. All the program can do is simple tasks in vocabulary and such with no real understanding of those words. Nothing to see here.
The headline is the usual attention grabbing junk, but the article itself does a decent job of explaining it:
Sloan said ConceptNet 4 did very well on a test of vocabulary and on a test of its ability to recognize similarities.
“But ConceptNet 4 did dramatically worse than average on comprehension—the ‘why’ questions,” he said.
One of the hardest problems in building an artificial intelligence, Sloan said, is devising a computer program that can make sound and prudent judgment based on a simple perception of the situation or facts–the dictionary definition of commonsense.
Commonsense has eluded AI engineers because it requires both a very large collection of facts and what Sloan calls implicit facts–things so obvious that we don’t know we know them. A computer may know the temperature at which water freezes, but we know that ice is cold.
“All of us know a huge number of things,” said Sloan. “As babies, we crawled around and yanked on things and learned that things fall. We yanked on other things and learned that dogs and cats don’t appreciate having their tails pulled. Life is a rich learning environment.”
IQ tests mean little enough for a human being, for AI they're little more than cute. Most 4 year old's know if someone is mad at them (expression, tone of voice, etc.) and, from past experience, often know why someone is mad at them. They're also clever enough to pretend they don't know why someone is mad at them. Most importantly (and practically), they know to start acting cute before somebody kills them. Let me know when an AI program can do that.
P.S. This is not to disparage the AI work, just to keep things in perspective.
Re: (Score:3)
What AI researchers seem to be doing is skipping the sensory input part, because it's hard, and just trying to codify the intelligent r
Re: (Score:1)
I've often had thoughts along similar lines: we expect AI systems to be good after a few days or weeks of learning and are surprised they can't match what humans can do after years of learning. There are some projects that do long-term learning like NELL [wikipedia.org] which continually reads text from web pages and uses what it has learned already to better understand the text and therefore learn more from it. Also, maybe it's not quite sensory, but a lot of (most?) AI research these days is in machine learning [wikipedia.org], which is
Re: (Score:2)
What AI researchers seem to be doing is skipping the sensory input part, because it's hard
The sensory input part is easy. It's the subjective experience part that's hard. Well, calling it hard is misleading; it's impossible with purely computational approaches.
Re: (Score:2)
Most importantly (and practically), they know to start acting cute before somebody kills them.
Is this true? I agree with most of what you say, but I feel like when people are really angry at kids, the kids just get scared, not cute
Re: (Score:2)
I feel like when people are really angry at kids, the kids just get scared, not cute
The trick is to get cute just before the adults hit that stage. As to whether this approach works, the proof is that my daughter is still alive.
Re: (Score:2)
Most importantly (and practically), they know to start acting cute before somebody kills them. Let me know when an AI program can do that.
Brb, writing a grant proposal, "KAW-AI-I: Towards technologies for emotionally manipulative artificial agents".
Re: (Score:2)
A four-year-old can project and read emotions with surprising sophistication (especially the project part), understand and communicate with spoken language, climb around and use motor skills for pretty complex tasks, and about a million other things that no one AI is even close to being able to do. Even the best AI's still struggle with such tasks *individually*, much less as a whole package. My daughter could understand plenty of things at 4 that would give Siri a fit, and Siri isn't even potty trained!
Re: (Score:3)
For those NOT utterly terrified at the concept of a strong AI, I suggest you read some of Yudkowsky's writings on the subject here. [yudkowsky.net]
Re: (Score:2)
There's no reason to worry. Computational approaches are all doomed to failure. We can only assume that the researchers are well aware of this well-established fact and that headlines like these are little more than a gag to land some free press.
As for the singularity nuts, well, they might as well be writing about the pending war between the grays and the lizard men in the hollow earth. It's just as silly.
I think they have it backwards (Score:5, Insightful)
Re:I think they have it backwards (Score:4, Interesting)
Re: (Score:2)
the score on the IQ test is correlated highly with the intelligence of the person who took the test.
No, but thanks for playing...
Nearly all IQ tests simply attempt to make a normalized measurment of only two aspects** of intelligence: acquired knowledge (aka crystalized intelligence), and quantitative reasoning. They also often bulk up the tests with basic reading and writing skills assesment. Probably the only case that can be made is that if you do well on an IQ test, you have more than expected amount of acquired knowledge, can read and write the language you took it in and are probably not a slouch
Re: (Score:2)
Then why haven't my IQ scores gone up since I was 18? I have scored right in the range of 135-140 since that time. I did score slightly lower when I was 13 (but it actually wasn't that much lower - can't remember exactly now*).
If IQ is based heavily on acquired knowledge, then increases in IQ test scores should correlate to age...no?
The reason your IQ scores don't change much is that is measured relative to the standard deviation of the testing that was used to "curve" the test. Nearly all IQ indices are scaled so that 130 is 2 std-dev away from median. If you think about standard deviation from a population statistic point of view, if you get approximately the same answers right (you've accumulated the knowledge that is estimated to be approx 2-std above average), you'll score nearly the same all the time (especially if the test doe
Re: (Score:2)
OOOh IQ thread. (Score:1, Troll)
Let me get started:
* IQ tests don't measure intelligence
* IQ tests only measure a certain *type* of intelligence.
* Your jealous because I have an IQ of -2147483648.
* I'm too smart for IQ tests.
* You're book smart but I'm street/code smart
* random troll at -1
Have I missed anything?
Re: (Score:3, Funny)
Re: (Score:2)
Re: (Score:3)
You have an IQ of 0xFFFFFFFF80000000?
Re: (Score:1)
Yes. Vocabulary does not indicate intelligence, merely memory and language exposure and its continued use as an indicator shows how flawed IQ tests are.
Re: (Score:2)
Yes. Vocabulary does not indicate intelligence, merely memory and language exposure and its continued use as an indicator shows how flawed IQ tests are.
Don't feel too bitter, MENSA rejected me, too.
It had a birthday recently (Score:2)
The link seems to point to ConceptNet 5 now.
If they re-run their IQ test, I think they will gleefully find it is now as smart as a 5yr old.
Nice, meaningless score (Score:2)
While I am excited about advancements in AI it makes one wonder what is the use of such scores beyond some marketing?
IQ is debunked. It's not a true measure of intelligence. If anything it can measure of much a person is willing to invest (time/effort) in scoring well on said test.
Compared to other children the scores vary wildly unlike any normal child.
While it's still an achievement to have a sophisticated program worthy of an "AI" label we are, unfortunately nowhere near true AI.
Re: (Score:2)
Reason (Score:2)
That's not a bug, that's the beta version of GPP [wikia.com].
IQ tests only apply to humans (Score:2, Insightful)
These tests don't tell us much about the power of an AI and here is why. If you give a human test with a million questions, then giving one more question is not going to tell you much more. You could probably remove some of the questions too without removing much information about how smart the person is. It turns out some of the questions are much more valuable when it comes to figuring how smart someone is. If you put enough statistics work into that, you'll be able to condense those million questions int
Unfortunately (Score:5, Funny)
Not intelligent (Score:1)
Humans are not programmed to be intelligent. Intelligence is just an aspect of how the brain (specifically the cortex) works.
Even very stupid children or other mammals can learn to do things like catch a ball, walk, remember someone's face or voice or that they had spaghetti for dinner. AI is concentrating on trying to duplicate the wrong types of behavior, starting from the wrong end. It doesn't tell us anything useful about humans or intelligence.
Faking It (Score:3)
"ConceptNet 4 did dramatically worse than average on comprehension—the ‘why’ questions.” - Robert Sloan, lead author of the study.
This comment strengthens my feeling that current AI is making progress in faking many of the accidental attributes of intelligence, but has not discovered the essence.
The development of childrens' mental abilities seems to accelerate over time, as if there is positive feedback, but this does not seem to have emerged in AI yet, especially if we factor out Moore's law. On the contrary, any given exercise in developing AI through machine learning seems to hit a wall of diminishing returns at some point. Is anyone aware of a project that has not experienced this effect?
The actual study? How did they do it? (Score:3)
Anyone knows where to access technical information about the actual study, or how did they conducted the IQ test? ConceptNet is just a database + a library with some NLP parsing tools and database (the concept hypergraph) accessors, but I wonder how did they actually conducted the test as that doesn't seem to be a trivial extension of the available tools...