The New AI: Where Neuroscience and Artificial Intelligence Meet 209
An anonymous reader writes "We're seeing a new revolution in artificial intelligence known as deep learning: algorithms modeled after the brain have made amazing strides and have been consistently winning both industrial and academic data competitions with minimal effort. 'Basically, it involves building neural networks — networks that mimic the behavior of the human brain. Much like the brain, these multi-layered computer networks can gather information and react to it. They can build up an understanding of what objects look or sound like. In an effort to recreate human vision, for example, you might build a basic layer of artificial neurons that can detect simple things like the edges of a particular shape. The next layer could then piece together these edges to identify the larger shape, and then the shapes could be strung together to understand an object. The key here is that the software does all this on its own — a big advantage over older AI models, which required engineers to massage the visual or auditory data so that it could be digested by the machine-learning algorithm.' Are we ready to blur the line between hardware and wetware?"
no (Score:5, Insightful)
Are we ready to blur the line between hardware and wetware?
No. You can't ask that every time you find a slightly better algorithm. Ask it when you think you understand how the mind works.
Re: (Score:3)
So in what order does this happen?
1. 120 fps: orientation;
2. 60 fps: motion;
3. 30 fps: color.
The rest of the research can be done on one's own:
http://www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2Fjournal.pcbi.1000555 [ploscompbiol.org]
Re: (Score:3)
Nowhere near that simple. Take just the distance/depth for example. Close one eye. Notice that you can stil fairly well judge how far away an object is? That's because your brain can still recognise those objects and, based on prior experience, know how big they should be. Then estimate distance based on image size and object size. A common class of optical illusion involves objects deliberately made much larger or smaller than is typical, causing errors in distance estimation.
Re: (Score:2)
PS: the mistake you are making, that that the visual cortex of the human brain, does:
A. Everything in linear stages, with parallel computation, and;
B. Likes to not waste calculation power on mundane things like: "Realy? Is that lawn all grass? Let's examine every shape first, in order to make sure there is no soldier out there with cammo, who might just try to shoot me", and;
C. That's where optical illusions come into play: the cortex uses a shitload of shortcuts.
Of course you model would work if there are
Re: (Score:2)
http://xkcd.com/793/ [xkcd.com]
Re: (Score:2)
Get of my lawn:
http://science.slashdot.org/story/09/11/13/1545245/the-math-of-a-flys-eye-may-prove-useful [slashdot.org]
fly brains (Score:4, Interesting)
Are we ready to blur the line between hardware and wetware?
We can now almost convincingly partially recreate the wetware functions of Drosophila melanogaster. Whether we're *ready* for this is another question; as is whether this is what folks have in mind by "AI."
Saving everyone a few seconds on wiki (Score:5, Informative)
Re:Saving everyone a few seconds on wiki (Score:4, Interesting)
Do you really think AI will not follow a moore type law? It will probably be even more aggressive.
I personally expect Moore's Law to set a lower bound on the time needed for advancement. Doubling every 18-24 months means 20-30 years to get human-sized big ol' clusters of neurons. However, there's also so much work to do on understanding the specifics of how to get particular results (e.g. language and "symbolic thought") instead of just gigantic twitching masses of incoherent craziness.
In order to try out ideas and test hypotheses, you really need to be able to run a whole bunch of human-brain-scale simulators at far higher speed than the human brain (learning a language takes a couple years for a developing human brain, and you're very unlikely to get this "right" with only one or two tries). I think once we have 10^3 - 10^6 times more "raw neuron simulation" processing power than a single human brain (so another 10 to 40 years after the 20-30 years for single-brain neuron simulations), then we'll be able to crank out simulations of the "hard stuff" fast enough to make rapid progress on the high-level issues. Of course, this means once you do have a couple "breakthroughs" in generating self-aware, learning, human-language-understanding machines, you're very suddenly dropped into having far-exceeding-human artificial intelligences, without so much of a slow progression through "retarded chimpanzee" stages first.
Re:Saving everyone a few seconds on wiki (Score:5, Funny)
" However, there's also so much work to do on understanding the specifics of how to get particular results (e.g. language and "symbolic thought") instead of just gigantic twitching masses of incoherent craziness."
In the meantime we'll just have to settle for modeling a teenager.
Re: (Score:2, Informative)
Aka 'Purple Drank' and 'Sizzurp'. That stuff killed a lot of its early proponents [wikipedia.org]. The original recipe contained codeine and promethazine, not DXM.
Re:Saving everyone a few seconds on wiki (Score:5, Insightful)
That presumes that the approach you take is going to be using the same kind of models you have now and just running them on bigger, faster hardware. If our models lead us to *understanding* of how brains work, we could get there a good deal faster and find that present day computers are plenty complex to handle cognition on a human-equivalent level.
Take Google self-driving cars for example. Driving a car is definitely an AI task, and it can be handled by present day computers. It's a subset of the tasks humans can learn. Google didn't do it by modeling the part of your brain that drives a car. Hell, we don't even know what subset of our brain is sufficient to drive a car. They did it by understanding how to drive a car.
What I'm proposing is that human-level AI won't be created first by modeling a whole brain. It will more likely be created by scientists by studying the brain come to understand what the big-picture behavior of brain subsystems and modeling those subsystems at a behavioral level rather than at a neural-network level.
Re:Saving everyone a few seconds on wiki (Score:4, Interesting)
As I agree in another branch of this thread, we probably will find "non-brainlike" methods to generate all sorts of "intelligent" behavior, continuing the same type of progress (not particularly worrying about biologically accurate brain models) that gives us self-driving cars. On the other hand, it's a separate worthwhile field of study to learn how *our* brains work, through models that capture key features of biological brains.
If our models lead us to *understanding* of how brains work, we could get there a good deal faster and find that present day computers are plenty complex to handle cognition on a human-equivalent level.
Maybe; maybe not. Our understanding might well *not* allow much brain function (above the Drosophila level, which is about appropriate for a moderate sized supercomputer today) to be vastly simplified for lesser computing resources --- maybe you do *need* zillions of complexly interlinked neurons to see more interesting higher level behaviors (in a brain-like manner, not by creating non-brainlike intelligences like the self-driving car that have similar "skills"). The brain may not neatly "factor" into simple-to-computationally-model "subsystems". If you look at, e.g., chemical pathway maps for how a cell functions, everything is tangled together with everything else --- biological systems often evolve "spaghetti code" solutions to problems, without the neatly defined boundaries and modularity that a "top down" systems designer would impose.
Re: Saving everyone a few seconds on wiki (Score:2)
this presumes that the algorithms of the task in question are tractable, and in the brain in a aensible order.
in the cases of some tasks, it's looking like that's not really true. There is a sea of randomly interconnected neurons that get wired together by correlated inputs into the random sea.
These neurons learn they are associated, and wire together.
This may not lead to an extractable algorithm.
I_highly_ recommend the brain science podcast.
http://brainsciencpodcast.wordpress.com/2007/11/16/brain-science-p [wordpress.com]
Re: (Score:2)
Do you really think AI will not follow a moore type law?
Why should it? Moore's law applies to one specific technology, which happens to be a technology that scaled/improve more than almost any other in history. There used to be a popular analogy, if cars improved as much as chips have, a car would cost a nickel, travel a million miles an hour and go around the world twice on a teaspoon of (low octane) gas, or something like that. Unfortunately most technologies don't improve that much. If neural nets are implemented on chips, they'll run into the limits of Moore
Re: (Score:3)
The way I heard it, cars would cost a nickel, travel around the world on a teaspoon of gas, have a top speed of 30 trillion miles per second (never mind the speed of light) and spontaneously lock up their controls while driving at highway speeds.
Based on Moore's law type expansion of capabilities over a century.
Re:Saving everyone a few seconds on wiki (Score:5, Interesting)
What precisely are those long-standing problems?
I ask because I actually know people who are starting to demonstrate the rudiments of intelligence using simulations of ~100,000 neurons.
Per upthread, that's a long way from a brain, and in fact we don't even know how all of the brain is wired, let alone how it works. But you might want to consider this [engineerin...lenges.org] and this [wikipedia.org] and this [nature.com].
If they're attempting the impossible, you should let them know not to waste their money.
Re: (Score:2)
Are you claiming that symbol grounding is a non-solvable problem?
Re:Saving everyone a few seconds on wiki (Score:4, Insightful)
I'm saying that it's unsolved (er, well, I thought that would go without saying!) and that, at present, it and similar problems strongly suggest that this type of approach is fundamentally flawed.
My main point was that it's unreasonable to believe that those problems will be solved by magic and wishful thinking. This cargo-cult approach to AI purports to do just that. (If we just ignore the problems hard enough, technology will deliver us!)
Re:Saving everyone a few seconds on wiki (Score:4, Insightful)
Your assertion that a 'cargo cult' approach cannot achieve a given effect contains the assumption that it is necessary to first develop an accurate understanding of why and how a potential mechanism works before it can be implemented.
All crop development prior to Mendel or Darwin, for example, was essentially cargo cult directed evolution--and yet it resulted in incredible development (e.g., corn from teocinte).
More generally, achievement of an effect isn't just possible without understanding, it's possible without intent. Predators culling prey populations such that frequency of undesirable alleles within the prey population is minimized is an entirely unintentional effect. "Cargo Cult" solutions are simply scenarios where you have intent but lack understanding (which again does not mean that the solution will necessarily be ineffective).
With respect to the neuron modeling approach, it actually builds on lots of earlier successful work in computer science with respect to emergent properties of systems of finite automata. Essentially the approach follows the sequence:
Note that in the above approach you not only recreate something before you understand it or how it works--you do so specifically to gain a better understanding of how it works. This is certainly a realistic scenario of how strong AI could be developed via "cargo cult" methodology. It is entirely possible that creating synthetic intelligence will be a step towards the understanding intelligence as opposed to an outcome of that understanding.
Re:Saving everyone a few seconds on wiki (Score:4, Insightful)
No, that's not cargo cult. Cargo cult is when you imitate the actions of someone for whom those actions have meaning, without understanding their meaning yourself (or totally misunderstanding their meaning). Crop development was haphazardly experimental, not cargo cult.
Re: (Score:2)
Your assertion that a 'cargo cult' approach cannot achieve a given effect contains the assumption that it is necessary to first develop an accurate understanding of why and how a potential mechanism works before it can be implemented.
No. You came up with that all on your own.
To add: I mention "long-standing problems" which suggest that the effort in question is ultimately futile. These problems are well-established and fundamental to the AGI problem the summary implies that we're on the brink of solving. To ignore them expect that those problems will just vanish if we just build a better bamboo airplane is nothing short of magical thinking.
With respect to the neuron modeling approach
You mean the construction of bamboo airplanes and dirt runways? I won't argue their utility, b
Re: (Score:3)
That is the meaning that I usually assign to the term cargo cult as well, but I was using it in my post in the same manner as my original parent poster.
If we assume that I misinterpreted my parent poster's meaning, and it was in fact using the definition you provided, then the implication is that if neural net modeling is a cargo cult activity we must be imitating the actions of someone else who doesunderstand the fundamental nature and mechanism of intelligence. Unless my parent poster is insinuating the
Re: (Score:3)
The symbol grounding 'problem' isn't a problem for AI at all. It is merely a fundamental misunderstanding of the world born out of the arrogance that our consciousness and sentience are somehow special and must arise from something non-physical. If you disagree, I challenge you to define 'meaning' and show me how an artificial neural net cannot possess it.
Beyond the above, even if a sufficiently advanced AI being was somehow devoid of 'understanding' or 'meaning' (in the Searle's Chinese Room sense), it wou
Re: (Score:3)
Some people actually copy neural mapping of a tiny piece of the brain.
It function as if it is part of the brain.
Why is this a cargo cult? They copied part of the brain and it works. If the made a copy of the brain and it didn't work, then it would be a cargo cult.
Not only do you not seem to now what a cargo cult is, you also seem to be about 2 decades behind in research.
The whole article is about how this is working and being put into place.
Re: (Score:2)
Do you know anything at all about the Blue Brain project?
Serious question: if you do not then there is a video floating around from ICC'11 with Henry Markram explaining an overview of the project. Given that they are building artificial simulations of biology specifically so that they can explore how they work, build hypotheses and then experimentally validate them it is somewhat hard to see how this approach can be described as cargo-cult AI.
Re:Saving everyone a few seconds on wiki (Score:4, Insightful)
Forget the long-standing problems that make this approach a non-starter.
Did you actually watch IBM's "Watson" beat the snot out of the best Jepordy champions humanity could muster? I can't believe that anyone who knows anything about computers and AI is not blown away by Watson's demonstration, I know I was. My significant other who has a phd in marketing just shrugged and said "it's looking up the answers on the internet, so what?". In other words if your not impressed by Watson's performance, it's because you have no idea how difficult the problem is.
Re: (Score:2)
Watson is neat. It's also completely irrelevant to both the topic and my post.
Re: (Score:2)
"it's looking up the answers on the internet, so what?"
Ironic, since the answer on the Internet is also given by the AI.
Re: (Score:2)
Thanks for reminding me about Watson. Never saw that episode. Gotta love youtube...
https://www.youtube.com/watch?v=seNkjYyG3gI [youtube.com]
Re: (Score:3)
Whether we're *ready* for this is another question; as is whether this is what folks have in mind by "AI."
Since what folks have in mind by "AI" changes to exclude anything within the capability of machines, we're implicitly ready for whatever emerges.
Re: (Score:2)
AI certainly is a moving target --- I remember when "play chess at an advanced human level" was considered an (unachievable) goalpost for "real AI." On the other hand, I'm not certain we're ready by default for the capabilities of machines, intelligent or no.
Re:fly brains (Score:5, Insightful)
I say all of the following as a big fan of AI research. I just think we need to drop the rhetoric that we're somehow recreating brains -- why do we feel the need to claim that intelligent machines would need to be similar to or work like real brains?
Anyhow...
We can now almost convincingly partially recreate the wetware functions of Drosophila melanogaster.
Interesting wording. Let's take this apart:
In sum, we have a few algorithms that seem to take input and produce some usable output in a manner very vaguely like a few things that we've observed in the brains of fruit flies. Claiming that this at all "recreates" the "wetware" implies that we understand a lot more about brain function and that our algorithms ("artificial neurons"? hardly) are a lot more advanced and subtle than they are.
Re:fly brains (Score:4, Interesting)
Yes, I intended my very weasel-worded phrase to convey that even our present ability to "understand" Drosophila melanogaster is rather shallow and shaky --- your analysis of my words covers what I meant to include pretty well.
why do we feel the need to claim that intelligent machines would need to be similar to or work like real brains?
I don't think we do. In fact, machines acting in utterly un-brainlike manners are extremely useful to me *today* --- when I want human-style brain functions, I've already got one of those installed in my head; computers are great for doing all the other tasks. However, making machines that work like brains might be the only way to understand how our own brains work --- a separate but also interesting task from making machines more useful at doing "intelligent" work in un-brainlike manners.
Re: (Score:3)
why do we feel the need to claim that intelligent machines would need to be similar to or work like real brains?
I don't think we do.
I absolutely understand what you mean here. I don't think most AI researchers actually think they are "recreating wetware" explicitly or that the "artificial neurons" in "neural nets" are really anything like real neurons.
On the other hand, a lot of the nomenclature of AI seems to deliberately try to make analogies -- "deep learning," "neural nets," "blur the line between hardware and wetware," etc. -- to human or animal brain functions.
Hence my rhetorical question about why we feel the need to claim t
Re:fly brains (Score:4, Interesting)
I suppose some of the urge to "anthropomorphize" AIs comes from the lack of precedent, and even understanding of what is possible, outside of the two established categories of "calculating machine" and "biological brain." Some tasks and approaches are "obviously computery": if you need to sort a list of a trillion numbers, that's clearly a job for an old-fashioned computer with a sort algorithm. On the other hand, other tasks seem very "human": say, having a discussion about art and religion. There is some range of "animal" tasks in-between, like image recognition and navigating through complex 3D environments. But we have no analogous mental category to non-biological "intelligent" systems --- so we think of them in terms of replicating and even being biological brains, without appropriate language for other possibilities.
Re:fly brains (Score:4, Interesting)
Biological neurons are far more complex than ANN neurons. At this point it's unknown if we can make up for that lack of dynamic state by using a larger ANN, or by increasing the per-neuron complexity to try to match the biological counterpart. I do have my doubts about the former, but that doubt is merely intuition, not science. We simply don't know yet.
as is whether this is what folks have in mind by "AI."
In my own studies, this isn't the branch of AI I'm particularly interested in. I don't care about artificial life, or structures based around the limitations of the biological brain. I'd love to have a system with perfect recall, could converse about its knowledge and thought processes such that conversational feedback would have immediate application without lengthy retraining, and could tirelessly and meticulously follow instructions given in natural language.
I don't see modeling biological brains as being a workable approach to that view of AI, except maybe the "tirelessly" part. I'm more interested in cognitive meta-models of intelligence itself than the substrate on which currently known knowledge happens to reside.
Re: (Score:2)
I see a combinatorial explosion at the horizon.
CC.
Re: (Score:2)
Right, that's why dealing with such things requires intelligence, and if it could do such a thing would generally be considered intelligent. It's also why symbolic AI has failed to produce any general intelligence, because it simply cannot scale. In order for a system to exhibit such behavior, it needs to adaptively and "intelligently" prioritize what it's doing and on what it's working, as well as to predictively index and preprocess information, in order to even begin to achieve any sense of tractabilit
Re: (Score:2)
perfect recall
Conflicts with prioritizing if you have provisions for priority zero (forgetting, irrelevant if the link goes away or the information is erased).
could converse about its knowledge and thought processes
Telling more than we can know (Nisbett &Wilson, 1977, Psychological Review, 84, 231–259), protocol analysis, expert interviews: evidence that this is at least not always possible. My hypothesis is that too much metaprocessing would lead to a deadlock.
conversationa
Re: (Score:3)
I'm in a hurry, but I'll dump some disconnected thoughts at you. I appreciate my having to take these still-vague thoughts and being more specific with them in discussion.
(perfect recall)
Conflicts with prioritizing if you have provisions for priority zero (forgetting, irrelevant if the link goes away or the information is erased).
Recall of perceptions in particular is a very useful measure, considering "Oh, that's what you meant" style moments as new information is gained in the future, allowing reinterpretation of past input. This actual storage & recall is not a large technical challenge, especially when no continuous real-world sensors are involved (just d
Re: (Score:2)
Biological neurons are far more complex than ANN neurons. At this point it's unknown if we can make up for that lack of dynamic state by using a larger ANN, or by increasing the per-neuron complexity to try to match the biological counterpart.
Why? I mean, I vaguely remember reading about a decade ago about a discrete 'neuron' IC that mimics the electrical properties of a biological neuron with perfect accuracy. If it can be done in hardware, it can be done in software. There's no requirement for ANNs to be matrices of coefficients; arbitrary degrees of complexity are possible, both in terms of ANN size and the complexity of neurons. Basically, you're saying that we may not be able to increase per-neuron complexity without bound. Why?
Re: (Score:2)
Are we ready to blur the line between hardware and wetware?
We can now almost convincingly partially recreate the wetware functions of Drosophila melanogaster. Whether we're *ready* for this is another question; as is whether this is what folks have in mind by "AI."
Wake me up when the AI will be just as complex as my guts [wikipedia.org] (10^8 neurons the same magnitude as the cortex of a cat [wikipedia.org]) and then I'll ask them if they feel they are ready for the AI.
Geoffrey Hinton (Score:5, Informative)
He has a course on them at coursera that is pretty good.
https://www.coursera.org/course/neuralnets [coursera.org]
Re:Geoffrey Hinton (Score:5, Informative)
Look, this stuff goes back a long ways and has had some hiccups along the way, like the twenty year period it was treated with little more respect by the scientific establishment than has cold fusion for the last twenty years. There are plenty of heroics to go around.
I can recommend the book highly.
Re: (Score:2)
Re: (Score:2)
If you really want to learn about _working_ AI and not "when I was a boy we did it in the snow, both ways, uphill" then do
https://class.coursera.org/ml/class [coursera.org]
Machine Learning by Andrew Ng.
After that you can do
http://work.caltech.edu/telecourse.html [caltech.edu]
Learning from data by Yaser Abu-Mostafa
Half of Hintons course was about history and what didnt work in AI. Its great to know those things if you have interest in the field, but its not something you should start with (snorefest).
Re: (Score:2)
There's also a great tutorial by Andrew Ng's group at:
http://deeplearning.stanford.edu/wiki/index.php/UFLDL_Tutorial [stanford.edu]
There are two types of deep learning currently by the way:
- restricted Boltzmann machines (RBM)
- sparse auto-encoders
Google / Andrew Ng use sparse auto-encoders. Hinton uses (created) deep RBM networks. They both work in a similar way: each layer learns to reconstruct the input, using a low-dimensional representation. In this way, lower layers build up for example line detectors, and higher
The stank of (poorly) attempted hype (Score:2)
Re:The stank of (poorly) attempted hype (Score:5, Interesting)
For such a blatant, transparent, promotional, hyperbolic "story", I wish soulskill would at least throw in a sarcastic jab or two to balance out the stench a bit.
Agreed. This story smells of the usual Google hype.
I think it's great that there is more research in this area, but "The Man Behind the Google Brain: Andrew Ng and the Quest for the New AI" suggests that Google is at the forefront of this stuff. They're not. Look at the Successes in Pattern Recognition Contests since 2009 [wikipedia.org]. None of Ng, Stanford, Google or Silicon Valley are even mentioned. Google's greatest ability is in generating hype. It seems to be the stock-in-trade of much of Silicon Valley. Don't take it too seriously.
Generating this type of hype for your company is an art. I use to work for a small company run by a guy who was a wiz at it. What you have to understand is that reporters are always looking for stories, and this sort of spoon fed stuff is easy to write. Forget about "Wired". The guy I knew could usually get an article in NYT or WSJ in a day or two.
Re: (Score:2)
These new advances in deep learning are more recent than 2009.
That section is about successes since 2009 - it includes stuff up through 2012.
Check out Ng's paper where a deep learning algorithm spontaneously learned to detect cats by watching a bunch of youtube videos. It's recent and it's a big step forward.
Maybe it is, and maybe it's just better publicized than other recent accomplishments. How would I know? Do you know enough about this field to really say?
I'm not criticizing Ng. Maybe he's a great teacher, a genius, and plays a great backhand. From my very cursory search he doesn't seem like he's one of the big names in this field, like the article claims. Maybe I'm wrong. Either way this article stinks of the usual Google (and
Its not winning the Hutter Prize (Score:4, Informative)
The Hutter Prize for Lossless Compression of Human Knowledge [hutter1.net]
The last time anyone improved on that benchmark was 2009.
Re: (Score:2)
I may be misinterpreting or missing the intent here, but humans are demonstrably NOT lossless storage mediums.
HUGE amounts of data are lost, simply between your eyeballs and your visual cortex. That's kinda the point that the summary makes about neural net based vision systems. They take a raw flood of data, and pick it apart into contextually useful elements of artificial origin, which then get strung together to build a high level experience.
Lossless storange of human knowledge is strictly speaking, a com
Universal Artificial Intelligence (Score:3)
If the goal is to pass the Turing Test, that is one thing. But clearly they are trying for something more general in some of their contests. I'm just informing them (assuming they are watching) that better tests are available.
Re: (Score:2)
You're missing the point. Lossless text compression basically comes down to probability estimates - a direct analog for artificial intelligence. Your estimates don't have to be perfect, but the closer they get the better your compression ratio will be.
Re: (Score:3)
Its been my experience that (at least the implementation running on my meaty hardware) human intelligence actually makes more use of data-deduplication and semantic cross linking to cull as much information as possible, while leaving metadata cues to reconstruct the data on the fly.
This is why people misremember things, and why odd and spurrious sensory stimuli can cause a memory to surface.
Humans dont store data losslessly, and make lossy heuristic decisions instead.
the kind of intelligence you are referri
Re: (Score:2)
From the task description:
"Restrictions: Must run in 10 hours on a 2GHz P4 with 1GB RAM and 10GB free HD"
So, even if you could write an algorithm that fits in a couple of meg, and magically generates awesome feature extraction capabilities, which is kind of what deep learning can do, you'd be excluded from using it in the Hutter prize competition.
For comparison, the Google / Andrew Ng experiment where they got a computer to learn to recognize cats all by itself used a cluster of 16,000 cores (1000 nodes * 1
Some questions for Andrew Ng (Score:5, Insightful)
Andrew Ng is a brilliant teacher who I respect, but I have questions:
1) What is the constructive definition of intelligence? As in, "it's composed of these pieces connected this way" such that the pieces themselves can be further described. Sort of like describing a car as "wheels, body, frame, motor", each of which can be further described. (The Turing Test doesn't count, as it's not constructive.)
2) There are over 180 different types of artificial neurons. Which are you using, and what reasoning implies that your choice is correct and all the others are not?
3) Neural nets in the brain have more back-propagation connections than forward. Do your neural nets have this feature? If not, why not?
4) Neural nets typically have input-layers, hidden-layers, output layers - and indeed, the image in the article implies this architecture. What line of reasoning indicates the correct number of layers to use, and the correct number of nodes to use in each layer? Does this method of reasoning eliminate other choices?
5) Your neural nets have an implicit ordering of input => hidden => output, while the brain has both input and output on one side (ie - both the afferent and efferent neuron enter the brain at the same level, and are both processed in a tree-like fashion). How do you account for this discrepancy? What was the logical argument that led you to depart from the brain's chosen architecture?
Artificial intelligence is 50 years away, and it's been that way for the last 50 years. No one can do proper research or development until there is a constructive definition of what intelligence actually is. Start there, and the rest will fall into place.
Re:Some questions for Andrew Ng (Score:5, Interesting)
I'd mod you up if I could, but I think I can help out with a few points instead:
1) There is no concrete constructive definition of intelligence yet, and I think anybody at a higher level in the field knows that. Establishing that definition is a recognized part of AI research. Intelligence is still recognized comparatively, usually related to something like the capability to resolve difficult or ambiguous problems with similar or greater effect than humans, or can learn and react to dynamic environmental situations to similar effect as other living things. Once we've created something that works and that we can tangibly study, we can begin to come up with real workable definitions of intelligence that represent both the technological and biological instances of recognized intelligence.
4) Modern ANN research sometimes includes altering the morphology of the network as part of training, not just altering the coefficients. I would hope something like that is in effect here.
Re: (Score:2)
Intelligence is still recognized comparatively, usually related to something like the capability to resolve difficult or ambiguous problems with similar or greater effect than humans, or can learn and react to dynamic environmental situations to similar effect as other living things.
The second part is an important milestone for me. Take the Big Dog - a marvel of robotics in itself. If it interacted with environment and its operator on a level that real dogs interact with environment and their masters, we would have a real breakthrough. An essential thing would be to make it learn new tricks, like a dog learns, instead of programming them in.
Re: (Score:2)
If you look again it's really the first part (conceptualization & learning (mammalian-level)) that you wish to add to the second part (dynamic responses to environment (insectoid-level)) to complete the whole picture.
Getting just the former to work yields brains in a jar like the Star Trek ship computer. Getting just the latter to work gets you semi-autonomous "dumb but capable" things like Big Dog. These are the two main facets of we view as intelligence, and to get "real" AI or artificial life does
Re:Some questions for Andrew Ng (Score:5, Insightful)
That's a fool's errand. The goal of the developer should be to build a system that accomplishes tasks and is able to auto-improve the speed of accomplishing repetitive tasks with minimal (no) human intervention.
The goal of the philosopher is to lay out what intelligence "is". These tracks should be run in parallel and the progress of one should have little-to-no impact on the progress of the other.
-CF
Some questions for you (Score:2, Interesting)
That's a fool's errand. The goal of the developer should be to build a system that accomplishes tasks and is able to auto-improve the speed of accomplishing repetitive tasks with minimal (no) human intervention.
The goal of the philosopher is to lay out what intelligence "is". These tracks should be run in parallel and the progress of one should have little-to-no impact on the progress of the other.
Do you consider proper definitions necessary for the advancement of mathematics?
Take, for example, the [mathematics] definition of "group". It's a constructive definition, composed of parts which can be further described by *their* parts. Knowing the definition of a group, I can test if something is a group, I can construct a group from basic elements, and I can modify a non-group so that it becomes a group. I can use a group as a basis to construct objects of more general interest.
Are you suggesting that m
Re: (Score:3, Insightful)
Do you consider proper definitions necessary for the advancement of mathematics?
Take, for example, the [mathematics] definition of "group". It's a constructive definition, composed of parts which can be further described by *their* parts. Knowing the definition of a group, I can test if something is a group, I can construct a group from basic elements, and I can modify a non-group so that it becomes a group. I can use a group as a basis to construct objects of more general interest.
Are you suggesting that mathematics should proceed and be developed... without proper definitions?
That a science - any science - can proceed without such a firm basis is an interesting position. Should other areas of science be developed without proper definitions? How about psychology (no proper definition of clinical ailments)? Medicine? Physics?
I'd be interested to hear your views on other sciences. Or if not, why then is AI is different from other sciences?
The view of mathematics as proceeding from clear-cut definitions and axioms is really an artifact of the way we teach it. Over time theorems can become definitions, and we may choose definitions so as to make certain theorems that ought to be true, true.
If you want an example, look at how much real analysis was going on before we had a proper definition of continuity.
An obsession with rigorous definitions right at the start of a field serves only to force our intuitions to be more specific than they are, wi
Re: (Score:2, Insightful)
Mathematics is not a science, it's just used by science. Science is about studying phenomena in reality. Mathematics is not part of reality - mathematics is entirely apriori. You can't prove anything in mathematics by studying reality. Mathematics is made entirely of definitions and symbol manipulation, so of course you shouldn't be doing mathematics without definitions. Science isn't like that. Proper definitions can often help in Science, that's true, but it's not a prerequisite. The suggestions is not th
Yes--But the Trend is Toward Biological Realism (Score:5, Informative)
Neural Net's were traditionally based off old Hodgkins and Huxley models and then twisted for direct application for specific objectives, such as stock market prediction. In the process they veered from a only very vague notion of real neurons to something increasingly fictitious.
Hopefully, the AI world is on the edge of moving away from continuously beating their heads against the same brick walls in the same ways while giving themselves pats on the heads. Hopefully, we realize that human-like intelligence is not a logic engine and that conventional neural nets are not biologically valid and posses numerous fundamental flaws.
Rather--a neurons draws new correlating axons to itself when it cannot reach threshold (-55mv from a resting state of -70mv) and weakens and destroys them when over threshold. In living systems, neural potential is almost always very close to threshold--it bounces a tiny bit over and under. Furthermore, inhibitory connections are also drawn in from non-correlating axons. For example, if two neural pathways always excite when the other does not, then each will come to inhibit the other. This enables contexts to shut off irrelevant possible perceptions, e.g. If you are in the house, you are not going to get rained on. More likely, somebody is squirting you with a squirt gun.
Also--a neuron perpetually excited for too long shuts itself off for a while. We love a good song but hearing it too often makes us sick of it, at least for a while.. like Michael Jackson in the late 1980's.
And very importantly--signal streams that dissappear but recur after increasing time lapses stay potentiated longer.. their potentiation dissipates slower. After 5 pulses with a pause between a new receptor is brought in from the same axon as an existing one. This causes slower dissipation. It will happen again after another 5 pulses repeatedly, except that the time lapse between them must be increased. It falls in line with the scale found on the Wikipedia page for Graduated Interval Recall--exponentially increasing time lapses 5 times, each... take a look at it. Do the math. It matches what is seen in biology, even though this scale was developed in the 1920's.
I have a C++ neural modal that does this. I am mostly done also with a Javascript modal (employing techniques for vastly better performance), using Nodejs.
Good points (Score:3, Interesting)
You highlight important points, of which AI researchers should take note.
We don't know what intelligence actually is, but we have an example of something that is unarguably intelligent: the mammalian brain. Any proposed mechanism of intelligence should be discounted unless it behaves the same way as a brain. Most AI research fails this test.
I personally think in-depth modeling of individual neurons is too deep of a level - it's like trying to make a CPU by modeling transistors. We might be better off using
Re: (Score:2)
Any proposed mechanism of intelligence should be discounted unless it behaves the same way as a brain.
This doesn't make any sense unless you have a purely tautological definition in mind.
Re: (Score:2)
We don't know what intelligence actually is, but we have an example of something that is unarguably intelligent: the mammalian brain.
To further this point, a brain is also not necessarily just a pile of neurons. There is specialization in the brain, and while other portions of the brain can make up for damaged parts, that only goes so far.
Any proposed mechanism of intelligence should be discounted unless it behaves the same way as a brain. Most AI research fails this test.
I would offer some more subjectivity to that. Mammalian brains tire, need sleep, do not have perfect recall, run things out of time order and convinces itself otherwise, take a long time to train, and has strong emotional needs.
With a view of "intelligence" that would include tools and helpers, and no
Re: (Score:2)
Who has ruled out that these are preconditions?
CC.
Sure.. but.. (Score:2)
Modern neural science and biologically realistic neural simulations (such as some of the best Deep Learning systems) use the neuron as its most fundamental primitive. One neural does a lot, actually. It draws in new correlating axons (those firing when its other receptors are firing) when its total potentiation is insufficient to excite. It weakens and destroys them in the inverse case. It also draws in non-correlating axons as inhibitory receptors. And long term potentiation (widely viewed as the basi
Re: (Score:2)
"Asking whether a computer can be intelligent is like asking whether a submarine can swim".
An airplane doesn't flap its wings, but flies faster than birds can.
Submarines don't swim, but they move through the water faster than dolphins.
Not everything has to copy nature exactly in order to be effective.
Re: (Score:2)
I'm still mystified by the desire to make computational neural nets more like biological ones. Biological neurons are *bad* in many ways -- for one, they are composed of a large number of high signal-to-noise sensors (ion channels). This random behavior is necessary to conserve energy and space in a brain. But computers have random-access memory and energy isn't really a limiting factor; why impose these flaws?
Sure, there may be things that can be discovered by playing with network models more inspired by b
Re: (Score:2)
Re:Yes--But the Trend is Toward Biological Realism (Score:5, Insightful)
I could give a number of clearly unsubstantiated, but seemingly reasonable answers here.
1) the assertion that because living neurons have deficits compared against an arbitrary and artificial standard of efficiency (it takes a whole 500ms for a neuron to cycle?! My diamond based crystal oscillator can drive 3 orders of magnitude faster!, et al.)that they are "faulted" is not substantiated: as pointed out earlier in the thread, no high level intelligence built using said "superior" crystal oscillators exists. Thus the "superior" offering is actually the inferior offering when researching an emergent phenomenon.
2) artificially excluding these principles (signal crosstalk, propogation delays, potentiation thresholds of organic systems, et al) completely *IGNORES* scientifically verified features of complex cognitative behaviors, like the role of mylein, and the mechanisms behind dentrite migration/culling.
In other words, asserting something foolish like "organic neurons are bulky, slow, and have a host of computationally costly habbits" wit the intent that "this makes them undesirable as a model for emergent high level intelligence" ignores a lot of verified information in biology, that shows that these "bad" behaviors directly contribute to intelligent behaviors.
Did you know that signal DELAY is essential in organic brains? That whole hosts of disorders with debilitating effects come from signals arriving too early? Did you stop to consider that thse faults may actually be features that are essential?
If you don't accurately model the biological reference sample, how can you riggorously identify which is which?
We have a sample implementation, with features we find dubious. Only buy building a faithful simulation that works, then experimentally removing the modeled faults do we really systematically break down the real requirements for self directed intelligences.
That is why modeling accurate neurons that faithfully smulate organic behavior is called for, and desirable. At least for now.
Re: (Score:2)
"Did you know that signal DELAY is essential in organic brains? That whole hosts of disorders with debilitating effects come from signals arriving too early? Did you stop to consider that thse faults may actually be features that are essential?"
Are you saying that our maker created a system with severe race condition problems? I guess that is another issue to add to the existing; inclusion of obsolete and potentially dangerous features (the appendix), only poor and limited third party replacement parts avai
Re: (Score:2)
This is really interesting. How well does your code perform?
Peer reviews are overrated (Score:2)
Let us know when you have a peer reviewed publication on your "new" system. Untill then, you can stfu.
Let us know when a peer-reviewed publication tells us how to construct an intelligence.
When will that be - another 50 years, perhaps?
Really. Are you saying that, after AI has gone nowhere for the last 50 years that his position is completely without merit?
At the very least, you should entertain the possibility that the emperor does, in fact, have no clothes.
Re: (Score:3)
Of course, if everyone would just stfu until they have a peer reviewed journal article, there would never be any peer reviewed journal articles... Perhaps one reason AI hasn't progressed might be this kind of brutal cynicism toward new ideas.
Granted, every premise I provided in the modal derives from established science, though replicated, peer review journals.. in fact, much through basic text books in Neural Science... But let's stfu about that, too, since these things don't appear to be yet discussed a
Re: (Score:2)
At one time, slashdot was actually a mostly intellectually stimulating conversational environment...
And then someone came along and claimed that a javascript rewrite of their C++ code resulted in "vastly better performance", causing slashdot to implode.
Neural networks revisited (Score:5, Informative)
Neural networks are certainly not new, or groundbreaking. We already know their strengths and weaknesses, and they aren't a universal solution to every AI problem.
First of all, while they have been inspired by the brain, they don't "mimic" it. Neural networks are based on some neurons having negative weights, reversing the polarity of the signal, which doesn't happen in the brain. They are also linear, which bears similarities to some simple parts of the brain, but are very far from modeling its complex nonlinear processing. Neural networks are useful AI tools, but aren't brain models.
Second neural networks are only good at things when they have to immediately react to an input. Originally, neural networks didn't have memory, and while it's possible to add it, it doesn't fit right into the system and is hard to work with. While neural networks make good reflex machines, even simple stateful tasks like a linear or cyclic multi-step motion are nontrivial to implement in them. Which is why they are most effective in combination with other methods, instead of declared a universal solution.
Re: (Score:2)
First of all, while they have been inspired by the brain, they don't "mimic" it.
That is true.
Neural networks are based on some neurons having negative weights, reversing the polarity of the signal, which doesn't happen in the brain.
There are in fact inhibitory connections in the brain.
They are also linear
That is false. The only way an ANN could be linear is if each "neuron" used a squashing function f(x) = x. Then they'd just be doing linear algebra, namely change-of-basis computations. But no one uses that. Even the super-simple heaviside squash used in the 1950s perceptrons made them do nonlinear computations.
Second neural networks are only good at things when they have to immediately react to an input. Originally, neural networks didn't have memory, and while it's possible to add it, it doesn't fit right into the system and is hard to work with. While neural networks make good reflex machines, even simple stateful tasks like a linear or cyclic multi-step motion are nontrivial to implement in them.
That is also false. I suspect there are limits to what kind of stateful computations you can do with an ANN, but you can certai
Re: (Score:2)
What's actually new here? (Score:3)
What's actually new in the neural net business? That's a real question - not a sarcastic or rhetorical one.
Artificial neural nets were suggested and tried for AI at least 50 years ago. They were bashed by the old Minsky/McCarthy AI crowd, who didn't like the competition's idea (always better to write another million lines of Lisp). They wrote a paper that showed neural nets couldn't implement an XOR. That's true - for a 2 layer net. A 3 layer net does it just fine. Nevertheless M&M had enough clout to put bury NN research for years. Then in the 80's(?) they became a hot new thing again. One of the few good things about getting older is that you can remember hearing the same hype before.
However, I'm not saying there hasn't been progress. Sometimes a field needs to go through decades of incremental improvement before you can get decent non-trivial applications. It's not all giant breakthroughs. Sometimes just having faster hardware can make a dramatic difference. Loads of things that weren't practical became practical with better hardware. So what's really improved w/ neural nets these days?
Re: (Score:2)
The improvement has been using multiple ANNs together as communicating units that can both communicate information and dynamically train each other, instead of trying to make a single large ANN and using external training sets. Of course, this isn't that new, as these guys [imagination-engines.com] have been working on such models since at least the early 90s.
Re: (Score:2)
What's actually new in the neural net business? That's a real question - not a sarcastic or rhetorical one.
What's reportedly new is the ability to train feed-forward networks with many layers. They have never trained well with backpropagation because the backpropagated error estimate becomes "diluted" the further back it goes, and as a result most of the training happens to the weights closest to the output end.
The notion that the first hidden layer is a low-level feature detector and each successive layer is a higher-level feature layer is ancient lore in the ANN research community. The claims of the Deep Lea
Re: (Score:2)
http://en.wikipedia.org/wiki/Deep_learning [wikipedia.org] http://en.wikipedia.org/wiki/Restricted_Boltzmann_machine [wikipedia.org] http://en.wikipedia.org/wiki/Long_short_term_memory [wikipedia.org]
Re: (Score:3)
Re: (Score:2)
At least, sometimes, the hype spirals in a promising direction :)
CC.
Re: (Score:2)
Thanks to everyone above for their thoughtful and intelligent answers to my "what's new about it" question. Definitely some interesting reading.
P.S. Due to the aforementioned behavior, you're now all banned from Slashdot :)
Touring Test (Score:3)
"I like cats. Where can I pick one up?"
Let me know when AI can understand the difference between the preceding sentences.
-CF
Re: (Score:2)
A child will fail this test. A person who is not familiar with slang will fail this test. But they both are intelligent. That's the problem with the TT - it's testing for a characteristic that we cannot define, much like one of US judges [wikipedia.org], who proclaimed that "hard-core pornography" was hard to define, but that "I know it when I see it."
Similarly, a TT cannot be conducted if the parties don't speak the same language, or don't share the same culture, or just are of different genders. How would you think a
No (Score:3)
Recursion (Score:2)
I wonder when our new AI overlords will create AI themselves because they are too bored and tired of doing actual work themselves.
peopel doing this 40 years ago (Score:2)
I dont htink the problems of "brittleness"- unpredictable result if new inputs are presented- and "opaqueness"- weights are interpretable- have changed.
Re: (Score:3)
Neural networks? Is it news?
No, it's misrepresentation. This isn't any more akin to neuroscience than any of the other techniques used with artificial neural networks.
It will be a great thing, though, if it lives up to expectations.
Re: (Score:3)
I post as an anonymous coward so as not to harm my career (any further) by stating *truth* which is not politically nor academically correct.
Also, you wisely don't want your name associated with a positive view of Searle's nonsense.
Re: (Score:2, Troll)
Searle's nonsense, eh? There's a reason that he's Slusser professor of philosophy at U.C. Berkeley and why his work on AI stands strong today -- even after 30 years of constant assault.
No one is laughing at Searle except those who feel threatened by the problem he presents. If he were a just another easy-to-dismiss nut, we wouldn't still be talking about his 1980 paper (and related subsequent papers) today. Nor would he wouldn't hold such an esteemed position at one of the worlds finest institutions.
If y
Re: (Score:2)
Are you a follower of Ray Kurzweil, by any chance? I only ask because I don't often see Searle dismissed outright by anyone competent unless they also happen to be a singularity nut.
No, I think Kurzweil is a crank.
Re: (Score:2)
Why do we pour money and resources in building AI when we have so many people with under-utilized brains already?
Expected return.