A.I. Advances Through Deep Learning 162
An anonymous reader sends this excerpt from the NY Times:
"Advances in an artificial intelligence technology that can recognize patterns offer the possibility of machines that perform human activities like seeing, listening and thinking. ... But what is new in recent months is the growing speed and accuracy of deep-learning programs, often called artificial neural networks or just 'neural nets' for their resemblance to the neural connections in the brain. 'There has been a number of stunning new results with deep-learning methods,' said Yann LeCun, a computer scientist at New York University who did pioneering research in handwriting recognition at Bell Laboratories. 'The kind of jump we are seeing in the accuracy of these systems is very rare indeed.' Artificial intelligence researchers are acutely aware of the dangers of being overly optimistic. ... But recent achievements have impressed a wide spectrum of computer experts. In October, for example, a team of graduate students studying with the University of Toronto computer scientist Geoffrey E. Hinton won the top prize in a contest sponsored by Merck to design software to help find molecules that might lead to new drugs. From a data set describing the chemical structure of 15 different molecules, they used deep-learning software to determine which molecule was most likely to be an effective drug agent."
Re:Deep learning? (Score:4, Interesting)
It looks like you are seeing something that is not there. The majority of neural network research is about developing new and/or improved algorithms to solve problems, not to say anything about how the human brain works.
As someone who has read a lot of the founding literature of modern cognitive science and the philosophy of mind in the 1950s through 80s, which was hugely influential in setting up the early approaches to AI (including neural nets), I have to say -- this is where the stuff came from.
And frankly, a lot of applications in more obscure disciplines, such as in AI analysis in the humanities, researchers are still making claims about these models and their relationships to the actual brain. Hell, just a few years ago I heard a leading cognitive scientist claim that he found evidence for a sort of musical "circle of fifths" neural network in an actual circular physical structure of neurons in the brain... a made-up musical model grafted onto a made-up AI brain model, supported by noisy data... I admit this is an extreme example, but it's not unique.
I understand that modern researchers in "pure" AI may want to avoid recognizing the history or the implications of the terminology -- but there's a reason why the Starship Voyager was equipped with "neural gel-packs" that could get anxious and cause a warp-core breach at a temporal anomaly... words like "neural" actually mean something, and these "neural nets" have about as much connection to the biological function of actual neurons as Voyager's bizarre "neural gel-packs." Yet the implicit metaphor made in continuing to use the term should not be underestimated, not just in a general audience NYT article, but in the way fields are subtly shaped by their nomenclature.
It's both (Score:5, Interesting)
In the past few years, a few things happened almost simultaneously:
1. New algorithms were invented for training of what previously was considered nearly impossible to train (biologically inspired recurrent neural networks, large, multilayer networks with tons of parameters, sigmoid belief networks, very large stacked restricted Boltzmann machines, etc).
2. Unlike before, there's now a resurgence of _probabilistic_ neural nets and unsupervised, energy-based models. This means you can have a very large multilayer net (not unlike e.g. visual cortex) figure out the features it needs to use _all on its own_, and then apply discriminative learning on top of those features. This is how Google recognized cats in Youtube videos.
3. Scientists have learned new ways to apply GPUs and large clusters of conventional computers. By "large" here I mean tens of thousands of cores, and week-long training cycles (during which some of the machines will die, without killing the training procedure).
4. These new methods do not require as much data as the old, and have far greater expressive power. Unsurprisingly, they are also, as a rule, far more complex and computationally intensive, especially during training.
As a result of this, HUGE gains were made in such "difficult" areas as object recognition in images, speech recognition, handwritten text (not just digits!) recognition, and in many more. And so far, there's no slowdown in sight. Some of these advances were made in the last month or two, BTW, so we're speaking about very recent events.
That said, a lot of challenges remain. Even today's large nets don't have the expressive power of even a small fraction of the brain, and moreover, the training at "brain" scale would be prohibitively expensive, and it's not even clear if it would work in the end. That said, neural nets (and DBNs) are again an area of very active research right now, with some brilliant minds trying to find answers to the fundamental questions.
If this momentum is maintained, and challenges are overcome, we could see machines getting A LOT smarter than they are today, surpassing human accuracy on a lot more of the tasks. They already do handwritten digit recognition and facial recognition better than humans.
Re:Sources of improvements? (Score:3, Interesting)
On the other hand there have been some theoretical advances by Hinton and others where networks can be trained on unsupervised data (e.g. the Google cats thing).
Old News (Score:2, Interesting)
Re:Sources of improvements? (Score:2, Interesting)
The way they are trained is very different, and it's this change that improves the performance. It's more than just making them faster, a fast idiot is still an idiot.
Yes, but... no. (Score:5, Interesting)
This is a very misleading metric. First, some not-insignificant number of the neurons in the brain are involved in non-cognitive computations. Muscle control, hormone regulation, kinesthesia, vision (not thinking about what is seen, but simply recognizing it), heart rates and other system regulation and so on.
Examples also exist [fyngyrz.com] of low-neuron (and synapse) count individuals who retain cognitive (and all other major) function; these examples cannot be explained away by "counting neurons."
We don't know which yet, but given that high neuron count has been ruled out as the single way to accommodate intelligence, we do know that we need to look to other mechanisms for human cognition. Structure, algorithm, other features known or unknown may be responsible for intelligence; and it may be that something entirely disjoint is responsible for the rise of intelligence; but we know it isn't simply high neuron count.
--fyngyrz (anon due to mod points)