Google Brain's Co-inventor Tells Why He's Building Chinese Neural Networks 33
An anonymous reader writes "Here's an interview with Andrew Ng, former leader of Google Brain, discussing Baidu, Deep Learning, computer neural networks, and AI. An interesting excerpt from the interview on biological vs. computer neural networks: "A single 'neuron' in a neural network is an incredibly simple mathematical function that captures a minuscule fraction of the complexity of a biological neuron. So to say neural networks mimic the brain, that is true at the level of loose inspiration, but really artificial neural networks are nothing like what the biological brain does."
Re: (Score:3)
Re: (Score:1)
Having crossed paths with this fella about 10 years ago, you're not too far off. Minus the racism.
I did not realize that Chinese was a race. I thought it was a nationality. Is this like how people who question Islam are called racists? Islam is also not a race but a religion and potentially an ideology. Should people oppose communism be called racists too?
Han. (Score:2)
Chinese-as-race is probably referring to the Han Chinese ethnicity [wikipedia.org].
Re: (Score:1)
He's just doing what everyone else who succeeds in this business does. Downloading libraries to recreate popular results and doing better demos. You might as well accuse 90% of PhD students of doing the same thing. In fairness, you probably are.
Man, we get jaded in this business, don't we?
On the other hand ... (Score:5, Funny)
artificial neural networks are nothing like what the biological brain does.
... there are quite a few people around who tried to overclock their brains during the '60s and '70s.
Re: (Score:2)
More like stripped the insulation off the wiring ...
Which leads me to a question. If the 'neuron' of machine learning is so very different from a biological neuron, why are people insisting on calling it a 'neuron'. Sounds more like a 'synapse' that just takes an input and does some fairly simple manipulation of the signal. Is there some deeper analogy that isn't obvious? A car analogy perhaps?
Re: (Score:3)
Re: (Score:1)
Re:On the other hand ... (Score:5, Insightful)
If the 'neuron' of machine learning is so very different from a biological neuron, why are people insisting on calling it a 'neuron'.
You could say the same about the 'wing' of an airplane and a biological wing. The wing of a hummingbird or mosquito is vastly more complicated and capable than the wing of a 747. It provides thrust as well as lift, can do a vertical takeoff without a runway, and can go instantly from forward flight to hovering. On the other hand, a hummingbird can't go from SFO to Narita in 8 hours.
Re: (Score:1)
Technically these are called "perceptrons" in the parlance of machine learning. They're basically a really simple linear discriminant. They have an activation threshold and a bunch of connections to different intermediate layers of perceptrons, and that's where the "neuron" comparison comes from, but it's kind of a false equivalence that gets overplayed for the media.
Oh, the media, lol. (Score:2)
Actually, most of them are nonlinear. Sigmoid function is common, and there are much more exotic things going on too, such as fuzzy logic-based discriminants. Bottom line is that any discriminatory function is of interest.
There's also some fascinating stuff going on with time discriminants [numenta.com] where they're having very encouraging results.
Odds are excellent that both (time and transfer function) are part of a solution that is most human-neuron-like. But it
Bran and brain! What is brain? (Score:2)
Re:Brain and brain! What is brain? (Score:1)
Nothing like Biological (Score:3)
To say that "artificial neural networks are nothing like what the biological brain does" is no more correct than to say "artificial neural networks are just like the brain."
Machine learning neural networks do the same flavor of thing that a real organic brain does, but at a complexity that is -many- orders of magnitude smaller. They also tend to be directed at a single skill, and don't have to cohabit the network with, well, everything.
They're not the same, but they're not totally different, either. Truth is not well served by hyperbole.
Re: (Score:1)
Since not everyone can be a Polymath, hyperbole is a lossy compression algorithm for so-called truth. It summarizes the entirety of an individual's personal experience and amplifies the subtlety in to an signed char.
It's essentially Fuzzy Logic vs the one dimension analysis of Good/Evil "black and white" Boolean thinking.
More intelligent people are capable of understanding multidimensional analysis like a radar chart. With enough familiarity the intricacies no longer require amplification to get above the n
Re: (Score:3)
They also tend to be directed at a single skill
Yes, The single skill problem is the main reason neural nets were seen as curious toys for 50ys but I think IBM solved that problem with Watson in the mid-noughties. How? - I'm not sure.
As to biology (Score:2)
In addition to very high complexity, fixed topology (meaning, using primarily electrical, chemical and timing means as opposed to topological modification to operate), general problem solving networks, I am fairly confident that we develop plenty of what can accurately be described as single-skill networks, topologically tuned to individual problems by continuous cut-and-fit until the errors drop. I lay out why right here. [fyngyrz.com]
Re: (Score:2)
The problem with Chinese neural networks (Score:1)
An hour after you turn them one they are hungry for singularity.
Re: (Score:1)
Cey Lon
Biggest difference is timing. (Score:4, Interesting)
Certainly biological neurons are much more complex than artificial neural net neurons. The simplest "Integrate and Fire" (IF) model of a biological neuron perform a leaky integration over *time*, and if the voltage ever reaches the trigger value the fire. So the timing of stimulations is critical, whereas most Artificial Neural Networks (ANNs) does all its calculations (logically) at the same time. The ANN is both simpler and cleaner to work with. Biological synapses are very complex, but much of that complexity just reflects the wet technology that they are made from.
If you want to understand how the brain works, study biological neurons. If you want to understand how to build an intelligent machine, engineer ANNs.
No, dude (Score:1)
That is NOT what a Chinese Room is!
Is this going to be the Chinese Room? (Score:2)
https://en.wikipedia.org/wiki/... [wikipedia.org]