Nobel Prize in Physics Goes To Machine Learning Pioneers Hopfield and Hinton (nobelprize.org) 46
John J. Hopfield of Princeton University and Geoffrey E. Hinton of the University of Toronto were awarded the Nobel Prize in Physics on Tuesday for their groundbreaking work in machine learning. The Royal Swedish Academy of Sciences recognized the scientists for developing artificial neural networks capable of recognizing patterns in large data sets, laying the foundation for modern AI applications like facial recognition and language translation.
Hopfield, 91, created an associative memory system for storing and reconstructing data patterns. Hinton, 76, invented a method for autonomous data property identification. "This year's physics laureates' breakthroughs stand on the foundations of physical science," the Nobel Committee stated. "They have shown a completely new way for us to use computers to tackle many of society's challenges." The laureates will share the 11 million Swedish kronor ($1.1 million) prize.
Hopfield, 91, created an associative memory system for storing and reconstructing data patterns. Hinton, 76, invented a method for autonomous data property identification. "This year's physics laureates' breakthroughs stand on the foundations of physical science," the Nobel Committee stated. "They have shown a completely new way for us to use computers to tackle many of society's challenges." The laureates will share the 11 million Swedish kronor ($1.1 million) prize.
Physics? (Score:5, Insightful)
Re: Physics? (Score:2)
I was thinking the sane thing. But they're trying to stay relevant and be hip. There have been other recent changes that only make sense because of social pressure and not scientific excellence.
I could though defend them a bit in the sense that particle physics is stuck in a bit of a rut. String theory and loop quantum gravity have not gone anywhere. All the test results have been negative. Sabine Hossenfelder has been pointing this out for a while. So they're a bit desperate for something that is a genuine
Re: (Score:2)
The urban legend as to why there's no Nobel Prize for Mathematics is that Nobel's wife had an affair with a mathematician. However, seeing as Alfred Nobel was never married, this legend is extremely Urban...
Re: Physics? (Score:5, Interesting)
That's not it at all. They've gone into great detail as to why it was awarded for physics [nobelprize.org].
TL/DR: Hopman is a physicist. Like, his thesis literally was "A quantum-mechanical theory of the contribution of excitons to the complex dielectric constant of crystals". When he switched to neural networks, he did so from a physics basis: his model was based on energy minimization landscapes. Hinton expanded on that by bringing in statistical physics, originating from the work of Boltzmann - he literally called his initial network a "Boltzmann Machine" [wikipedia.org].:
Today, the field of physics is deeply dependent on the machine learning enabled by Hopman and Hinton to sift through the reems of data in many fields to find meaningful signals - including, for example, finding the Higgs Boson, running gravitational wave detectors, detecting exoplanets, and on and on. The quantities of data processed today in many fields, except in cases where signals can be trivially distinguished from noise by handwritten algorithms, are far too large to rely on just human analysis.
Re: (Score:2)
Or if the TL/DR itself was TL/DR:
Physics gave birth to machine learning, and now heavily relies on it for further discoveries.
Re: (Score:2)
Machine learning had many parents. Neural networks in the 90s was a collaboration across many fields. At least when I was in grad school the local neural nets teams were coming from physics, computer science, biology, neurology (via the medical school), electrical engineering, and cognitive science departments.
Re: (Score:2)
Re: (Score:2)
This is not physics. This a math. The idea may be borrowed from physics but it is math. Physics is math but math is not physics.
Sabine also agrees:
https://www.youtube.com/watch?... [youtube.com]
It looks like since there is no Nobel for Math, the Nobel Committee had to hack the Nobel. They already did this with the "Nobel for Economics".
Re: (Score:2)
Wow, the author of Lost in Math: How Beauty Leads Physics Astray has an opinion about whether this is math or physics? Amazing.
Re: (Score:2)
Computation is physics. If inventors of telescope could be awarded prize, and I think they could, any AI that helps physics research similarly can be awarded prize. The thing is both are physical proxies to gain information about the universe.
Re: (Score:2)
Thanks for the concise summary! (I don't have mod points or I'd have used 'em)
Re: (Score:2)
This. My guess is the secret at the end of the long AI road is that when the human brain is perfectly understood, the shocking truth will be that none of it is special in any way whatsoever. It's just following a set of dynamics that can and do arise all over the physical universe.
The thing that strikes me in studying the attention mechanisms that make the LLMs work, is that there is no satisfying explanation, there is this meta explanation, training the query, key and value matrices separately, it just wor
Re: (Score:2)
One thing Hinton has talked about is that what we're building is not us, but a sort of alien intelligence - something that is very much an "intelligence", and has hallmarks of us, but also differences. For example, it's commonly talked about how our models take more data to learn than we do, as we "mull over" new info (though models kind of can do that by training on synthetic data as well, but this really should be done simultaneous with new data acquistion. Or how we can iterate over tasks (though so ca
Re: (Score:2)
This prize award is a reflection of two things:
1) Computer (or Computing, or Information) Science still can't get no respect in its own right. (Imitation is the sincerest form of flattery I guess.)
2) Physics is a dumpster fire of unverifiable theories these days, due to having reached beyond the size and time bounds of feasible measurability.
I feel that physics will have
Re: (Score:2)
Firstly, neither Hopfield nor Hinton originated the architectural insights which lie at the foundation of neural networks. If the goal was to reward important foundational work, then the Nobel committee should have rewarded the inventors of the perceptron [wikipedia.org] while they were still alive.
If the "physics connection" through ensembles was the goal, then there ar
Re: (Score:2)
Hinton was awarded the Turing Award in 2018, which is "The Nobel Prize of Computer Science".
Re: (Score:2)
The Nobel Prize categories are physics, chemistry, physiology or medicine, literature and peace, as laid out by Alfred Nobel himself, plus the economics prize added in his memory. Short of going against Alfred Nobel's will and adding new categories, it seems sensible of the Nobel Committee to be somewhat flexible when there are eminently deserving nominees whose work doesn't quite fit into any of the existing ones. Hopfield and Hinton's work may not be traditional physics, but it is rooted in statistical ph
Re: (Score:2)
Short of going against Alfred Nobel's will and adding new categories, it seems sensible of the Nobel Committee to be somewhat flexible when there are eminently deserving nominees whose work doesn't quite fit into any of the existing ones.
Nobel wanted to give an award specifically for physics, so torturing the definition of physics to give awards to those in other fields isn't going against his wishes? There already well respected awards for computer science and math, why does the Nobel committee need to include those categories into physics as well?
Re: (Score:2)
If it wasn't for Physics, then Hinton and Hopfield would not have gone from Physic to Neural Networks, as they used lots of physics ideas in it.
Re: (Score:3)
Ellen Moons, a member of the Nobel committee at the Royal Swedish Academy of Sciences, said the two laureates "used fundamental concepts from statistical physics to design artificial neural networks that function as associative memories and find patterns in large data sets."
from https://phys.org/news/2024-10-... [phys.org]
An interesting article regardless of what you think.
Physics is Measurement (Score:4, Interesting)
It's strange to me that this invention has been categorized as "physics", when it sounds like it has much more to do with mathematics or computer science.
I'm of the opinion that AI is a subset of physics and not mathematics.
AI is fundamentally finding patterns within noisy measurements and learning to make predictions.
The relevant difference here is that AI is based on *measurements*, while mathematics is pure and not tied to anything physical. One could look at non-Euclidean geometry as a good example: space could be curved like a sphere, or space could be curved like a saddle, or space could be not curved (flat). All three lead to interesting results, only one will be correct in our actual universe.
Real world measurements have to deal with noise and other measurement artifacts, and as a researcher you need to keep that clearly in mind. As an example, NeuraLink data is scaled so that the data comes out with maximum dynamic range within the WAV format. All well and good, except that WAV samples are integers, and if you multiply your sample measurements by, say, 10.3 to get maximum range you introduce quantization noise in your data that didn't originally exist. The result integers have to be rounded up or down when converted to integer.
In pure math this isn't a problem because you can multiply your inputs by anything because you're using real numbers.
ChatGPT has to deal with misspellings and bad grammer, and still figure out what the user actually meant. Such as you just did with my previous sentance, when I misspelled "grammar". Or "sentence".
There's some actual math involved in AI as well, but there's actual math in Chemistry and Biology. The actual math is fairly simple in current models, although the results are generally hard to predict, and the research seems to be more experimental than deductive. People set up new situations and "try it" to see if that works, if that works better than current solutions, and so on. There are metrics and current standards to test on (recognition accuracy of postal zip codes, for example).
For example, the learning rate (scale factor for back propagated errors) for ANN's has to be "tuned" to a proper value to avoid well known problems. That's more of a "we do this because it works" thing rather than a "we can predict this is the correct value to use" thing.
Because AI inherently comes from measurements of the physical world, and because of its experimental nature, I'm of the opinion that AI should be considered physics and not math.
Re: (Score:2)
AI is more like statistical mechanics which is ... well let's just say it started as very clearly not physics and the deeper we dig the more it seems that it is basically the entirety of physics.
Re: (Score:2)
Nobel hates mathematician. Which is why there's a Fields Award just for mathematician.
Re: (Score:1)
When the price was invented there was no computer science to speak of as a branch. Physics is applied mathematics, the Nobel prize is intended for practical inventions and progress.
Re: (Score:2)
Yes, this confused me too. Turing Prize, sure, as his work is now standard computer science and electrical engineering. This is essentially simulated biology using math. It's great work deserving of recognition, but... physics?
Though I can see some who claim that physics is merely applied mathematics, and everything else in science is merely applied physics...
Can't wait for an AI to win (Score:2)
Physics (Score:3)
Re: (Score:3)
Re: (Score:3)
"Hopfield Net" (Score:3)
hah! that's the first network we studied in AI class waaaay back in the 90's. "Does it think or does it stink?" That was our mantra while developing.
Training back then took absolute ages though and didn't produce results that were especially useful. Back then, computer IO and peripherals were still pretty limited. OCR was about the most useful application at the time. Certainly nothing with audio.
Of course one of the bigger challenges with developing any neural network is to come up with an efficient, effective way to introduce data into the input layer. We just didn't have the computer power and memory needed for the broad input layers we can use today.
But that's way in my past now, I'll gladly let the new generation of geeks take on the new challenges. I assume they've found better structures than the Hopfield Net to process broad input sets!
91 years old ? (Score:1)
91 years old ? And in a field that's sort of bleeding edge?
Re: (Score:2)
AI is not bleeding edge at all.
Vast majority of foundational concepts in AI - and ML, which used to be not exactly the same thing - are 50 years old.
They simply missed the raw computing power to apply those ideas in meaningful ways.
PS: Not to say that nothing happened in 50 years of course, but the field is way older than people usually recognises.
Re: (Score:3)
AI is not "bleeding edge". Most of what is used today I learned 35 years ago when studying CS. Sure, there have been performance and size improvements and the actual language pre- and post-processing (which is not AI) is impressive. But the actual capabilities of the current models are nothing special. Even "hallucinations" are an old observation.
Re: (Score:2)
"The Nobel committee is trying to remain important"
Have you seen who they've awarded their "peace" prizes to recently? That ship sailed LONG ago.
Re: (Score:2, Informative)
Peace prizes are separate from the science prizes, which is something to which I alluded in my comment.
The peace prizes indeed were rendered meaningless long, long ago, but the science prizes have generally still been worth taking seriously.
Re: (Score:2)
Not sure I agree. They must have had these two on the radar for a while. It would have been better to give them the prize before the current AI craze though.
Hinton seems to be a bit unhinged these days (Score:2)
Probably what happens to an ageing Physicist, many of them start to hallucinate "deep truths". Does not diminish his earlier accomplishments though.
Nobel Prize in Literature (Score:2)
Re: (Score:2)
Of the 120 or so Physics prizes so far (Score:2)
Maybe five were for inventing gizmos (gas valve, fiber optics, CCD, various kinds of microscopes) and the rest were for discoveries about the structure of matter and energy. Even the gizmos were kind of about harnessing new discoveries about the structure of matter and energy, or for better probing the structure of matter and energy.
AI isn't about discovery and testing that insight against reality. It's about regurgitation and hallucination. The opposite of physics. These guys do not belong on the same lis