Scientists Recreate Pink Floyd Song By Reading Brain Signals of Listeners (nytimes.com) 28
An anonymous reader quotes a report from the New York Times: Scientists have trained a computer to analyze the brain activity of someone listening to music and, based only on those neuronal patterns, recreate the song. The research, published on Tuesday, produced a recognizable, if muffled version of Pink Floyd's 1979 song, "Another Brick in the Wall (Part 1)." [...] To collect the data for the study, the researchers recorded from the brains of 29 epilepsy patients at Albany Medical Center in New York State from 2009 to 2015. As part of their epilepsy treatment, the patients had a net of nail-like electrodes implanted in their brains. This created a rare opportunity for the neuroscientists to record from their brain activity while they listened to music. The team chose the Pink Floyd song partly because older patients liked it. "If they said, 'I can't listen to this garbage,'" then the data would have been terrible, Dr. Schalk said. Plus, the song features 41 seconds of lyrics and two-and-a-half minutes of moody instrumentals, a combination that was useful for teasing out how the brain processes words versus melody.
Robert Knight, a neuroscientist at the University of California, Berkeley, and the leader of the team, asked one of his postdoctoral fellows, Ludovic Bellier, to try to use the data set to reconstruct the music "because he was in a band," Dr. Knight said. The lab had already done similar work reconstructing words. By analyzing data from every patient, Dr. Bellier identified what parts of the brain lit up during the song and what frequencies these areas were reacting to. Much like how the resolution of an image depends on its number of pixels, the quality of an audio recording depends on the number of frequencies it can represent. To legibly reconstruct "Another Brick in the Wall," the researchers used 128 frequency bands. That meant training 128 computer models, which collectively brought the song into focus. The researchers then ran the output from four individual brains through the model. The resulting recreations were all recognizably the Pink Floyd song but had noticeable differences. Patient electrode placement probably explains most of the variance, the researchers said, but personal characteristics, like whether a person was a musician, also matter.
The data captured fine-grained patterns from individual clusters of brain cells. But the approach was also limited: Scientists could see brain activity only where doctors had placed electrodes to search for seizures. That's part of why the recreated songs sound like they are being played underwater. [...] The researchers also found a spot in the brain's temporal lobe that reacted when volunteers heard the 16th notes of the song's guitar groove. They proposed that this particular area might be involved in our perception of rhythm. The findings offer a first step toward creating more expressive devices to assist people who can't speak. Over the past few years, scientists have made major breakthroughs in extracting words from the electrical signals produced by the brains of people with muscle paralysis when they attempt to speak.
Robert Knight, a neuroscientist at the University of California, Berkeley, and the leader of the team, asked one of his postdoctoral fellows, Ludovic Bellier, to try to use the data set to reconstruct the music "because he was in a band," Dr. Knight said. The lab had already done similar work reconstructing words. By analyzing data from every patient, Dr. Bellier identified what parts of the brain lit up during the song and what frequencies these areas were reacting to. Much like how the resolution of an image depends on its number of pixels, the quality of an audio recording depends on the number of frequencies it can represent. To legibly reconstruct "Another Brick in the Wall," the researchers used 128 frequency bands. That meant training 128 computer models, which collectively brought the song into focus. The researchers then ran the output from four individual brains through the model. The resulting recreations were all recognizably the Pink Floyd song but had noticeable differences. Patient electrode placement probably explains most of the variance, the researchers said, but personal characteristics, like whether a person was a musician, also matter.
The data captured fine-grained patterns from individual clusters of brain cells. But the approach was also limited: Scientists could see brain activity only where doctors had placed electrodes to search for seizures. That's part of why the recreated songs sound like they are being played underwater. [...] The researchers also found a spot in the brain's temporal lobe that reacted when volunteers heard the 16th notes of the song's guitar groove. They proposed that this particular area might be involved in our perception of rhythm. The findings offer a first step toward creating more expressive devices to assist people who can't speak. Over the past few years, scientists have made major breakthroughs in extracting words from the electrical signals produced by the brains of people with muscle paralysis when they attempt to speak.
HEY! TEACHER! (Score:3, Funny)
Leave them kids alone
If only they could scan Dead Heads' heads (Score:5, Funny)
Re: (Score:1)
Tinfoil Hat Time? (Score:2)
Maybe tinfoil lined hats are not a bad idea after all?
That's all I have, TFA wouldn't load in SeaMonkey.
Part I vs.Part II (Score:5, Informative)
Re: (Score:3)
Missed the opportunity to use “Brain Damage” instead
Re: (Score:2)
Re: (Score:2)
Brain Damage was Eclipsed by The Wall...
slashdot (Score:3)
Scientists recreate goatse by reading brain signals of slashdotters
Re: (Score:2)
I bet it could be 3D if you incorporate their sphincter spasms.
Cue RIAA lawsuits in 3...2...1... (Score:3)
In all seriousness, this is just yet more evidence that the human brain is just a copying device. Rulers have known that since time immemorial. "I'd tell ya, but then I'd have to kill ya" is just an indirect reference to this fact in popular culture. It's long been overdue to cut out the IP crap. As we've made our own brains illegal just to prop up some broken business model.
What could they possibly reconstruct? (Score:2)
Their model constructs a magnitude-only spectrogram with a measly 128 bins (over ~7kHz range) /100ms which they fed to a separate program to reconstruct the audio.
I'm no audiophile, but could you really get anything recognizable out of that? There's hardly anything left!
Re: (Score:2)
Bah. That should be 10ms.
Re: (Score:3)
Rather than reconstructing the music, it seems like it would be easier to just identify the song. That's how Google and apps like Shazam do it - sample a bit of audio, send a few spectrograms over time to a server, and fuzzy match against a database.
Re: (Score:2)
I'm no audiophile, but could you really get anything recognizable out of that? There's hardly anything left!
Gotta start somewhere. The state of the art is like the telectroscope was to modern TVs, a barely functional device when compared to human vision, with little to no practical applications. 150 years later we have cheap flat screen TVs with high resolution. The electrodes only work on large clusters of neurons, many aren’t even monitored, and it completely ignores the chemical signaling altogether but give it 150 years and we may finally have some black mirroresque technology.
Re: (Score:2)
Yeah, i've got serious problems with them calling it 'reconstructed a song'.
It was barely recognizable as a song.
Old news (Score:3, Informative)
Re: (Score:2)
To paraphrase Beyoncé, if you knew it then you should have done a paper 'bout it.
Two questions come to mind (Score:2)
1) What were the researchers smoking?
2) Are they willing to share?
Trainingdata (Score:2)
Obviously the song was in the training set, as stated. So that is hardly an achievement. Had they, however, been able to recreate a song that was outside the training set, at any discernible fidelity I would have been impressed.
But since the neural net already knew what patterns was from what song, it is hardly a great task to recreate that song.
Re: Trainingdata (Score:2)
why are they doing this (Score:1)
only can think of bad things that can happen with this tech. even the few mildly beneficial things are still bad.
Re: (Score:2)
Imagine what would happen if they used the Bob Newhart theme song. It'd be comedic psychological torture.
Pinhead (Score:2)
Does Pinhead from Hellraiser listen to Pink Floyd?
Do you have a licence for that brainwave (Score:1)
Use it to improve cochlear implants (Score:4, Interesting)
We already know it works in the other direction - we can implant electrodes, then send signals to the brain that it can interpret as sound.
The quality is poor though, in part because of using way fewer "electrodes" than natural hearing, but also in part because we just don't know how to truly recreate the natural signals, and the brain just has to learn to use the ones that we do send.
Maybe by studying what comes out of natural hearing brains, we can better learn what to put in with cochlear implants.
Detect people who listen to lyrics? (Score:2)
The lyrics part of the song is interesting, I think, for brain function research.
When I listen to music, I rarely parse the language of the lyrics - they're mostly just more music. I might be able to sing along with most of the chorus and still not really know what the song is about.
My daughter, on the other hand, always listens to the story of the lyrics.
I wonder if there are measurable differences in our brain response to the same song?