Brain Imaging Reveals the Movies In Our Mind 141
wisebabo sends word that scientists from UC Berkeley have developed a method for scanning brain activity and then constructing video clips that represent what took place in a person's visual cortex (abstract). The technology is obviously quite limited, and "decades" away from any kind of sci-fi-esque thought reading, but it's impressive nonetheless. From the news release:
"[Subjects] watched two separate sets of Hollywood movie trailers, while fMRI was used to measure blood flow through the visual cortex, the part of the brain that processes visual information. On the computer, the brain was divided into small, three-dimensional cubes known as volumetric pixels, or 'voxels.' ... The brain activity recorded while subjects viewed the first set of clips was fed into a computer program that learned, second by second, to associate visual patterns in the movie with the corresponding brain activity. Brain activity evoked by the second set of clips was used to test the movie reconstruction algorithm. This was done by feeding 18 million seconds of random YouTube videos into the computer program so that it could predict the brain activity that each film clip would most likely evoke in each subject. Finally, the 100 clips that the computer program decided were most similar to the clip that the subject had probably seen were merged to produce a blurry yet continuous reconstruction of the original movie."
This is late, House did it (Score:3)
A couple seasons ago...
Re: (Score:2)
Re: (Score:2)
Recording (Score:3)
Re: (Score:3)
Wake me when I can record on my PC dreams with Scarlett and Natalie, then view them in my dreams when I can fully interact with them.
On second thought, don't wake me then ;).
Re: (Score:3)
I suppose that in principle, if you can synthesise an image from a set of neural patterns you can do the reverse. Give the computer your video, and it'll give you the neural activity associated with that video. Just get your brain wired up at a sufficiently fine resolution and let the computer stimulate your visual centres to create the appropriate visual pattern. (Of course this is probably so coarse-grained that you'd just wind up recreating the "sex" scene in Demolition Man.)
Re:Recording (Score:4, Interesting)
Well, what this system is doing is selecting from a library of video clips the 100 clips whose brain signatures when viewed were closest to the brain signatures sensed later. So the video is lo-fi because of averaging of the images. More signal processing, particularly a stats model for excluding red herrings, will give this system higher specificity in selecting the video to match the sensed signature.
It will take another breakthru to generate a 3D image (more likely than a 2D image, which is really just a derived artifact of the 3D brain activity) directly from the sensed brain activity rather than selecting a correlating video. But that breakthru is at least as likely to occur in the SW/data video realm as in the brain/sensor realm. Because we might be able to use large video libraries, and swatches within their 2D images, as primitives from which to synthesize the 3D visual image, rather than building the recreated images from raw voxels. Which is, I believe, precisely how the brain does it.
What will also come along is reading and stimulating other brain sensory (apperceptory, really) regions that are active when we think we're having a hi-fi memory. Often the details are not remembered, but we "remember" that we are remembering the details, in the "metadata". That level of resynth will require the breakthru of stimulating those brain regions, not just reading mappable sensory regions. But with this research at UC Berkeley I think we are now in the (very long and difficult) phase of "working out the details". We are over the watershed into the new age.
Re: (Score:1)
Sex is one thing, but think of the big picture:
You could finally play WoW 24/7 without missing your sleep!
Re: (Score:1)
imagine GOATSE in 5D!
Re: (Score:2)
Re:Recording (Score:4, Insightful)
Re: (Score:2)
Watch out the MPAA might sue you for having dreams about their precious content.
They will insist that you either pay lifetime royalties or whipe your mind clean.
Re: (Score:2)
Images in the mind? (Score:1)
There are no images in the mind. And no dogs in dog biscuits.
Re: (Score:1)
Re: (Score:1)
There are no voices in your head either. To say so is just abuse of language.
Re: (Score:2)
The voices may not be real but they have some pretty good suggestions.
Re: (Score:1)
Re: (Score:1)
Any no baby in baby oil.
Re: (Score:1)
You'd think that wouldn't you...
Re: (Score:1)
There are no images in the mind. And no dogs in dog biscuits.
There was a debate [lesswrong.com], in the late 1800s, about whether "imagination" was simply a turn of phrase or a real phenomenon. That is, can people actually create images in their minds which they see vividly, or do they simply say "I saw it in my mind" as a metaphor for considering what it looked like?
The debate was resolved by Francis Galton, a fascinating man who among other achievements invented eugenics, the "wisdom of crowds", and standard deviation. Galton gave people some very detailed surveys, and found that some people did have mental imagery and others didn't. The ones who did had simply assumed everyone did, and the ones who didn't had simply assumed everyone didn't, to the point of coming up with absurd justifications for why they were lying or misunderstanding the question. There was a wide spectrum of imaging ability, from about five percent of people with perfect eidetic imagery to three percent of people completely unable to form mental images.
Perhaps you are one of the people who do not see images in your mind. Then you would think that nobody saw them and post what you did here.
Thought Police (Score:1)
not mentioned (Score:2)
Re: (Score:3)
Re: (Score:2)
Finally the RIAA can charge for every repeat of that song that's stuck in your head.
Re: (Score:2)
Finally the RIAA can charge for every repeat of that song that's stuck in your head.
Damn. I would be soooooo screwed if that happened. Ever since I recently discovered Hungry Like the Wolf (yes, I know I am about 3 decades too late on this one, but even though the 80s were my prime years, this one somehow managed to escape my notice back then), the song has been playing in my brain several times a day for a couple hours straight.
I would owe the RIAA a gajillion dollars or so, give or take a few.
Re: (Score:2)
Finally the RIAA can charge for every repeat of that song that's stuck in your head.
Damn. I would be soooooo screwed if that happened. Ever since I recently discovered Hungry Like the Wolf (yes, I know I am about 3 decades too late on this one, but even though the 80s were my prime years, this one somehow managed to escape my notice back then), the song has been playing in my brain several times a day for a couple hours straight.
I would owe the RIAA a gajillion dollars or so, give or take a few.
Thank you so much...
...I'm on a hunt I'm after you... Aaagh! Shut up!
Re: (Score:2)
Can I charge them instead? After all, their song is occupying space in my head - they really ought to be paying rent for that space.
Re: (Score:1)
Actually the MPAA are investing in the technology so that every time you think of any movie they can charge you
The plan is almost complete.
Card transaction data can be sold and monitored; movies one has watched can be plucked out of their head. Ones who didn't purchase a movie but DO think about images and scenes from that movie are targets for MPAA's ultimate message to the world: "Told ya we had bigger balls. IN YO' FACE!"
Shhhhhh.. Don't tell.
I wonder.. (Score:2)
If I go watch a movie in the theater, then replay it to my friends later from my mind.. Would that be an illegal bootleg?
Re: (Score:3)
That would be a very cool way to make spoilers.
Re: (Score:2)
If I go watch a movie in the theater, then replay it to my friends later from my mind.. Would that be an illegal bootleg?
It would only be a bootleg if the transmission came from your leg instead of your head, and only if you are wearing boots...and then only if the boots are stolen.
MAFIAA is pleased to announce that... (Score:2)
MAFIAA is pleased to announce that in their effort to combat piracy, lobotomy is now included in the list of remedies for three-strikes infraction. Naturally any attempt to recollect any part of any film is considered as one strike.
You could, at your option, pay a ... modest fine instead of receiving the lobotomy.
misleading demo (Score:5, Informative)
The output is not synthesized directly from the fMRI data.
Rather, they take a bunch of samples from youtube and try to find a sample that generates the closest match the fMRI data.
Still impressively neat. It's just that they need to more explicitly explain what they're doing.
Re: (Score:1)
Re: (Score:1)
Re: (Score:2)
Ah right.. I was wondered why there was typed text in the right clip.. Seemed unlikely the brain would subtitles everything internally. :)
Re: (Score:2)
It doesn't just "try to find a sample" that closely matches the data. It interpolates several together. It's expanding in a basis of YouTube clips and their corresponding fMRI patterns.
Re: (Score:2)
Yes doing exactly what you say you are doing is very misleading.
Re: (Score:2)
The resolution of a reconstituted neural image is higher if you directly wire electrodes to the brain; see Looking through cats' eyes [bbc.co.uk] and the PDF Reconstruction of natural scenes from ensemble r [harvard.edu]
Re: (Score:2)
2. Index the world's fMRI patterns. <-- missing step for Google
3. Reconstruct any movie and image from our minds.
4. ???
5. Profit!!!
Really constructed? (Score:2)
Starting at @0:06 and the words. Who the heck did they test this on and why do they apparently subtitle their thoughts?
@0:07, the words "Powershot" 2 secs in the bottom right.
"Lot4Life" in the elephants.
This is going to be a RTFA story it appears.
Re: (Score:3)
Its output is generated by interpolating together YouTube clips. The computer has a model of how a video maps onto a pattern of fMRI activity. Given a pattern of fMRI activity, it can attempt to generate an interpolation of YouTube clips that closely recreates that pattern.
Re: (Score:2)
For definitions of closely. That looks a lot more like a painting of a headcrab zombie on a grassy field in front of a lake of blood than a parrot.
Re: (Score:1)
True. When I first looked at it, I thought, "This is pretty nifty." Come to think of it, maybe I still think that... ...but upon further examination of the two images in the summary, I think the only direct correlation I can see between the images is the very general color of the bird and that the parrot's tail is on the correct side of its body in the "result" picture. Maybe if I were to RTFA, it would explain that the image had to be flipped to correspond or something.
Re: (Score:2)
It's also limited to the training data. Even if the brain activity can be accurately correlated to the closest training images, averaging 100 of the closest matches still has to come reasonably close to the actual scene for you to recognize it.
Re: (Score:2)
i am still skeptical, if the youtube data was really completely off limits to the reconstruction, or if the clips only show where things were similar.
we speak about tons of data - its enough if the color definitions is taken from the clips "to find them again" as one factor and you are screwed.
it all depends on the theory, that we know how the brain maps the seen things in neurons, and if fmri data does really show this data. i do not think, that is the case. so it seems too biased to say it really was a su
Time to dust off my tinfoil hat (Score:1)
The MPAA called... (Score:2)
they want to know when you'll be sending your royalty checks to them. Please make the check out to "All Your Thought Belong to Us"...
nightmarish (Score:3)
that video validated every single nightmare i've ever had
This is IT: Brain/Computer Interface (Score:1)
This science is the watershed in human/machine interfaces. An improved version of this tech will give computers direct reads of our visual mind. We will imagine scenes that computers will interpret to execute.
How far along are we getting in cheap, low-power SQUID [wikipedia.org] caps (or alternative gear) that we can wear to express to our Internet and personal processors what we imagine happening, so our machines can make it so?
copyright (Score:1)
so what now? What kind of an encrypted link of - image to brain (or sound or text to brain), are music and movie studios going to require government mandates? Because clearly, this shit is an unauthorized copy right there.
This is phenomenal! (Score:2)
The fact that this is compiled from superimposed youtube clips in no way detracts from how absolutely awesome this is. If I were to tell you what I'm seeing, you would also compare my description with images that you know and imagine my vision via metaphors.... which, all things considered, works quite well in everyday life, by the way.
Re: (Score:2)
I was going to say there's no reason why they needed to use YouTube footage, and they could've used abstract shapes and patterns as the basis for reassembling the image. Then it occurred to me that given our visual system is an evolved artefact, you might actually be able to work more efficiently using naturalistic images.
Scary (Score:1)
Re: (Score:2)
Why would you post this sentiment online, from a computer?
Re: (Score:1)
Fishing expedition. He's an AI.
Re: (Score:2)
There's a lot of technology we don't want around but are stuck with. Who would not want to put the nuclear genie back in it's bottle, if only we could? Alas, man does not often unlearn technologies.
Just like youtube.... (Score:1)
Hey thanks Slashdot editors! (Score:2)
On a whim I submitted this story (as you can see I'm the submittor) with the URL and a ONE WORD summary:
"Unbelievable"
The Slashdot editors either have a sense of humor and/or they don't mind doing some background research. They produced the paragraph long summary you see above and they went to the original article to clip out sone text and pictures.
So thanks! Do you get paid? :)
Re: (Score:2)
Maybe they can help out some aging astronauts [slashdot.org], too..
Re: (Score:3)
I could be getting paid? We actually do this with a lot of stories, and you're welcome!
Result is many arguments (Score:2)
Man: Honey, I had this hilarous dream last night, youv'e got to watch it! *plugs himself into the tv*
Woman: Why is your secretary wearing that skimpy dress in your dream?!?!?
Man: I.. uh.. that's a weird coicidence!
Re: (Score:1)
Woman: Why is your secretary wearing that skimpy dress in your dream?!?!?
Man: Damn, honey, you're right! Why in the hell IS she wearing it? Giggity giggity giggity!
Very Impressive (Score:1)
This is extremely impressive. That's better quality than I got on my first TV.
Can the same set of data be used on another person (Score:2)
In other words, is the scanning that is done only valid
for the test subject, or can you now use that same data
to analyze another test subjects visual cortex ?
Re: (Score:2)
The summary strongly implies that they had to calibrate it for each of the three study participants.
Re: (Score:1)
The summary strongly implies that they had to calibrate it for each of the three study participants.
Collectively or separately? :)
Re: (Score:2)
I imagine a lot of calibration would be needed - though with time that could be automated, unless it's some fundamental wiring difference in the way the data is transmitted.
Would be interesting to see them go the other direction and transmit images into the brain (who needs physical screens?), but that would likely be quite invasive.
Amazing (Score:2)
I was just thinking that.....amazing.
Innovative VJ technique (Score:1)
Fantastic (Score:1)
FINALLY!! (Score:2)
Hook me up to this and let my wife plug in a monitor whenever she's around. Then I won't ever have to answer, "What are you thinking?" again!!
She NEVER believes me when I give answers like, "If we give robots hands, will they pick their noses? And would running oil through air intake ducts do the same job as snot?"
Dammit! Those are LEGITIMATE engineering questions, and yet she thinks I'm just making things up. This could just save us the whole argument, and after a while she would just give up and quit
this is just a stunt (Score:2)
wipe (Score:1)
What movie did they watch? Brainstorm? (Score:2)
That would be wild to record someone watching Brainstorm, especially if they died.
Re: (Score:3, Funny)
yay for /.'s lag that made me think I forgot to press submit...
Re: (Score:2, Funny)
Yay for nailing first and second post, dude. You rock.
Re: (Score:1, Funny)
Re: (Score:1)
If you'd linked to what you're talking about we'd know you do more than watch TV hospital dramas.
Re: (Score:1)
I don't. I hibernate the whole time between House episodes.
Re: (Score:1)
Do you? Or is that all just a crazy dream?
Re: (Score:1)
I hibernate the whole time between House episodes.
Then House is on right now. Why are you posting on /. during House!?
Re: (Score:1)
Then House is on right now. Why are you posting on /. during House!?
If he'd been watching the computer screen he would have known he had already submitted the post the first time.
Re: (Score:1)
No, I'm under a sonambulism spell, since at this very moment, House is off in my country.
Re: (Score:2)
I don't think anyone's going to MRI scan you without your knowledge any time soon.
Re: (Score:2)
You'll be long dead by the time this technology becomes useful.
Re: (Score:3)
Re: (Score:3)
Conversely, I think this is an AWESOME development.
Just think. Once perfected this could be used in trials to definitively prove innocence or guilt. Massively cut back on the slimeball lawyers and jackasses lying in court to get off.
Re: (Score:2)
Just think. Once perfected this could be used in trials to definitively prove innocence or guilt.
They probably said that about polygraphs and DNA evidence too, and that hasn't worked out so well to be honest.
Re: (Score:2)
Re: (Score:2)
DNA is only really evidence of a persons presence during some huge window of time (even w/o contamination)...
Thus, for many crimes, DNA is merely circumstantial evidence, but somehow hollywood (professional expert witnesses), have convinced many lay-people that DNA=guilty. DNA as a general rule doesn't say who did what crime or if that person was even present when the crime was committed.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Or unless the statistical interpretation is flawed, which is easily done (especially when none of the lawyers or the judge understand Bayes' theorem).
Re: (Score:2)
Er, this technique records directly from a person's visual cortex, it doesn't replay memories.
And even if it did, it's been amply demonstrated that for many of us our memories aren't nearly as accurate or incorruptible as we like to believe.
On the other hand, if we gene-modded humanity so we all had perfect recall, and developed the parallel of this technique for memory, and had multiple independent witnesses, then we might have something the courts could consider "definitively" reliable. Maybe.
I suspect so
Re: (Score:2)
Re: (Score:2)
That might be your lawyer's job, but my lawyer's job is to get me off at all costs.
However, I have to tell you, I don't think your lawyer is very good then.
Re: (Score:2)
That might be your lawyer's job, but my lawyer's job is to get me off at all costs.
No, that is exactly my point - the lawyer is not supposed to get someone off at all costs. It's this attitude that results in paid witnesses lying on the bench, ridiculous arguments that just confuse juries, illegal destruction of evidence, an just all around perjury. It's there actions that have resulted in the justice system going to complete shit.
Justice *should* be the search for the truth, and protection of rights, nothing more.
Re: (Score:2)
Because it doesn't seem nearly as impressive. This is the same reason that fires are reported in acres rather than square miles, and wind chill or heat index are used to report weather extremes.