Slashdot is powered by your submissions, so send in your scoop

 



Forgot your password?
typodupeerror
×
Medicine AI Robotics Software Technology

University of Columbia Researchers Translate Brain Signals Directly Into Speech (columbia.edu) 84

dryriver writes: There is good news for people who have limited or no ability to speak, due to having suffered a stroke for example. Researchers at Columbia University have managed to turn brain signals in the auditory cortex of test subjects into somewhat intelligible speech using a vocoder-like system with audio output cleaned up by neural networks. The findings have been published in the journal Nature. Here's an excerpt from the Zuckerman Institute's press release, which contains example audio of a number sequence being turned into robotic speech: "In a scientific first, Columbia neuroengineers have created a system that translates thought into intelligible, recognizable speech. By monitoring someone's brain activity, the technology can reconstruct the words a person hears with unprecedented clarity. This breakthrough, which harnesses the power of speech synthesizers and artificial intelligence, could lead to new ways for computers to communicate directly with the brain. It also lays the groundwork for helping people who cannot speak, such as those living with amyotrophic lateral sclerosis (ALS) or recovering from stroke, regain their ability to communicate with the outside world."
This discussion has been archived. No new comments can be posted.

University of Columbia Researchers Translate Brain Signals Directly Into Speech

Comments Filter:
  • Transcript (Score:2, Funny)

    by Anonymous Coward

    Dear aunt let's so double the killer delete select all

  • by spiritplumber ( 1944222 ) on Friday February 01, 2019 @02:14AM (#58054024) Homepage
    Seriously though, this is amazing.
    • by PolygamousRanchKid ( 1290638 ) on Friday February 01, 2019 @02:38AM (#58054066)

      Seriously though, this is amazing.

      I agree. However, I'm afraid that the police, NSA, CIA, DHS and their pals will discover the usefulness of this neat new toy.

      Well, at least they might stop water boarding people.

      Before you enter the US, we'd like you to put on this funny hat, and ask you a few questions. Don't bother answering; the computer will speak your thoughts for you.

      • by Anonymous Coward

        Seriously though, this is amazing.

        I agree. However, I'm afraid that the police, NSA, CIA, DHS and their pals will discover the usefulness of this neat new toy.

        Well, at least they might stop water boarding people.

        Before you enter the US, we'd like you to put on this funny hat, and ask you a few questions. Don't bother answering; the computer will speak your thoughts for you.

        Seriously though, this is amazing.

        I agree. However, I'm afraid that the police, NSA, CIA, DHS and their pals will discover the usefulness of this neat new toy.

        Well, at least they might stop water boarding people.

        Before you enter the US, we'd like you to put on this funny hat, and ask you a few questions. Don't bother answering; the computer will speak your thoughts for you.

        Look on the bright side, perhaps they'll be less scared of the left wing if they can finally confirm beyond any doubt that all that most of us are thinking about 99% of the time is boobies and weed, ... except the women, they are mostly thinking about shoes .... and weed.

        • by Anonymous Coward

          Or the computer's speech synthesis code sees "non-white" or "from bad country" or "fuck with them" flagged in their profile data and starts spitting out things like "alalalalalalalahhhbhhbh" or "death to America" bullshit regardless as to what the person is actually thinking.

          After all the computer said it so it must be true......

        • Look on the bright side, perhaps they'll be less scared of the left wing if they can finally confirm beyond any doubt that all that most of us are thinking about 99% of the time is boobies and weed, ... except the women, they are mostly thinking about shoes .... and weed.

          I never think about weed...except when spraying roundup. It's all boobies for me.

          It would be embarrassing though...

          We would like to ask you a few questions, ignore this mind reading device...

          Now, Mr. McWeany,
          oh shit! they think my name is really McWeany, that's an alias, my real name is Dickbutt
          OK, Mr.Dickbutt...
          Why did my brain just say Dickbutt, that's not my name either... I'm so scatterbrained, I wonder if I left the oven on
          Where were you on the evening of the 17th?
          The female agent has nice lips, I wo

      • I'm sure that people could be trained to defeat this technology for when it's used for interrogation purposes, but still: the Ultimate invasion of privacy, and the possibility of a new dark era of human history: thought crime.
      • Except this would only be the things that you would be saying. There is a big gap between thinking something and forming the words and speaking them...

        Even if it was just your thoughts, who's to say that the first thing that comes to mind is truth?

      • I'm afraid that the police, NSA, CIA, DHS and their pals will discover the usefulness of this neat new toy

        My worry too. I'm guessing that since the technology monitors the auditory cortex, it's not going to pickup on anything other than what you would consciously want to say - maybe a passing neuroscientist can weigh in.

  • by SensitiveMale ( 155605 ) on Friday February 01, 2019 @02:20AM (#58054030)

    "Yes, I'd like to order a Big Mac BOOBS! Good Lord look at those boobs! Ummm, some fries and a My God those boobies are huge! Umm and diet coke. Damn, those of tig ass bitties!"

    That's what every guy's brain is thinking.

    • by Anonymous Coward

      It doesn't translate your subconscious thoughts, just the vocalized region of the brain.

      Unless you're schizophrenic, then I guess this makes sense.

      • by sad_ ( 7868 )

        from TFA;

        “In this scenario, if the wearer thinks ‘I need a glass of water,’ our system could take the brain signals generated by that thought, and turn them into synthesized, verbal speech,” said Dr. Mesgarani.

        it clearly says it will vocalize what the brain is thinking. unless there is an easy way to turn that off/on i don't think anybody actually wants something like that.

        • it clearly says it will vocalize what the brain is thinking. unless there is an easy way to turn that off/on i don't think anybody actually wants something like that.

          Yeah - I think Ima let that thing miss me.

          If someone asks me a question that indicates they weren't paying attention, I do think "You stupid fuck!", while in practice going back to what I said and re-pointing it out.

          I pick up concepts quickly. When others don't, I might think "y'all are a little slow", while in practice I sit politely.

          If an attractive woman walks into the room, I think thoughts for a moment that every heterosexual male would. In practice, I say "Hello, how are you today?"

          Look on

      • by Vintermann ( 400722 ) on Friday February 01, 2019 @08:24AM (#58054714) Homepage

        It would be creepy as hell, but very cool, to get a recording of the voices schizophrenics hear.

        • It would be creepy as hell, but very cool, to get a recording of the voices schizophrenics hear.

          Boy howdy would it. I have always been intrigued by Schizophrenic speech patterns. But to hear the voices in their heads alonng with them? Whoa!

    • by Anonymous Coward

      You laugh but this already is happening with eye trackers. Seriously, look on youtube for "fail" videos wherein people are judged for looking at a cute girl (or guy) in some other place besides their head. Pewdiepie for example.

      My mind is mine, no one elses.

    • That's what every guy's brain is thinking.

      I'm a DICK gay guy DICK, you insensitive DICK clod!

    • It would be like the Belcerabons [wikipedia.org] who were sentenced to telepathy. Everyone would have to go around talking continually about inane trivia.

      > OHHH, YOU'RE VERY TALL AREN'T YOU?
      >> YES I AM VERY TALL. LOVELY WEATHER TODAY ISN'T IT?
      etc etc.
    • by k6mfw ( 1182893 )
      reminds me of this episode "Do Larger Breasts Equal Bigger Tips? | MythBusters" https://www.youtube.com/watch?... [youtube.com]

      They would have collected a lot more data besides tip amounts.

  • by Anonymous Coward

    I just hope the system has push to talk button...

  • Obligatory (Score:5, Funny)

    by Miles_O'Toole ( 5152533 ) on Friday February 01, 2019 @02:50AM (#58054094)

    Now they just have to figure out how Trump manages to generate speech while demonstrating no measurable brain activity.

    • Oh, that's already been figured out. Seth Meyers has been using the University of Columbia's technology for a few months now and broadcasting the results in the form of...

      The Tiny Voice In The Back Of Donald Trump's Head

      For example, Trump's inner voice on Don't Lose The Queen [youtube.com]
  • by abies ( 607076 ) on Friday February 01, 2019 @02:52AM (#58054098)
    Please note that while it seems like they cracked translating thought into speed, it is really about something very different. They decode what person is hearing into sound. This is very, very far from being able to interpret actual thoughts - they attach themselves to very early stage of hearing process, where things are still looking very much like sound waves. Additionally, it is tested on just 10 words (numbers from 0 to 9). On top of that, given neural nets are magic, I wouldn't be surprised if instead on brain activity, they focused on some measurement artifact, which is caused by skull vibrating to the sound, turning all that setup into overcomplicated and underperforming microphone. Sure, it is a worthy research, but it is light years away from thoughts to speech conversion.
    • by teslar ( 706653 ) on Friday February 01, 2019 @03:47AM (#58054198)

      Yes, this.

      The only thing I would add is that we do know that higher-level cognitive processes re-activate sensorimotor parts of the brain; so thinking the number "nine" might cause some activation similar to hearing that number.

      So, assuming they didn't indeed just pick up on some non-neural artifact, this shows that there is potential - at least in the ideal case where we know the signal is as good as it's ever going to get. It's one step in the direction of being able to deal with weaker, noisier versions you might get from reactivation.

      But yeah, noone's going to read your thoughts anytime soon.

    • This is very, very far from being able to interpret actual thoughts - they attach themselves to very early stage of hearing process, where things are still looking very much like sound waves.

      Yes. My first thought when I read the headline was: what language is the speech in?

      But it seems that this work, while interesting in an academic sense, is far from what the headline indicates.

    • by mentil ( 1748130 )

      translating thought into speed

      Apparently, the parent needed more thought and less speed in order to avoid that typo.

  • So these are scientists living in Bogota? Those Columbian researchers are pretty amazing. Must be all that coffee.

  • Have they tried using this on a dog yet?
  • Here, clean speech was used as both the input and output of the system. The subjects listened to the speech, and the speech was the target, having being reconstructed with the brain activity. (This technique uses invasive electrocorticography; they happened to have some epileptics around to use as subjects)

    The target was on the speech perception. Speech production would need to focus on another region of the brain (Maybe Broca's Area or straight to the motor cortex). The training data for speech production

  • Frylock made one for Travis.
  • by Anonymous Coward

    Be good for people in comas and the brain dead.

  • by elrous0 ( 869638 ) on Friday February 01, 2019 @08:33AM (#58054752)

    We hooked it up to Ariana Grande and it just keeps printing out "Need attention" over and over.

  • Ok. That's simply click-bait.

    If you're able to turn thoughts into speech then you're able to record it. How soon until memories can be saved? Such as the Potterverse' Pensieve.

    These advances are close to fantastical. And the world goes ho-hum.
  • Governments will argue this can be used to detect criminal evidence. We need laws to stop this cold because it will be abused in dictatorships.

  • But scientists just announced a laser that can cause a person to hear words. Connect this gadget up to that laser (and maybe put it in a bionic eye) and boom, something not too different from telepathy.

Veni, Vidi, VISA: I came, I saw, I did a little shopping.

Working...