Catch up on stories from the past week (and beyond) at the Slashdot story archive

 


Forgot your password?
Close
typodupeerror
×
Science

How Seeing Can Trump Listening, Mapped In the Brain 51

Posted by timothy from the you're-just-envious-of-the-sensors dept.
cortex writes "University of Utah bioengineers discovered our understanding of language may depend more heavily on vision than previously thought: under the right conditions, what you see can override what you hear. In an article published in PLOS One, 'Seeing Is Believing: Neural Representations of Visual Stimuli in Human Auditory Cortex Correlate with Illusory Auditory Perceptions,' the authors showed that visual stimuli can influence neural signals in the auditory processing part of the brain and change what a person hears. In this study patients were shown videos of an auditory illusion called the McGurk Effect while electrical recordings were made from the surface of the cerebral cortex."
This discussion has been archived. No new comments can be posted.

How Seeing Can Trump Listening, Mapped In the Brain More

How Seeing Can Trump Listening, Mapped In the Brain

Comments Filter:

  • Kind of expected this... (Score:5, Interesting)

    by Thantik ( 1207112 ) on Saturday September 07, 2013 @09:37AM (#44783117)

    Ever happen to be looking for a street in an unknown area, and you end up turning down your radio? This actually increases your visual acuity slightly even though many may question your sanity when doing it. Many blind people have increased auditory capacity, and this has been known for a very long time. It doesn't seem all that far fetched that one (or more) of those senses could override the other. Maybe that meal that you hate tastes absolutely amazing...but looks so terrible that you taste it differently.

  • Trump listening? (Score:1)

    by Anonymous Coward

    I think the Donald only listens to other right-wing conspiracy nutters

    • Re: (Score:1)

      by Anonymous Coward

      But for a mere $45K you'll get to pose next to a lifesize cardboard cutout of Trump, and get a piece of paper certifying why nobody should let you get anywhere near their money.

  • Narrator annoying (Score:3)

    by I'm New Around Here ( 1154723 ) on Saturday September 07, 2013 @09:46AM (#44783145)

    Why can't the woman shutup for 10 seconds so I can concentrate on the effect she keeps prattling on about? It makes it difficult to study when her voice is over top of the illusion half the time.

  • After googling for the McGurk Effect and watching a bunch of videos I have concluded that I can't really sense this effect at all. I'll take their word that most people can.

    Probably what is more interesting is some of the info on Wikipedia about the sorts of things that make the effect more or less pronounced. I'd be interested in the results for the average Slashdotter - I suspect that things like mild autism are much more common here.

    • Re: (Score:2)

      by Qzukk ( 229616 )

      Hadn't heard of this McGurk effect before. I was thinking it was something related to how I can still hear the noisy user interface elements in apps and games in my head when I have the sound muted and mouse over those menus that go "tink".

      As for the McGurk video I saw of a man saying baa, it worked on me until the voice over told me the man was saying baa. Then I could look at the guy's lips and still hear baa.

    • I heard the different sounds, bar and far, quite clearly and distinctly.

      • I heard the different sounds, bar and far, quite clearly and distinctly.

        Me too. The effect works on me. If I watch the video, I hear "fa". If I close my eyes I hear "ba". Even when I know he is saying "ba", I hear "fa". So I am normal.

    • I too did not suffer from the "effect". I believe it has to do with the fact that I had 8+ years of vocal performance training and sang in various choirs and madrigal groups in many euro-classical forms and languages (German, Italian, French, multiple Latin dialects (yes, there are at least three that I am aware of, German Latin (used in many pieces, especially those of Bach and Mozart), Liturgical Latin (this is the Latin handed down and used through the Roman Catholic Church), and Italian/Roman Latin) and
      • Oh, as a side note, 80-90% of all people are tone deaf and can not tell the difference between notes even as far apart as a septimal quarter-tone, unless they are played at the same time (causing a harmonic dissonance wave).
        • Oh, one more aside, many tuning systems do not even allow for notes of that different in tone to even be played (such as a piano, harp, trumpet, clarinet, flute, sax, etc., etc.,). Only stringed instruments (which are fretless or which the string can be bent to change the tension) and the trombone have this ability. Some instruments can tune their notes to use a scale which does include which uses septimal quarter-tone intervals as a part of it, but once so tuned, it then can not play the standard tuning no

          • Re: (Score:2)

            by Rich0 ( 548339 )

            Hmm - that might or might not be the cause. I do play the cello and can sing reasonably well. I wouldn't put myself among the masters when it comes to intonation, but in general I would say that my intonation is quite good.

            On the other hand, I have a friend who can sing harmony with just about anything by ear with excellent intonation and she definitely noticed the effect. This friend also has anomic aphasia, so who knows - there are a bunch of different neurological anomalies between us...

          • Oh, one more aside, many tuning systems do not even allow for notes of that different in tone to even be played (such as a piano, harp, trumpet, clarinet, flute, sax, etc., etc.,).

            That's mostly untrue for wind instruments. I can modulate a note on the
            trumpet more than a halftone up and down easily. There's - mostly modern - music
            for winds that notates quarter tones, and it's played with standard instruments.

            Flutes seem to be especially easy to modulate, but I know many people capable of
            doing this, playing all kinds of winds.

            Very well known example: The opening of Gershwin's Rhapsody in Blue [youtube.com]...

    • Re: (Score:2)

      by Jmc23 ( 2353706 )
      It's all about ability to take cues from another persons face and be able to replicate them. If you can't put yourself 'inside' another person to learn a physical motion, then this 'illusion' won't work on you. Illusion isn't the correct term since it's in fact just a more dominant sense replacing input of a less dominant sense. They should actually be able to turn this 'illusion' on its head by testing people whose auditory senses are more dominant than their visual ones.

      • It's all about ability to take cues from another persons face and be able to replicate them. If you can't put yourself 'inside' another person to learn a physical motion, then this 'illusion' won't work on you.

        There is a theory that all language relies on reconstructing the speaker's movements, employing the mirror neurons. The most sophisticated model of this is V S Ramachandran's notion of language being a combination of mirror neuron action and synaesthesia (where stimulus from one sense maifests itself in another sense -- eg seeing colours tied to musical notes). The important thing about synaesthesia in Ramachandran's work is that he considers it a normal state of affairs, and demonstrates this with examples

        • Re: (Score:2)

          by Jmc23 ( 2353706 )
          Well, very easy to disprove that theory. There are, after all, many unfortunate cases of people, like me, very deficient in these sympathetic mirror neurons that are/were totally unable to learn from others. Yet, i still had language. Granted, i did not understand words where the energetic purpose behind them created a specific subjective kinesthethic 'feeling', e.g., looking down, stand tall, chin up, stiff upper lip,etc...

          I'm actually working on a system that breaks down subjective human experience int

          • Well, very easy to disprove that theory. There are, after all, many unfortunate cases of people, like me, very deficient in these sympathetic mirror neurons that are/were totally unable to learn from others. Yet, i still had language. Granted, i did not understand words where the energetic purpose behind them created a specific subjective kinesthethic 'feeling', e.g., looking down, stand tall, chin up, stiff upper lip,etc...

            "Very deficient" is still very different from "totally deficient" -- the amount required for any base function is unclear. Besides, the role of the mirror neuron system in ASDs (which I'm assuming you're referring to) is still very much open to debate.

    • After googling for the McGurk Effect and watching a bunch of videos I have concluded that I can't really sense this effect at all. I'll take their word that most people can.

      Doesn't work for me either, not even a bit. "Ba" all the way.

      I have made an observation that is semi-related to this though:

      Watching a subtitled movie where the spoken language is totally opaque
      to me, fast-paced dialogue can be tricky to follow. In this case, turning the
      sound up helps me read the subtitles.

      Brains are really weird.

  • Hmmm (Score:3)

    by slick7 ( 1703596 ) on Saturday September 07, 2013 @10:21AM (#44783273)
    One picture, one thousand words, who'd a thunk it?

  • There was an article a few days ago about software to create the illusion of eye contact [slashdot.org] with video calls. As some people pointed out, continuous eye contact is actually somewhat disconcerting. When I'm listening to someone, I tend to watch their lips, particularly in high background noise environments.

  • How can he physically produce the "Baa" sound when his bottom lip is tucked behind his teeth? His lips don't press together when the illusion is supposed to make it sound like he's saying "Faa" instead. I think he's actually making two different sounds.

    Isn't the "Baa" sound impossible to make without the lips pressing together? Isn't the "Faa" sound impossible to make without blowing on the lip-teeth connection with the top and bottom lips separate?

    • Re: (Score:3, Informative)

      by GreyWanderingRogue ( 598058 )

      How can he physically produce the "Baa" sound when his bottom lip is tucked behind his teeth? His lips don't press together when the illusion is supposed to make it sound like he's saying "Faa" instead. I think he's actually making two different sounds.

      Isn't the "Baa" sound impossible to make without the lips pressing together? Isn't the "Faa" sound impossible to make without blowing on the lip-teeth connection with the top and bottom lips separate?

      This is the perfect description of what your brain is doing. Unfortunately, it appears you're misunderstanding what is actually going on. There is only one audio recording. It is dubbed over both clips. Check the section where there is a side by side with both videos. What you hear depends on what side of the screen you look at, even though there is only one audio track.

    • How can he physically produce the "Baa" sound when his bottom lip is tucked behind his teeth?

      He can't...

      Isn't the "Baa" sound impossible to make without the lips pressing together? Isn't the "Faa" sound impossible to make without blowing on the lip-teeth connection with the top and bottom lips separate?

      ...that's why it's the same sound dubbed over different videos.

  • There was a supposedly entertaining example of this on one of the astronomy blogs a while back, but they got a takedown notice before I saw it. Changed what you heard in a music video, IIRC. Does anyone know where we can find that?

  • It reminds me of this:
    http://www.economist.com/news/science-and-technology/21583974-top-musicians-are-judged-much-their-movements-their-melodies [economist.com]

    In short, people were bad at guessing who won the competition when they could only hear the music and not see the performers. Professional judges were just as bad as novices (worse when they could see the performance but just as bad when they could only hear it).

    Interesting stuff.

  • Being a Canuck that has lived north of 60 I have seen incredible instances of the Northern Lights. What I always thought was strange was when the lights were in certain patterns and intensities I heard a strange crackling like static electric charges. Knowing that the lights originated from well beyond the atmosphere I just dismissed this as a fantasy. UNTIL one day having a beer in a pub in Watson Lake Yukon I overheard several people claiming to hear the lights the same way as I do!

    THIS article finally

  • They "discovered" that? I remember a demonstration of this in a phonetics class, which I took in like 2002. Though my guess is that it's only the summary that's wrong, and the researchers themselves didn't claim to "discover" anything, but were simply doing new neurological research on the effect, given that the summary itself mentions the McGurk effect. Quoth Wikipedia, "The McGurk effect is sometimes called the McGurk-MacDonald effect. It was first described in 1976 in a paper by Harry McGurk and John Mac

Slashdot Top Deals

What is research but a blind date with knowledge? -- Will Harvey

Close