McGurk Description

The McGurk effect is an auditory illusion produced by a visual experience. To produce the effect, the talker was recorded saying the syllable "ma", and videotaped while saying the word "ka." The spoken utterance was carefully aligned with the video, resulting in the clip you just saw.

The "m" in "ma" is produced with closed lips (bilabially). "Na" and "ka" are both produced with open lips. In the video clip, the talker has open lips. The visual experience of a talker producing an open-lip sound seems to override our auditory experience of a closed-lip "ma" syllable.

Not everyone experiences the McGurk illusion, though the "ma" + "ka" = "na" form is one of the strongest combinations to produce the effect. Another form that will produce the illusion, at least for some people, is "ba" (auditory) + "ga" (visual) = "da." Again, "ba" is produced with closed lips, while "da" and "ga" are produced with open lips. Of course, the effect has its limits, and you have probably seen those limits in a badly-dubbed movie. At some point, you realize the words coming out of the actor or actresses mouth are not the same words you are hearing. So the McGurk effect is limited to similar sounds.

Although it might seem a curiosity, the McGurk effect is important in several respects. Perhaps most importantly, the effect compellingly illustrates the multi-sensory nature of our internal descriptions: we don’t experience a visual world, a (perhaps different) auditory world, a (perhaps different) tactile world, a (perhaps different) olifactory world, and so on. Instead, we experience ONE WORLD through multiple senses. The information we perceive through hearing and the information we perceive through sight are coordinated and inter-related. Typical discussions of sensation and perception tend to overlook this important fact. Doubtless this occurs because of the complexity of the constituent senses. It is hard to understand how the eye works. It is also hard to understand how the ear works. So we treat each sense separately, but somehow overlook the mental coordination that occurs between the senses.

Lip readers exploit this coordination to make up for a deficit in hearing. Again, this ability to use sight in the place of sound might seem strange, but it can be very helpful to the hard-of-hearing.

The Circle of Thought is designed to make multi-sensory coordination explicit. The first function in the Circle of Thought is "Description." Not sensation, not perception. We try to internally describe the world around us. This description is multi-sensory, and relies on the processes of sensation and perception, but goes beyond the simple world of the individual senses.

The McGurk effect is also important because it reveals something about the dominance of various senses. Humans are highly-visual animals, and our sense of vision often dominates our sense of hearing. The McGurk effect occurs because of visual dominance. Other mammals, such as dolphins, rely much more on sound than on vision. We might imagine they could experience a reverse McGurk effect, where the sound produced by an object influenced its visual interpretation. Of course, just as in humans, we would expect the effect to occur only for similar-looking objects. But a nurse shark that sounded like a hammerhead might look like a hammerhead. You never know!

Here are some additional video clips you can look at.