A Review of audiovisual speech perception based on McGurk effect: evidences and models

In face-to-face speech communication, people perceive phonetic signals using both acoustic and visual cues. Particularly in a noisy or sound-degraded environment, listeners will naturally take advantage of visual cues to help them better perceive phonetic signals. How-ever, previous studies demonstrate that when visual cue and acoustic cue are mismatched, a perceiver may produce a kind of cognitive illusion, known as McGurk effect or McGurk illusion.
In the past few decades, researchers have made a great deal of investigations in an effort to elucidate the integrating processes of auditory and visual information underlying the McGurk illusion and have had lots of achievements. The current article reviews a variety of significant empirical findings on audiovisual speech perception involved in the McGurk effect. An introduction and discussion on the major existing theories and models as well as the application of advanced neuroscience techniques as approaches to explore the illusionary phenomenon are given here. Based on a wide study on the related literature, we conclude that audiovisual integration of speech is a multifaceted and non-single-stage (multistage) process, which is reflected in brain areas as dynamic and complex spatiotemporal activation patterns, and that that different features of bimodal stimulus might be constrained by different mechanisms. Furthermore, the Motor Theory of Speech Perception is commonly accepted for its informative explanation by those researchers who apply neuroscience approaches in their study involved the McGurk effect. Finally, we suggest that more cutting edge neuroscience and modeling methods should be used to explore specific aspects of factors involved perception, attention, development and culture, etc. in the processing of audiovisual integration to clarify how audiovisual integration underlying McGurk effect is processed in the future research of this field.