Incongruent Visual Cues Affect the Perception of Mandarin Vowel But Not Tone

In the past few decades, a large number of audiovisual speech studies have focused on visual cues of consonants and vowels rather than on lexical tones. In the present study, we investigated whether incongruent audiovisual information interfered with the perception of lexical tones. We found, for both Chinese and English speakers, incongruence between auditory and visemic mouth shape (i.e., visual form information) significantly interfered with reaction time and reduced the identification accuracy of vowels. However, incongruent lip movements (i.e., visual timing information) did not interfere with the perception of auditory lexical tone. We conclude that, in contrast to vowel perception, auditory tone perception seems relatively impervious to visual congruence cues, at least under these restricted laboratory conditions. The salience of visual form and timing information is discussed based on the finding.