Primacy of mouth than eyes to perceive audiovisual Mandarin lexical tones

This study investigated Chinese and English speaker’s patterns of eye movement when they were asked to identify audiovisual Mandarin lexical tones. The Chinese and English speakers were presented a clip of Mandarin monosyllable (/ă/, /à/, /ĭ/, /ì/) in audiovisual mode and asked to identify whether the syllable was a dipping tone (/ă/, / ĭ/) or a falling tone (/ à/, /ì/). These audiovisual syllables were presented in clear, noisy and silent conditions. An eye-tracker recorded the participants’ eye movement. It was found that the participants gazed more at mouth than eyes and the mouth took more than 40% gaze duration at average. In addition, when the auditory information descended through three acoustic conditions, both Chinese and English speakers increased the gaze duration at mouth rather than eyes. The findings suggest, for audiovisual lexical tone, mouth is the primary area, not eyes. The similar eye movement patterns between Chinese and English speakers implies that mouth might play a perceptual cue relevant to articulatory than semantic information.