Primacy of mouth over eyes to perceive audiovisual Mandarin lexical tones

Biao Zeng, Guoxing Yu, Nabil Hasshim, Shanhu Hong

Research output: Contribution to journalArticlepeer-review

7 Downloads (Pure)

Abstract

The visual cues of lexical tones are more implicit and much less investigated than consonants and vowels, and it is still unclear what facial areas contribute to facial tones identification. This study investigated Chinese and English speakers’ eye movements when they were asked to identify audiovisual Mandarin lexical tones. The Chinese and English speakers were presented with an audiovisual clip of Mandarin monosyllables (for instance, /ă/, /à/, /ĭ/, /ì/) and were asked to identify whether the syllables were a dipping tone (/ă/, / ĭ/) or a falling tone (/ à/, /ì/). These audiovisual syllables were presented in clear, noisy and silent (absence of audio signal) conditions. An eye-tracker recorded the participants’ eye movements. Results showed that the participants gazed more at the mouth than the eyes. In addition, when acoustic conditions became adverse, both the Chinese and English speakers increased their gaze duration at the mouth rather than at the eyes. The findings suggested that the mouth is the primary area that listeners utilise in their perception of audiovisual lexical tones. The similar eye movements between the Chinese and English speakers imply that the mouth acts as a perceptual cue that provides articulatory information, as opposed to social and pragmatic information.
Original languageEnglish
Number of pages12
JournalThe Journal of Eye Movement Research
Volume16
Issue number4
DOIs
Publication statusPublished - 29 Nov 2023

Keywords

  • lexical tone
  • eye movement
  • gaze
  • audiovisual speech
  • Chinese speakers
  • English speakers

Fingerprint

Dive into the research topics of 'Primacy of mouth over eyes to perceive audiovisual Mandarin lexical tones'. Together they form a unique fingerprint.

Cite this