Primacy of Mouth over Eyes: Eye Movement Evidence from Audiovisual Mandarin Lexical Tones and Vowels

Biao Zeng, Rui Wang, Guoxing Yu, Christian Dobel

Research output: Contribution to conferencePaperpeer-review

151 Downloads (Pure)

Abstract

This study investigated Chinese speakers’ eye movements when they were asked to identify audiovisual Mandarin lexical tones and vowels. In the lexical tone identification task, Chinese speakers were presented with an audiovisual clip of Mandarin monosyllables (/ă/, /à/, /ĭ/, /ì/) and asked to identify whether the syllables were presented in a dipping (/ă/, / ĭ/) or falling tone (/à/, /ì/). In the vowel identification task, they were asked to identify whether the vowels were /a/ or /i/ regardless of lexical tone. These audiovisual syllables were presented in clear, noisy, and silent conditions. An eye-tracker recorded the participants’ eye movements.
Results showed participants gazed more at the mouth than the eyes in both lexical tones and vowels. Additionally, when acoustic conditions degraded from clear to noisy and eventually silent, Chinese speakers increased their gaze towards the mouth rather than the eyes. These findings suggest the mouth to be the primary area that is utilised during audiovisual speech perception. The similar patterns of eye movements between vowels and lexical tones indicate that the mouth acts as a perceptual cue that provides articulatory information.
Original languageEnglish
DOIs
Publication statusPublished - 2021
EventInterSpeech 2021 - Brno University of Technology, Brno, Czech Republic
Duration: 30 Aug 20213 Sept 2021
https://www.interspeech2021.org/

Conference

ConferenceInterSpeech 2021
Country/TerritoryCzech Republic
CityBrno
Period30/08/213/09/21
Internet address

Fingerprint

Dive into the research topics of 'Primacy of Mouth over Eyes: Eye Movement Evidence from Audiovisual Mandarin Lexical Tones and Vowels'. Together they form a unique fingerprint.

Cite this