Primacy of Mouth over Eyes: Eye Movement Evidence from Audiovisual Mandarin Lexical Tones and Vowels

Biao Zeng, Rui Wang, Guoxing Yu, Christian Dobel

    Research output: Contribution to conferencePaperpeer-review

    88 Downloads (Pure)

    Abstract

    This study investigated Chinese speakers’ eye movements when they were asked to identify audiovisual Mandarin lexical tones and vowels. In the lexical tone identification task, Chinese speakers were presented with an audiovisual clip of Mandarin monosyllables (/ă/, /à/, /ĭ/, /ì/) and asked to identify whether the syllables were presented in a dipping (/ă/, / ĭ/) or falling tone (/à/, /ì/). In the vowel identification task, they were asked to identify whether the vowels were /a/ or /i/ regardless of lexical tone. These audiovisual syllables were presented in clear, noisy, and silent conditions. An eye-tracker recorded the participants’ eye movements.
    Results showed participants gazed more at the mouth than the eyes in both lexical tones and vowels. Additionally, when acoustic conditions degraded from clear to noisy and eventually silent, Chinese speakers increased their gaze towards the mouth rather than the eyes. These findings suggest the mouth to be the primary area that is utilised during audiovisual speech perception. The similar patterns of eye movements between vowels and lexical tones indicate that the mouth acts as a perceptual cue that provides articulatory information.
    Original languageEnglish
    Publication statusPublished - 2021
    EventInterSpeech 2021 - Brno University of Technology, Brno, Czech Republic
    Duration: 30 Aug 20213 Sep 2021
    https://www.interspeech2021.org/

    Conference

    ConferenceInterSpeech 2021
    Country/TerritoryCzech Republic
    CityBrno
    Period30/08/213/09/21
    Internet address

    Fingerprint

    Dive into the research topics of 'Primacy of Mouth over Eyes: Eye Movement Evidence from Audiovisual Mandarin Lexical Tones and Vowels'. Together they form a unique fingerprint.

    Cite this