Visual tone facilitates the early auditory tone processing: evidence from N1 and P2

Salient visual feature facilitates to predict the corresponding auditory speech signal, hence leading to a greater audiovisual (AV) reduction effect in N1/P2 latency (van Wanssenhove, Grant & Poeppel, 2005). The current study investigated whether such AV reduction occurred to Mandarin lexical tone. For lexical tones, the visual feature is less salient than the one in consonants, e.g. mouth shape or lip movement. Based on this, the AV reduction effect should be smaller in lexical tone response compared the one in consonant response. Same-different discrimination paradigm was used in the study. Participates were asked to judge whether two sequential stimuli were same or different for their auditory information. The two sequential stimuli were played in three conditions; 1) auditory-only and audio-only; 2) auditory-only and audiovisual; 3) auditory-only and visual-only. Twenty participants (aged 25.75 ± 4.4 years; female: 13) participated in the lexical tone experiment, and 19 participants (aged: 26.58 ± 5.69 years; female: 11) took part in the consonant experiment. The results showed a few major electrophysiological features of AV lexical tone integration. First, the auditory processing of lexical tones was suppressed and was accelerated by visual tone input starting from the auditory N1 time range, which further supports the AV benefit effect on lexical tones. Second, AV lexical tone integration maximised later than AV consonant integration and lasted longer (over N1 to P2). Third, lexical tone interaction processing showed more right-lateralised activation.