Tolerance for audiovisual asynchrony is enhanced by the spectrotemporal fidelity of the speaker's mouth movements and speech

作者：Shahin Antoine J^*; Shen Stanley; Kerlin Jess R

来源：Language Cognition and Neuroscience, 2017, 32(9): 1102-1118.

DOI：10.1080/23273798.2017.1283428

• Audiovisual integration
• audiovisual onset asynchrony
• degraded speech
• spectrotemporal fidelity
• noise vocoded speech

We examined the relationship between tolerance for audiovisual onset asynchrony (AVOA) and the spectrotemporal fidelity of the spoken words and the speaker's mouth movements. In two experiments that only varied in the temporal order of sensory modality, visual speech leading (exp. 1) or lagging (exp. 2) acoustic speech, participants watched intact and blurred videos of a speaker uttering trisyllabic words and nonwords that were noise vocoded with 4-, 8-, 16- and 32-channels. They judged whether the speaker's mouth movements and the speech sounds were in-sync or out-of-sync. Individuals perceived synchrony (tolerated AVOA) on more trials when the acoustic speech was more speech-like (8-channels and higher vs. 4-channels), and when visual speech was intact than blurred (exp. 1 only). These findings suggest that enhanced spectrotemporal fidelity of the audiovisual (AV) signal prompts the brain to widen the window of integration promoting the fusion of temporally distant AV percepts.

• 出版日期2017