A Dynamic Approach to the Audiovisual Integration of Socio-Linguistic Information

Background:  

Individuals with ASD show deficits in the integration of audiovisual information, specifically in regard to temporal synchrony in speech (Bebko et al., 2006). Children with ASD perceive temporally discrepant audiovisual information as synchronous twice as often as typically developing (TD) children (Foss-Feig, et al., 2010).  One way of measuring sensitivity to temporal synchrony is through the temporal binding window (TBW), the range of temporal offsets within which an individual perceptually binds inputs across modalities. Wider TBWs result in inadvertent associations between sights and sounds in the day-to-day environment, leading to chaotic representations of the world.

TBWs vary across experimental tasks and type of stimuli (e.g., speech, non-speech; Vroomen & Keetels, 2010). Common strategies to measure the TBW include synchrony judgment (SJ) tasks and temporal-order judgment (TOJ) tasks, both of which depend on passive cognitive judgments (i.e., dichotomous choice). In this study, a novel task was developed to elicit active synchrony manipulation, allowing participants to manually adjust the audio track of an audiovisual event in decisions regarding synchronicity.

Objectives:  

Characterize audiovisual integration in TD adults and adults with ASD. Specific objectives:

• Explore the relationship between audiovisual integration and characteristics of ASD.
• Examine the temporal binding window using a novel, participant-manipulated task and compare to a standard SJ task.
• Determine how stimulus content influences the size and symmetry of the TBW. 

Methods:  

• Participants: Data for 30 TD adults have been collected and analyzed. Data collection for the ASD sample is partially complete and in progress.
• Stimuli:
  • Social Linguistic (SL), Non-Social Non-Linguistic (NSNL), and Social Non-Linguistic (SNL)
• Measures:
  • A participant-manipulated, audiovisual task allowing participants to manually adjust synchrony from a point of extreme audio delay or advance to a point of perceived synchrony.
  • A synchrony judgment task (yes-no) in which participants judge the synchronicity of video clips with varying degrees of audio delay and advance.
  • A combination of questionnaires (Autism Spectrum Quotient; Empathy Quotient; Baron-Cohen, et al., 2001), diagnostic reports, and diagnostic assessments (i.e., Autism Diagnostic Observation Schedule) to measure degree of autism-like traits in both samples.

Results:  

1. Temporal Binding. Analyses revealed significantly larger TBWs in the participant-manipulated task compared to the SJ task with NL (t=8.24, p<.001) or SNL (t=5.68, p<.001) stimuli. However, for linguistic stimuli, no difference between tasks suggests relative stability in processing of speech information across tasks.
2. Audiovisual integration and ASD. Preliminary analyses in the ASD sample indicate larger TBWs across all three trial types (SL, SNL, NSNL) compared to TD participants.
3. Autism traits and TBWs. For TD participants, the AQ Social Skills subscale was correlated with TBW width for the linguistic (r=.402, p=.03) and social non-linguistic (r=.458, p=.01) conditions. ASD data are being analyzed. 

Conclusions:  

This novel task provides a new perspective on audiovisual integration by allowing individuals to actively control and monitor synchrony versus simple yes-no responding to experimenter-defined stimuli. Correlations between autistic traits and TBWs provide novel evidence that sensory-processing deficits contribute to core deficits including linguistic and social skills.