Visual Fixation Patterns During Emotional Face-Voice Matching

Background: Individuals with autism spectrum disorders (ASD) have significant social communication deficits, particularly in the realm of non-verbal communication, such as maintaining eye contact, or the decoding of emotion from facial expressions and tone of voice, or prosody (Grossman et al. 2009, Shriberg et al. 2001). Eye-tracking data has also shown preferential gaze patterns to the mouth region of the face, rather than the eyes (Pelphrey et al. 2007).

Objectives: The purpose of our study was to use eyetracking to analyze the spatial and temporal looking patterns of adolescents with ASD and their typically developing (TD) peers in an emotional face voice matching task. Our hypothesis was that adolescents with ASD would show significant differences in the timing of gaze patterns to faces compared to their TD peers, and preferentially gaze to the lower, rather than the upper face.

Methods: We used eight semantically neutral sentences recorded in happy, surprised, sad, and angry emotions with high and low emotional intensity (Grossman et al. 2009). The 64 prosodic stimuli were presented to 25 adolescents with ASD and 25 TD controls. Participant groups were matched on age (mean = 12), IQ, and receptive vocabulary. After each sentence, participants saw two static facial expressions side-by-side on a computer screen. Their task was to determine which of the two faces was more likely to have spoken the preceding sentence. One facial expression in each pair matched the emotion and intensity level of the sentence. The other also matched on intensity level, but represented either an emotion with opposite valence (e.g. angry sentence with happy and angry facial expressions) or the same valence (e.g. angry sentence with sad and angry facial expressions).

Results: We recorded the percent of looking time to Area of Interest (AOIs) for trials with accurate responses and calculated a 2 (group) x 6 (AOI: upper face, lower face, eyes, mouth, face, and non-face) repeated measures ANOVA. Results show no group differences for overall looking time to any AOI and both groups spent significantly more time looking at the eyes than the mouth. However, the participants with HFA shifted their gaze from the fixation point located between face images to the non-face area of the correct image significantly later (p = .008) and also accumulated significantly more fixations to the non-face (p = .01) than the TD group.

Conclusions: Our data indicate that adolescents with ASD allocate overall looking time similarly to their TD peers during a face-voice matching task. They do not avoid the eye region or preferentially gaze at the mouth area of the face. There are, however, significant timing differences between the gaze patterns of the two groups. Adolescents with HFA shift their gaze from a central fixation location to the non-face of the correct image significantly later than their TD peers and return their gaze to the non-face significantly more frequently. Future analyses of gaze patterns to social stimuli in ASD should focus on timing, in addition to purely spatial differences.