Visual Orientation during Game Play: Using Eye-Tracking and Conversation Analysis to Examine the Interactional Use of Eye-Gaze in Children with Autism Spectrum Disorders

Background: Children with autism spectrum disorders (ASD) are characterised as having challenges in the social use of eye-gaze (e.g., Chawarska, Macari, & Shic, 2013). The research on eye-gaze behaviours is often conducted in carefully controlled settings; less is known about the use of eye-gaze during naturally occurring interactions (but see Noris, Nadel, Barker, Hadjikhani, & Billard, 2012). Such examination can provide valuable insight into the contingencies under which social use of eye-gaze might occur in children with ASD.

Objectives: Our study utilises mobile eye-tracking technology to map the use of eye-gaze in a multiparty Kinect game playing environment. In this game, the children are gazing at a screen displaying their full-body movements. Rather than identifying children’s gazing practices within the game, the study focuses on instances of eye-gaze shifting away from the screen. The aim is to identify the location of such gaze shifts and what (if anything) is accomplished interactionally. The gaze shifts are mapped in relation to the events on the screen and the actions of the co-present adults. Thus, the study considers the wider interactions within the playroom, moving beyond the interactions between the user and the technology.

Methods: Seven children with ASD, aged six to 13 years, participated in the study. Our data corpus consists of approximately 8 hours of video material capturing the children playing Microsoft Kinect-based body movement games. The games required the children to independently catch virtual objects using their hands and feet in the presence of other people, namely, school staff members and researchers. Mobile eye-tracking glasses were used to objectively measure the eye-gaze of the children. Our analysis draws on multimodally informed Conversation Analysis (CA) (Sidnell & Stivers, 2005) to examine the video-recorded interactions. CA is concerned with the sequential organisation of interactions: how participants produce initiating (e.g., questions, requests) and responsive (e.g., answers) actions (Schegloff, 2007). Multimodally informed CA research considers how body movement and eye-gaze relate to the production of these actions, for example, how eye-gaze can pursue a response. CA uses detailed transcription of the occurring events, capturing talk, eye-gaze, and other bodily conduct of the participants. The present study extends prior CA work on eye-gaze in interactions (e.g., Korkiakangas & Rae, 2014) by using eye-tracking technology to map the children’s gazing activity during these interactions.

Results: The detailed examination showed that the children with ASD used eye-gaze for interactional work during the game play. The gaze shifts away from the screen and towards a co-present adult were used to pursue a response from the adult. The adults responded to these gaze shifts, indicating that they treated the child’s gaze aversion from the screen as an interactionally relevant action.

Conclusions: The study shows that children with ASD can demonstrate social orientation to others during a game play. The findings contribute to the few existing CA studies that have elaborated on the interactional use of eye-gaze in children with ASD. The combination of CA and eye-tracking suggests a new fruitful approach to study eye-gaze behaviours in naturally occurring contexts.