Surprising Events within a Virtual Environment: A Catalyst for the Initiation of Spontaneous Social Interactions by Children with ASD

Background:  Children with ASD are known to have problems with social communication skills. Virtual environments (VEs) and virtual characters (VCs) have been identified as a promising support tool: they can be highly structured and predictable, and potentially less threatening than direct interaction with a human partner. Little is yet known about which specific virtual elements may best support social communication, especially spontaneous initiation (e.g. sharing interest, affect, and events) which is generally agreed to be disproportionately difficult for this population compared to responding to others’ initiation; its absence is an important diagnostic criterion (DSM-IV, 1994).

Objectives:  The ECHOES technology-enhanced learning project has developed a multi-modal, touch screen based, virtual learning environment to support social communication in young children (aged 5-8) with and without an ASD (see www.echoes2.org). A childlike VC models game-like activities in a ‘Magic Garden’ setting, collaborating with the child to complete them. Activities emphasise joint attention skills and the initiation of social interactions.

Methods:  15 children with ASD participated in the formative evaluation of early ECHOES activities. A further 28 children with ASD from 4 UK schools participated in the summative evaluation, completing multiple 10-20 minute sessions with ECHOES. A researcher at a side screen controlled transitions between activities, gave support (e.g. clarifying task instructions), and could be an additional social partner. The ECHOES touch-screen with VC, child and researcher were videoed in order to capture their social interactions. 

Activities deliberately introduced novel elements and behavioural fantasy, such as “pulling” flowers to transform them into bubbles, or visual “fireworks” rewarding task completion. Intermittent software errors yielded unintentional surprises by altering the environment’s customary behaviour and violating child expectations. For example, the VC might correctly demonstrate a sorting activity, but later try to put an item in the wrong box. Balls usually bounced within the screen, but a specific touch action sent them soaring off-screen instead.

Results:  Researchers observed that novelty and surprises frequently resulted in spontaneous child initiations towards the VC or researcher. These ranged from sharing gaze and positive affect to overtly directing the partner’s attention and/or commenting. Novel elements eventually stopped eliciting initiations, but unplanned surprises—particularly regarding VC behaviour—continued to elicit child interest and initiations even over multiple sessions. Especially noteworthy were instances where children spontaneously indicated the correct action to the VC after his mistake or told him to “try again.”

Conclusions:  The rigidity of thought and desire for routine which characterise ASDs might yield predictions that novelty and surprises within VEs would be upsetting. Instead, they have repeatedly catalysed initiations with human and virtual social partners. The “boundedness” of the VE may be a factor in these events being perceived as fun, rather than as threatening disruptions. Building expectation-violating events into future VEs could be a tool for supporting spontaneous and positive social initiations from children with an ASD, and generally heightening interest and engagement.