Emotion production and perception are a crucial part of human interaction. Emotions modulate vocal, facial, and physical expressions in natural communication. Previous research has demonstrated that children with autism have difficulty recognizing and understanding emotions. Quantitative evaluations can aid in identifying child-specific social deficits, potentially producing, in the long-term, more effective and targeted interventions. Embodied Conversational Agents (ECA) provide an important platform for analyzing a child's communicative patterns. ECAs produce consistent and modifiable scenarios permitting a controlled evaluation of a child's communication abilities. The goal of the current work is to determine how a suite of ECA technologies can be used to elicit and evaluate natural affective child communication patterns.
Objectives:
The ECA employed in this work is designed to elicit, and analyze, complex, but phased, structured, and naturalistic interactions. The tool is intended to encourage affective and social behavior through the design of the accompanying scenarios.
Methods:
The tool is adapted from earlier studies at the USC. These studies suggested that an ECA could elicit natural communication patterns in typically-developing children and children with autism. In the updated design, the interaction scenarios are emotion problem-solving tasks. In these tasks, children are presented with emotional imagery. They are asked to identify emotional inconsistencies (e.g., missing faces, mismatched faces) and explain why these inconsistencies were identified.
The ECA serves as an emotional coach. It introduces the scenario and produces queries whose difficulties are dependent on the child’s ability. If the child easily identifies the emotions in the images, the ECA coaches him through low-level points, asking him to determine the emotional causes. If the child is unable, or unwilling, to identify the emotions present, the ECA coaches him through empathetic exercises, relating the child's preferences to the presented scenarios.
The tool is designed to elicit affective and social behavior for analysis. The child’s behaviors are recorded using audio-visual sensors and the ECA’s prompts are logged for post-hoc analysis. The interaction follows the Wizard-of-Oz paradigm, permitting the creation of a structured interaction simulating a discourse between a child and machine while avoiding the technological difficulties inherent in speech understanding.
Results:
The ECA technologies discussed in this abstract have been designed, implemented, and tested on typically developing children. Previous studies demonstrated the usability of this type of technology on typically developing children and children with autism. We are currently recruiting children with autism to study affective child-machine interaction patterns. The presentation at the conference will include a demonstration of the technology.
Conclusions:
ECA technologies provide an effective platform for eliciting and analyzing children’s communicative abilities. This tool is a modular system; scenarios can be interchanged or even created based on the needs of the experimental protocol. It also records time-stamps of the ECA behavior. The combination of the modularity and detailed record facilitates detailed and quantitative post-hoc analyses for the identification of social affective deficits in children with autism. This work is supported by the National Science Foundation and Autism Speaks.