Eyes up: Interactive Tangible-Digital Puzzles for Learning about Eyes

Background: Autism is a complex and heterogeneous condition that is typified by difficulty with social and emotional cue recognition and expression. These social-emotional processing differences make recognizing subtle social-emotional cues difficult for persons diagnosed with Autism Spectrum Disorder (ASD). In particular, people diagnosed with an ASD typically perform worse then neurotypical (NT) individuals in recognizing facial expressions, especially when recognition is dependent solely on information surrounding the eyes. This may be due, in part, to an aversion to eye contact and/or atypical scanning behavior of the face region by those with ASD. However, eyes provide a valuable source of social-emotional information during interaction. Therefore, it is important that we address these difficulties in recognizing these valuable signals by developing methods and tools to help people better recognize and understand these social-emotional cues.

Objectives: Our goal is to develop a physical, tangible-digital puzzle game, Frame It, that can be used by any child, with the purpose of facilitating teaching or therapy through playful interaction while recording play characteristics. In particular, we seek to develop a game that will be able to affect the visual-scanning behavior and expression recognition ability of children diagnosed with ASD. Frame It requires the player to construct a puzzle of a person's eye region and then assign an expression label to that region.

Methods: The development of Frame It was approached using a user-centered design methodology that included consulting with users, teachers, therapists and educational experts. Design and testing sessions were conducted with both ASD and NT children. We additionally conducted usability sessions with eight ASD and eight NT children, measuring several aspects of their experience with the novel game.

Results: Usability tests showed that Frame It was accessible by both groups, and interaction was enjoyable. Because of the game's ability to record play interaction, we were also able to gain quantitative insight. We found a significant difference in the number of pieces used while attempting to construct the correct puzzle from a set of multiple puzzles. We also found a significant difference in the spatial ordering of pieces during construction. In addition, we were able to see a significant difference in correct expression labeling of the eye region, with the ASD group having more mislabels. Another interesting result found was that the ASD group completed more puzzles and had a lower average puzzle construction time than the NT group, despite manipulating more pieces during construction. This may suggest that the two groups used different strategies for solving the puzzles.

Conclusions: The user-centered design approach allowed for the development of a system accessible by a wide range of users, with varying physical and cognitive abilities. In addition, the designed system allowed for, and recorded, playful interaction. The recorded data set was used to reveal significant differences between groups, and highlights the usefulness of physically embedded computational devices. Future studies will explore the use of Frame It as a tool to affect the visual-scanning behavior and expression recognition ability of children with respect to other people's eye region.