AMA, a Tool for Annotation, Monitoring and Analysis of Behavioral Activity

Friday, May 18, 2012
Sheraton Hall (Sheraton Centre Toronto)
11:00 AM
J. Hernandez, A. Sano, M. S. Goodwin and R. W. Picard, Media Lab, Massachusetts Institute of Technology, Cambridge, MA
(First and second authors contributed equally to this work)

Background: One of the most disruptive features of Autism Spectrum Disorders (ASD) is the occurrence of behaviors such as self-injury, tantrums and persistent hand flapping. Although some research has suggested that these behaviors are aimed to reduce stress levels, to increase attention or to avoid certain events, there is little research on how problem behaviors are related to the daily environment of each individual. The definition of this relationship is critical for designing personalized behavioral interventions and, consequently, improving the quality of life of people.

In order to understand the behaviors of people with autism, annotation has sometimes been conducted by teachers, therapists and family; however, traditional annotation methods are based on the use of pen and paper. While this methodology is flexible and inexpensive, it is very slow and prone to human error. To address these problems, more automatic annotation tools such as CareLog and BabySteps have been used successfully in controlled environments (e.g., classroom, home). However, these systems cannot easily be deployed in different settings and modified to incorporate new behavioral annotations.

Objectives: We propose AMA, a new software platform that runs on Android (widely available on low-cost tablets or smart phones) to Annotate, Monitor and Analyze the occurrences of problem behaviors easily in real-life settings. This poster describes the Annotation tool that allows teachers, staff, and family members to record multimodal information (videos, audio and images) as well as the occurrence of behavioral annotations (e.g., self-injury, tantrums) to better capture problem behaviors in the field, at the moments they occur.

Methods: We iteratively designed and developed the annotation tool incorporating the advice of behavioral scientists and therapists of the Groden Center in Rhode Island, a non-profit school for people with ASD.  We conducted a two-level evaluation: one with users at MIT verifying that the technology worked as designed, and another with therapists assessing usability while working with children with ASD.

Results: We identified and built technology to achieve the following design needs:

  • Speed: The tool can quickly annotate events with one or two clicks.
  • Customization: The interface and types of annotations can be easily customized by the user.
  • Portability: The software installs easily on and is compatible with today’s Android devices (cell phones, tablets) to annotate events in real-life settings.
  • Cost: The distribution of the software is free through Google's Android Market.

The poster will describe evaluations of the technology, identifying what worked well and what improvements are still needed.

Conclusions: With the collaboration of behavioral scientists and therapists, we have designed and developed a tool for easily annotating problem behaviors in real-life settings. Main capabilities include the possibility to capture multimodal information, and the flexibility to create new types of annotations which are not planned in advance. During the poster session, we will provide details about the design process, qualitative evaluation and iterative development, and a working demo of the application. Moreover, we will create a website to share the application with the entire community.

| More