Use of On-Body Sensing and Computational Analysis to Automatically Detect Problem Behaviors

Background:  Behavior problems such as destructive behaviors, aggression, and self-injury are part of the clinical picture of autism (Hartley et al., 2008), and thus represent the targets of many intervention efforts (Horner et al., 2002). Accurate data on the frequency and intensity of these behaviors is key to understanding why and when they occur and tracking response to treatment. Current approaches to measurement include standardized checklists and direct observation. The former provide quick and cost-effective means of gathering data and are widely used in research settings, but do not capture precise frequencies of behavior. The latter yield rich data regarding the frequency and context of the behavior, but are time-intensive and thus difficult to implement across many settings and longer time scales.

Objectives:  To explore the potential for automatic analysis techniques to detect and classify severe problem behaviors using small, body-worn sensors (accelerometers) and computational analysis methods.

Methods:  We collected three 2-minute sessions of simulated data with experienced staff from a severe behavior treatment clinic. The staff wore accelerometer sensors on the wrists and ankles while engaging in three classes of problem behaviors: aggression (hits, pushes, kicks directed at another person present in the room), disruption (throwing objects/furniture, hitting/kicking walls and furniture), and self-injury (hitting self). A researcher reviewed the videos from the sessions and using coding software, marked the onset of each instance of a problem behavior. She further classified each instance according to one of the three main categories (aggression, disruption, self injury) and the limb involved (right vs. left arm and leg).

Results:  Data obtained from the sensors included 3-axis accelerometry (energy) and the orientation of the sensor. We applied machine-learning techniques to data from the four sensor streams (right and left ankle and wrist) to automatically: (1) segment events of interest (i.e., identify moments where movement was occurring versus not occurring), and (2) classify each event as falling into one of four categories: aggression, disruption, self-injury, and other (not problem behavior related). We then compared the results of the automatic classification against the human-coded data, separately for each limb. For segmentation, the automated method correctly identified 95% of events identified by the human coder. For classification of these events, the automated method differentiated among aggression, disruption, self-injury, and other with an accuracy of 72-74% for events involving the hands, and an accuracy of 94-97% for events involving the legs. 

Conclusions:  These preliminary results indicate that our automated methods attain high levels of sensitivity in detecting problem behaviors and good accuracy in differentiating among aggression, disruption, self-injury, and movements unrelated to problem behaviors. Our findings highlight the great potential for technology and computational methods to facilitate the gathering of objective, accurate measures of the frequency of problem behaviors in a range of contexts and over larger time scales. Data collection utilizing the accelerometers with children with autism currently undergoing treatment at the clinic is ongoing. The next set of analyses will evaluate the performance of our classification algorithms when applied to data collected from these children.