Prospective Clinical Evaluation of a Machine-Learning Based Test for Rapid Detection and Triage of Autism

Background:

The incidence of autism has increased dramatically over recent years, making this mental disorder one of the greatest public health challenges of our time.  The standard practice of diagnosis is based on behavioral characteristics, as the genome has largely proved intractable for diagnostic purposes. Yet, the most commonly used behavioral instruments take up to 3 hours to administer by a trained specialist, contributing to the substantial delays in diagnosis experienced by many children, who may go undiagnosed and untreated until ages beyond when behavioral therapy would have had more fundamental positive impacts. In an effort to mitigate these challenges, we have developed a machine-learning based system for accurate classification of autism that requires minutes to administer and that can be delivered via mobile technologies. 

Objectives:

Enabled in part by the 2013 Slivka/Ritvo Innovation Award, we completed a clinical study at the Developmental Medicine Center at Boston Children’s Hospital.  The objective was to test the sensitivity and specificity of a new, electronically administered, 7-question autism spectrum disorder (ASD) screen to triage those at highest risk for ASD in a prospective clinical population of children at risk for autism and/or other developmental delays between the ages of 16 months and 17 years. 

Methods:

We created a mobilized web-accessible system for obtaining answers to a parent-directed classifier and a short home video of the child for machine classification. We administered the test in advance of the clinical team’s evaluation at the DMC (via iPhone, iPad, or personal computer). All subjects were recruited from a convenience sample of children referred for consultation of developmental/behavioral concerns. Once the predetermined sample size of 200 was exceeded, we abstracted data from the medical record to compare the best estimate clinical diagnosis against the outcome from the ML classification tool.

Results:  

A total of 222 families participated in the study, with a 69% rate of assent. The children assessed ranged in age from 1 to 16 years of age, with a median age of 5.8 years. 76.1% were male, and most participants had an intelligence/developmental quotient score >85; 69 of participants (31%) received a clinical diagnosis of ASD.  The sensitivity of the MCDC in detecting ASD was 89.9% [95% CI = 82.7-97]; the specificity was 79.7% [95% CI = 73.4-86.1].

Conclusions:  

Approaches that enable families to bridge the gap between initial warning signs of developmental delay and clinical diagnosis of autism quickly and effectively are critically needed for the field. The results from this clinical trial in a representative tertiary care facility suggest that the ML approach has clinical reliability across a range of ages and likelihood of high adoption by families. The study demonstrates the feasibility of accurate pre-clinical assessments and highlights the possibility of using mobile techniques for clinical triage, to reduce bottlenecks and reach a larger percentage of the population in need.