Comparing the Performance Characteristics of ASD Screening Measures in Toddlers

Background:  Although there has been growing emphasis on early screening and detection of ASD, the average age of diagnosis in the U.S. still remains between 4 and 5 years of age (CDC, 2012). In an effort to lower the age of diagnosis and take advantage of early intervention services, the American Academy of Pediatrics recommends universal screening for ASD beginning at 18 months of age. The most widely used screening measure, the Modified Checklist for Autism in Toddlers (M-CHAT), has been shown to identify many children with ASD at young ages, but often results in substantial over-identification of children with other developmental concerns particularly when clinicians do not utilize the embedded follow-up interview or other validation procedures (Chlebowski et al., 2013; Miller et al., 2011). This over-identification is particularly problematic given that most tertiary assessment centers struggle to provide diagnostic assessment without substantial waits.

Objectives:   To determine if a new screening instrument, the Vanderbilt Scales for ASD, utilizing graded parental ratings of early core symptoms of ASD, would possess improved performance characteristics over the M-CHAT in accurately identifying children with and without ASD. The Vanderbilt Scales for ASD consists of 23 items. These items were pulled from developmental checklists and interviews commonly used at Vanderbilt clinics. Those items with the highest weights, determined by a standardized canonical function, were chosen to be included in the Vanderbilt Scales. We also investigated the incremental clinical value of use of a structured observation of behavior in conjunction with the parent report screening measures.

Methods:  Participants included 161 children, ages 36 months or younger, participating in first-time diagnostic appointments or autism research protocols at a university based autism clinic. Examined measures included the M-CHAT, Vanderbilt Scales for ASD, and Autism Diagnostic Observation Schedule (ADOS). Three items from the ADOS that could potentially be easily administered by community clinicians were chosen as a structured observation of behavior. Sensitivity, specificity, and PPV were calculated for each screening measure as well as the simultaneous testing of each screening measure with the three ADOS items.

Results:   Reliability analysis of the Vanderbilt Scales for ASD yielded a cronbach’s alpha of .803. A cutoff score of 9 on the Vanderbilt Scales resulted in sensitivity= 80.65%, specificity= 66.18%, and positive predictive value= 76.53%. M-CHAT: sensitivity= 87.10%, specificity=52.94%, and positive predictive value= 71.68%. Screen positive on both the Vanderbilt Scales and ADOS items: sensitivity= 77.22%, specificity= 86.54%, and positive predictive value= 89.71%. Screen positive on both the M-CHAT and ADOS items: sensitivity= 75.28%, specificity= 83.61%, and positive predictive value= 87.01%. 

Conclusions:  The Vanderbilt Scales was slightly less sensitive than the M-CHAT, but resulted in a higher specificity and positive predictive value in this population. Simultaneous testing requiring positive screening on both the ADOS and screening measure resulted in a higher positive predictive value than when either respective screening measure was used alone. These results suggest further investigation into the use of a parent report measure, clinician follow-up, and structured observation of behavior to identify children in need of an autism specific evaluation.