Comparing ASD Screening Measures for Toddlers and Preschoolers

Background:  With burgeoning waitlists for comprehensive diagnostic assessment of ASD, finding accurate methods and tools for advancing diagnostic triage becomes increasingly important. The current study compares the efficacy of three oft used paper and pencil measures, the Modified Checklist for Autism in Toddlers Revised with Follow-up (MCHAT-R/F), the Social Responsiveness Scale, Second Edition (SRS-2), and Social Communication Questionnaire (SCQ), to a novel measure developed by Cognoa, Inc. (Cognoa) using machine learning technology. Cognoa incorporates parent-report questions along with ratings of short video segments collected via parent smartphones and was designed to cover an age range from toddlers to preschoolers.

Objectives:  To compare the sensitivity and specificity of four separate ASD screening measures in the same group of toddlers and preschoolers referred for a question of autism.

Methods:  Data were collected from 230 children (M age = 40.3 months, SD = 14.9, range = 18.2 to 71.9) who were referred to one of three autism centers in the United States for a diagnostic question of ASD. Each participant completed the Cognoa screener and the other screening instruments (appropriate to age) prior to their scheduled appointment. Clinicians were blind to the Cognoa results. As part of a comprehensive developmental assessment, all participants completed traditional measures of cognitive abilities, adaptive functioning, and ASD symptoms (ADOS-2). Clinical Best Estimate Diagnosis was assigned based on DSM-5 criteria.

Results:  Receiver operating characteristic (ROC) curves were plotted for each measure in their respective age groups along with sensitivity and specificity calculations. The results were as follows (at the suggested cutoffs for each measure): MCHAT R/F sensitivity = .87, specificity = .40; SCQ sensitivity = .63, specificity = .40; SRS-2-Pre sensitivity = .66. specificity = .42; SRS-2-SA sensitivity = .87, specificity = .26; Cognoa sensitivity = .75, specificity = .62.

Conclusions:  Overall, Cognoa appears to have the strongest balance between sensitivity and specificity. This pattern remains in each measure’s respective age group. For example, in the 18–30 month age range, Cognoa’s optimal point was a sensitivity of .71 and specificity of .60 compared to MCHAT R/F’s .87 and .40. Results suggest that combining traditional parent-report questions with clinician-scored video samples may result in improved screening performance for ASD. The current sample is a clinical population of 1.5 through 5 year olds all referred for a question of ASD; thus, the psychometric properties of the screening measures may be impacted by a greater presentation of symptomatology compared to a more general population sample (e.g., reduced sensitivity and specificity). Given that Cognoa spans the entire age range in question, and in the context of its relatively stronger and better balanced sensitivity and specificity, it may be more pragmatic for clinical use. A post-hoc revision of the Cognoa algorithm resulted in further improvements in sensitivity and specificity, particularly among younger children aged 18 to 30 months. Overall, results suggest that combining parent report and behavioral coding may result in improved performance in screening measures for ASD. Future studies are planned to prospectively evaluate the revised algorithm.