Validation of an Observational Screening Measure of Red Flags of ASD at 18-24 Months

Background: The American Academy of Pediatrics recommends that all children be screened for autism spectrum disorder (ASD) at 18 and 24 months (Johnson & Myers, 2007), as research indicates early intervention maximizes child outcomes (NRC, 2001; Dawson et al., 2011; Wetherby et al., 2014). Screening tools assessing risk for ASD should provide a time- and cost-efficient method of determining whether further diagnostic testing is warranted. The updated Systematic Observation of Red Flags (SORF-22) is a 22-item coding system recently revised to detect symptoms outlined in the new diagnostic criteria (as described in the DSM-5), providing a brief screening measure covering both core domains of ASD.

Objectives: To determine the sensitivity and specificity of the SORF-22as a screening tool for ASD symptoms at 18-24 months and to identify a cutoff to maximize the efficacy of its use in detecting risk for ASD.

Methods: 18-24 month old toddlers were selected from (1) a primary care sample screened for communication delays, and (2) a sample referred for developmental concerns. Data for the present study were coded based on behavior exhibited in the Communication and Symbolic Behavior Scales Behavior Sample, a standardized clinician-administered measure of social communication.

Outcome measures of developmental level (Mullen Scales of Early Learning; Vineland Adaptive Behavior Scales, Second Edition) and autism symptoms (Autism Diagnostic Observation Schedule – Toddler Module)were obtained to determine diagnosis at 18-24 months. Children with a confirmed diagnosis were classified into one of three groups (ASD, Developmentally Delayed [DD], or Typically Developing [TD]) based on a best estimate diagnosis using these measures. 

Receiver operating characteristic (ROC) curve analyses were used to determine sensitivity and specificity of the SORF-22, examining the total score of overall symptoms and the total number of red flags in predicting diagnosis. Further analyses will also be reported, utilizing Item Response Theory techniques to examine item-level performance in order to determine whether fewer items are needed to efficiently measure ASD risk.

Results: The total number of red flags offered the best prediction of diagnosis, providing good discrimination (area under the curve = .80) when comparing non-spectrum children (DD and TD) and children with ASD. Maintaining sensitivity above 80% (i.e., 82%) resulted in a specificity of 63% when using a cut-off score of six red flags. Using a cutoff of 17 for the total SORF-22score provided similar discrimination (area under the curve = .78), sensitivity (82%), and specificity (60%).

Conclusions: The SORF-22 provides a measure of ASD risk with good discrimination when comparing children with ASD and children who are TD and DD at 18-24 months. Prioritizing sensitivity above 80% results in lower specificity, though it reduces the likelihood that children with developmental concerns will be overlooked for further diagnostic testing. Additional analysis will also be discussed, examining symptom scores individually to determine which items are most effective in predicting diagnosis.