Measuring the Importance of Fyi Screener Items in Predicting Adost Totals at 12 and 18 Months: A Machine Learning Approach

Background:  The First-Year Inventory (FYI) parent questionnaire is designed to screen for autism (Reznick et al., 2007). Given the potential benefits of early intervention (Filipek et al., 1999), it is important to identify those active ingredients most informative of later outcome.

Objectives:  To identify FYI items from 12 months that predict levels of autism symptoms at 12 and 18 months measured by the ADOST algorithmic total, among high-risk (HR) infant siblings.

Methods:  Parents of HR siblings (N=76) completed the FYI at 12 months. Those children received ADOST evaluations at 12 and 18 months. We used random forest machine learning (#trees=500, #tries=35; similar to Wu et al. 2003) to measure the importance of individual FYI items, predicting ADOST algorithm totals at 12 and 18 months. Random forests measure the importance of each training variable in terms of resultant change in sample variance when moving that training item’s position within the step-wise predictive model. Items with larger changes are more important. We selected as important those variables whose scores affected variance at a higher rate.

Results:  Random forest variable selection isolated three FYI questions, important in predicting ADOST totals at both 12 and 18 months: Q37 (restricted interest in parts of toys, Sensory-Regulatory Functions—SRF—domain), Q49 (social interactive play, social communication—SC—domain), and Q24 (vocal imitation, SC). Two items were important for predicting 18 but not 12 month ADOST totals: Q57 (reactivity, SRF) and Q12 (attention to others, SC). Six items were important for predicting 12 but not 18 month ADOST totals: Q43 (stuck postures, SRF), Q48 (fixated interest in certain toys, SRF), Q6 (avoiding eye contact, SRF), Q11 (isolated play, SRF), Q45 (kicking, SRF), Q41 (feeding, SRF).

Conclusions:  We have applied machine learning to identify early screening questions that are most predictive of later autism symptom levels. In predicting ADOST total scores, the importance of an FYI item may reflect three properties: [1] prevalence of the behavior at 12 months, [2] good operationalization in the FYI and ease of evaluation of the behavior by parents, and [3] developmental stability of the behavior. Items that were important in either prediction may be prevalent and easily detectable at 12 months. Items Q37, Q49, and Q24 were important for predicting both 12 and 18 month ADOST totals, suggesting developmental stability, especially because these are closely related to core symptoms of ASD. Items Q57 and Q12 were important for predicting 18- but not 12-month ADOST totals, possibly indicating association with the development of more refined and advanced social communication behaviors (Q12), and with impairments in sensory-regulatory functions (Q57), that emerge between 12 and 18 months. These items may represent early, recognizable precursors of greater impairments that develop between 12 and 18 months. The six items that were important for predicting ADOST totals at 12 but not 18 months all fall within the sensory-regulatory function domain. These may be easier for parents to evaluate at 12 months than social communication behaviors. Their decline in importance at 18 months may reflect the specificity of those behaviors to an earlier developmental epoch.