20287
Promises and Limits: Exploring Relationships Between the First Year Inventory (FYI) and Autism Diagnostic Observation Schedule (ADOS) from 12 to 18 Months of Age with Machine Learning
Objectives: To evaluate the performance of machine-learning techniques for (1) using the First Year Inventory (FYI), a parent questionnaire given to parents of children 12 months of age, to predict levels of autism symptoms as given by the ADOS-Toddler Module (ADOS-T) at 12 and 18 months; and (2) using ADOS scores at 12 months to predict scores at 18 months.
Methods: Based on 76 HR children, we first examined Pearson’s correlation between the FYI totals at 12 months (FYI.12) and ADOS algorithm totals at 12 (ADOS.12) and 18 months (ADOS.18). Then, we used two widely-used machine-learning techniques, support vector machines (SVM) and random forests using variable selection, for the same predictions with all items from both measures (FYI.12–ADOS.12 and FYI.12-ADOS.18). The same methods were repeated to examine the correlation between ADOS.12 and ADOS.18. For machine-learning techniques, we performed 1000 repeated random sub-sampling validations, utilizing 80% of data for training and 20% for testing. We also report results while using all data as training sample [reported in [brackets]).
Results: We found a moderate correlation between FYI.12 and ADOS.12 (r=0.38). Both SVM and random forest resulted in comparable correlations, r=0.47[0.91 for training sample] and r=0.51[0.96] respectively. We found a milder correlation between the FYI.12 and ADOS.18 (r=0.13), but the SVM and random forest improved the correlations significantly, r=0.52[0.94] and r=0.58[0.98] (differences significant at p’s<0.01). Not surprisingly, we found a moderate correlation between ADOS.12 and ADOS.18 (r=0.53). The SVM and random forest resulted in comparable correlations, r=0.61[0.88] and r=0.63[0.96].
Conclusions: As the FYI is designed to capture current developmental profiles at 12 months, it was already moderately associated with concurrent clinical presentations measured by the “gold standard” clinician observations (ADOS) without implementing machine-learning techniques. Similarly, as the ADOS is designed to assess core autism symptoms that are fairly stable over time, the use of the ADOS algorithm was sufficient to predict clinical outcomes at 18 months. However, the prediction of clinical outcomes measured by clinician observations while using parent reports obtained 6 months earlier was more challenging. For this prediction, machine learning resulted in improved correlations, suggesting that it can be especially helpful when adaptations to measures are needed to make predictions that go beyond what they are designed for. Therefore, understanding the role of machine learning to add to ongoing explorations regarding measurement in ASD requires care, collaboration, open discussion, and humility regarding strengths and limits of statistical tools.
See more of: Diagnostic, Behavioral & Intellectual Assessment