Development of a Diagnostic Algorithm for the PDD Behavior Inventory Based on Classification Trees

Background:  The PDD Behavior Inventory (PDDBI) is a reliable and valid assessment tool that has been shown to yield excellent sensitivity and specificity for children whose diagnoses were confirmed by the ADOS-G and ADI-R (Cohen et al. 2010).  Recently, we have shown that the PDDBI can differentiate children with ASD from those with Attention Deficit Hyperactivity Disorder using a Classification Tree, a non-parametric type of machine learning (Cohen, 2013). Due to these results and need for Level 2 screeners for ASD, we explored the use of Classification Trees as a means of developing a diagnostic algorithm for the PDDBI.  

Objectives:  To develop a reliable and valid diagnostic algorithm for the PDDBI.

Methods: To date, 649 parent and 202 teacher PDDBI forms have been collected from the NYS Institute for Basic Research (IBR), and from Queens University.  Approximately 83% of cases have been diagnosed with "ASD" with the remainder (“OTHER”) having ASD ruled out as a diagnosis after extensive clinical evaluation or identified by parents as unaffected siblings or who were toddlers taking part in a longitudinal investigation of at-risk infants.

The Classification and Regression Trees module (Statistica, Version 12) was used to develop the algorithm. Inputs included the Repetitive, Ritualistic and Pragmatic Problems Composite T-score, the Approach-Withdrawal Problems Composite T-score, the Expressive Social Communication Abilities Composite T-score, the Autism Composite T-Score, the Social Discrepancy score, and the Semantic-Pragmatic Problems Discrepancy score. Sixty percent of the dataset was used for training and 20% for testing during the development process. The remaining 20% (the “validation set” not used in the model development process) helped validate the final model.

Results:  A number of models were explored and yielded similar results.  The selected model divided the ASD sample into two parts, a “typical” ASD group and a “high social-functioning” ASD group while the OTHER sample was divided into three groups, a relatively unaffected group, and two smaller sets: 1) a “rigid” group, and 2) a “severe behavior problem” group.

The two ASD groups differed on IQ, Vineland, and ADOS severity scores; parent reports of seizures; and association with a gene polymorphism linked to autism severity (Cohen et al. 2003).  

Sensitivity and Specificity were 83% and 87% for the training set, 86% and 81% for the test set, and 82% and 81% for the validation set. Sensitivity and specificity were 80% each for cases <4 years and 85% and 89%, respectively, for cases >4 years.

Overall agreement between parent and teacher global algorithm diagnoses (ASD vs OTHER) was 78% (Kappa = 0.47) and was 66% (Kappa = 0.52) for the more fine grain groupings. Using only cases in which parent and teacher forms yielded identical groupings, increased sensitivity and specificity to 90% each.

Conclusions:  Results confirm previous studies suggesting two forms of ASD, a classic presentation (often associated with intellectual delays and seizures), and a group with better social and language skills having a more optimal outcome.  These results  suggest that the PDDBI can serve as a useful Level 2 screener.