A New Scoring Algorithm of Autism Diagnostic Interview-Revised (ADI-R) Using Signal Detection Theory Based Analysis

Background: The Autism Diagnostic Interview-Revised (ADI-R; Lord et al., 1994) is a standardized, semi-structured, investigator based interview for parents or caregivers of individuals referred for a possible Autism Spectrum Disorder (ASD).  Diagnostic algorithms for the instrument classification of autism and ASD have been developed based on item response models on individual items and on performing Receiver Operating Characteristic (ROC)  analysis on overall algorithm scores (Rutter et al. 2003, Kim & Lord, 2011). Following previous statistical approaches, here we applied Signal Detection Theory (SDT) analyses at individual item level in order to generate better discriminability measure and to determine each item’s contribution towards overall diagnostic sensitivity and specificity.

Objectives: SDT, including ROC analysis, offers a standard paradigm for determining instrument sensitivity and response bias. Our goals were to use SDT (with ROC) to confirm diagnostic validity of ADI-R algorithms and to explore alternative methods of generating the instrument classifications of autism and/or ASD. 

Methods: We selected those items (a total of 30) in the ADI-R on a 0- to 3- point scales (with higher scores indicating more severe impairment). For each item, we constructed distributions of scores for both the Autism/ASD (“A”) group and an appropriately matched Control (“C”) group that included children with typical development (TD) and non-spectrum developmental disorders (NS). From the two distributions, we constructed an ROC curve and the likelihood ratio values “L” at each scale point (i.e., 0, 1, 2, and 3). In SDT, an ROC curve depicts the tradeoff of false-positive (Type II error) and false-negative (Type I error) caused by shifting the diagnostic criterion (threshold on the scale), while the Area under ROC Curve (AUC) is used as a measure of item discriminability. Since an ROC curve is invariant against a monotone transformation of the likelihood threshold (see Zhang & Mueller, 2005), we used L/(1+L) and 1/(1+L) to convert a subject’s item scores into relative likelihood values for both A and C groups. The sum of the item-by-item relative likelihood values yielded an ASD Tendency Score (ATS) to be used for ASD classification. Preliminary test for this method was done for 381 children with ASD, 63 children with NS (e.g., language delays, intellectual disabilities), and 52 TD children from 12 to 47 months of age. ATSs were calculated either using the aforementioned 30 items in ADI-R or using a subset of 13 items from existing “12-20/NV21-47” algorithm (See Kim & Lord, 2011).

Results: Correlations (Pearson r) between 13-item and 30-item ATSs algorithm scores ranged between 0.91 and 0.96 respectively. AUC for individual items ranged from 0.8-0.99, confirming high item discriminability. AUC for Autism/ASD diagnosis based on 13-item or 30-item ATS were compatible (0.978 and 0.980) to algorithm totals (0.979), confirming ADI-R validity. 

Conclusions: New scoring method for the ADI-R generated based on a signal detection theory analysis demonstrated strong correlations with existing ADI-R algorithms, confirming their diagnostic validity. Results on a larger sample will be examined and reported.