Evaluation of the Diagnostic Accuracy and Reliability of the Pediatric Autism Early Diagnostic (PAED) Assessment: A New Tool for Community-Based Pediatricians

Background:   Practice guidelines emphasize using standardized diagnostic tools with high specificity and sensitivity to directly assess features of Autism Spectrum Disorders (ASD) during diagnostic assessment. In an effort to construct an assessment for community pediatricians, our team developed the Pediatric Autism Early Diagnostic tool (PAED). Its six activities (free play, response to name, bubbles, snack, pretend play and social games) were modified from the Autism Diagnostic Observation Schedule (ADOS), a well-validated diagnostic tool for Autism. ADOS administration requires 30-60 minutes and standardized equipment. The PAED takes 15-20 minutes and uses readily-available equipment. Information from the PAED Observation Form is mapped by the examiner onto a 12-point DSM-IV criteria scoring form for diagnostic classification

Objectives:  To compare the PAED against the validated ADOS Module 1 for diagnostic accuracy with pre-verbal children, and evaluate PAED scoring reliability between/within raters.

Methods:  Twenty eight pre-verbal children (mean age 33 months) referred to one of three child development clinics with suspected Autism, language/social concerns, and/or repetitive behaviours received both a PAED and ADOS by independent clinicians prior to the developmental history/physical examination being completed. The PAED was administered and scored by one of four raters (child development practitioners) who received PAED training by an ADOS trainer. None of the PAED raters were ADOS trained. After the rater completed PAED and DSM-IV scoring, a diagnostic category of ‘Autism’, ‘PDD-NOS/ASD’ or ‘Not Autism (NA)’ based on DSM-IVTR criteria was assigned.  PAED videotapes were independently scored by two raters to evaluate intra/inter-rater reliability. The ADOS Module 1 was administered and scored at a separate visit by a different ADOS reliable rater (blinded to PAED results). Following ADOS assessment, the rater scored the DSM-IV, and assigned a diagnostic category as above.

Results:  Following ADOS assessment, 6 children were classified as NA, 3 children with PDD-NOS, and 19 children with Autism. Following the PAED, 8 children were classified as NA, 4 children with PDD-NOS, and 16 children with Autism. ADOS and PAED classification agreement was excellent: weighted kappa (K)=0.83. Category agreement was lowest for PDD-NOS (K=0.51). Sensitivity and specificity were from 91%-100% for Autism/PDD-NOS versus NA and Autism versus PDD-NOS. Intra-rater reliability (n=28) of the PAED DSM-IV score was high when done from video-viewing (ICC = 0.92; 95%CI =0.83-0.96), and moderate when live versus video-viewing was evaluated (ICC=0.63, 95%CI=0.34-0.81). Inter-rater reliability (n=16) was excellent when two raters not involved in initial live rating scored from video (ICC=0.86, 95%CI=0.65-0.95).

Conclusions:   When used alongside the history/physical examination, the PAED demonstrated strong ability to classify children correctly and holds promise as a shorter but still reliable observational tool for community pediatricians who would not do a full ADOS. Further testing is needed to confirm results with a larger sample and more verbal children.