Examining the Cross-Cultural Accuracy of Three Early Autism Screening Instruments

Background:  Autism Spectrum Disorder (ASD) is often diagnosed later than the time at which it can be reliably identified.  Differences exist between minority racial and SES groups with regard to age at first diagnosis of ASD.  Population-wide screening for ASD has been recommended to aid in early identification as well as to reduce disparities in timing of diagnosis, particularly for underserved groups.  Although findings related to the psychometric properties of ASD assessment instruments are fairly well established, little information is available regarding use of ASD assessment instruments with culturally diverse populations. 

Objectives:  Investigators documented accuracy of three commonly used screening instruments to detect ASD within a referred sample.  Investigators compared scores across three screeners as well as compared their accuracy in correctly detecting ASD.  The overall goal of the investigation was to compare the validity of three early ASD screening instruments across cultural groups to inform clinical use of the instruments with diverse populations. 

Methods:  Social Communication Questionnaire (SCQ), Modified Checklist for Autism in Toddler (M-CHAT; Total Score and Critical Item Score), and Pervasive Developmental Disorders Screening Test-II (PDDST-II) parent ratings were collected for 121 children (M age = 3.74 years, SD = 1.14) completing a comprehensive diagnostic evaluation between 2010 and 2012.  Comprehensive diagnostic evaluation included use of the Autism Diagnostic Observation Schedule (ADOS) and ADOS-Second Edition (ADOS-2) and Childhood Autism Rating Scale-Second Edition (CARS-2); final clinical diagnosis was established via team consensus.  A total of 70 children met criteria for ASD (57.9%); 51 children met criteria for another disorder (42.1%).  Screeners were completed in counterbalanced order and clinicians were blind to the results of the screeners.  Correlations between test scores were calculated and test accuracy was calculated via area under the curves (AUCs) derived from receiver operating characteristic analysis.  AUCs were compared for the entire sample and across race groups (i.e., Caucasian, African-American, Other) and maternal education (i.e., No High School Graduate, High School Graduate, Some College or Higher) or each screener via z tests.

Results:  All screeners were positively correlated at the p < .01 level with Pearson correlations ranging from .42 to .85.  For the entire sample, AUCs ranged from .46 (PDDST-II) to .66 (M-CHAT-Critical Items; see Figure 1).  M-CHAT-Total and M-CHAT-Critical Item AUCs were significantly greater than the SCQ AUC (z = 2.29, p = .02 and z = 3.08, p =.002).  The M-CHAT-Critical Item AUC was significantly greater than the PDDST-II AUC (z = 2.14, p = .03).  For African-American children, the M-CHAT-Total was more accurate when compared to children of Other races (z = 2.14, p = .03).  No differences in accuracy were found across maternal education groups. 

Conclusions:  In general, screeners showed poor accuracy (i.e., AUCs = .46 - .66) for discriminating between children with and without ASD in a referred sample.  The M-CHAT performed significantly better than the SCQ and PDDST-II screeners.  Few differences were noted across race and SES groups.  The findings should be extended to population-based screening as the present sample consisted of a referred group of children.