An Item Response Theory Analysis of the Childhood Autism Rating Scale, Second Edition (CARS-2)

Background: Evidence-based autism-specific rating scales yielding valid and reliable scores are often desired in combination with gold standard measures to render the diagnosis of ASD.  The Childhood Autism Rating Scale, Second Edition (CARS-2) is an autism-specific rating scale that developers propose is effective in discriminating between children who have autism and those who have severe cognitive deficits. Previous studies examining the CARS have used Rasch scaling; however, there are no studies of CARS-2 psychometric properties and validity using similar techniques.

Objectives: Using a clinic-based secondary dataset, investigators used IRT techniques to examine the distribution of items across the ASD continuum as well as to determine the level of measurement precision across the construct continuum in a sample of participants with ASD, intellectual disability (ID), or ASD with comorbid ID.

Methods: A total of 173 participants (81.5% male; M_age = 44.19 months; SD = 12.9 months; 50.9% African-American) with a diagnosis of ASD-only (35.8%; n = 62), ID-only (13.3%; n = 23), or ASD and ID (50.9%; n= 88) were administered the CARS-2, Standard Version (CARS-2-ST).

Results: All 15 items on the CARS-2-ST fit to a Generalized Partial Credit Model (GPC) indicated that items were locally independent based on the standardized LD-χ²(i.e., |LD| < 2 for all items).  Inspection of step parameters and option response functions (ORFs) for each items shows that only item 15 has a unique curve for each response category, while most items show fewer than the suggested number of categories are used as expected. 6 items (4, 6, 9, 10, 13, and 14) had poor discrimination (i.e., a < .5), indicating little contribution to the differentiation between children with varying degrees of ASD.  Item 10 (i.e., Fear or Nervousness) performed particularly poorly (see Figure 1). The S-χ2 item-fit statistic for polytomous data was used to determine absolute fit of the model to each item indicating that 4 items (1, 2, 8, and 14) had poor fit. Global fit was deemed acceptable, M₂ = 27.12, p= 0.08 , RMSEA = 0.05 . Marginal reliability for response item scores was deemed acceptable at .93, however, this estimate over and underestimates the level of precision depending on the location along the continuum. Consequently, a revised 15 item scale collapsing items down to 4 response categories reduced the number of unclear ORF’s (see Figure 2), improved item discrimination, good fit across items, and acceptable marginal reliability at .90. Additional IRT analyses as well as nominal logistic regression analyses will be conducted to further examine the psychometric properties and validity of the CARS-2. 

Conclusions: Preliminary analyses indicate that mid-point ratings (e.g., 3.5) may not be as helpful as expected at discriminating among children on the ASD continuum.  In addition, based on content reasons removing item 10 from the sample studied could be considered due to poor differentiation of individuals across the ASD continuum.