Using the Tools of the Trade: The ADOS as a Measure of Treatment Change

Background: One of the most complicated issues involved in autism intervention research is how to measure outcome. The majority of studies measure outcome by changes in scores on intelligence tests, educational placement and services, adaptive scores, or language abilities (Rogers & Vismara, 2008; Legoff & Sherman, 2006; Kasari et al., 2008). Relatively few studies address changes in autism specific behaviors (Wolery & Garfinkle, 2002). Some have looked at changes in overall scores on standardized diagnostic measures, most often the Autism Diagnostic Observation Schedule (ADOS; Lord et al., 2000; Aldred et al., 2004; Owley et al., 2001; Gutstein et al., 2007). However, research on how well these instruments measure change has not yet been undertaken.

Objectives: This study was designed to evaluate whether ADOS scores could be used as a measure of change in young children with autism following a behavioral intervention.

Methods: Thirty-two nonverbal children diagnosed with autism were randomly assigned to one of two treatment conditions (PECS or PRT). Each child had a comprehensive evaluation prior to and following treatment that included Module 1 of the ADOS, the Mullen Scales of Early Learning (Mullen, 1995) and the Vineland Adaptive Behavior Scales (Sparrow et al., 1984). All ADOS videotapes were coded by one of the authors of this study after establishing inter-rater agreement of 80% or better including distinguishing between codes of '2' and '3'.

Results: Repeated measures univariate analyses of variance revealed that revised algorithm total scores decreased pre (M=21.28) to post treatment (M=19.03), with a small effect size (eta squared = .25). However, severity scores did not show a significant change pre to post treatment, with only six children showing a reduction in score of more than one and only one child moving to a less severe diagnostic classification. Adjusted total scores that included all ADOS items as well as codes of '3' and the substitution of a '4' in place of '8' resulted in a significant difference from pre (M= 50.81) to post treatment (M= 40.81), with a moderate effect size (eta = .5). There were significant differences between pre and post-treatment adjusted scores for the communication, social, and play domains. There was no difference in adjusted total scores between treatment conditions. Six items in Module 1 of the ADOS, distributed among the domains, demonstrated significant changes in scores from pre to post treatment. Three of these items were from the social domain (Giving, Response to Joint Attention, and Quality of Social Overtures) one from the communication domain (Use of other’s body to communicate) and one was from the play domain (Imagination/creativity).

Conclusions: ADOS scores are a possible measure of treatment change; however, diagnostic classification scores including algorithm totals and severity scores may not be the best measure of treatment change. Scores that allow for a greater range that include more items and more score possibilities within items may lead to a more robust measure of change. Items that showed change may provide information on how to improve the ADOS as a measure of treatment change.