Agreement Between Self-Report and Observational Measurement of the Use of Evidence-Based Practices in Community Early Intervention Programs for Children with Autism Spectrum Disorders

Background: Although a few specific treatment methods have been established as efficacious for some children with autism in laboratory settings, there has been little investigation of community implementation of interventions for children with autism. An examination of the use of community practices may assist researchers in facilitating knowledge transfer to practice settings.

Objectives: The goals of this study included (1) comparing provider report of strategy use to observed strategy use and (2) examining the relationship between community intervention and evidence-based practices (EBP) for ASD.

Methods: Seventeen providers serving preschoolers with ASD completed a telephone survey of techniques used in their programs and how they adapt these techniques in practice. Test/retest reliability was 90%. Providers were then videotaped in their classrooms for two hours per day across two days. Trained observers scored videos for activity type and intervention techniques used. Descriptive analyses examined provider use of EBPs and were used to determine agreement between self-report and observational data.

Results: Ninety-one percent of providers reported using at least one EBP in their program; however providers typically combined strategies (up to seven methods). Only 9% of providers chose techniques based on research evidence. All providers had concerns about limited training opportunities Agreement between self-report and observational data was poor. For example, 95% of providers reported using sensory integration strategies; however these methods only appeared in 5% of observations. Ninety-one percent of providers indicated using child choice as a teaching strategy, however this was observed in only 2% of observations.

Conclusions: These data highlight the importance of observational data in determining EBP use in community programs. The extensive use of eclectic models may be impacting the effectiveness of intervention by diluting the depth and structure of specific strategy use. Innovative methods of translation of EBP to community programs are needed to ensure effective implementation.