Understanding and Measuring Change and Impact of Novel Early Intervention Programs

Background:

State early intervention systems vary regarding their abilities to offer empirically validated treatments that specifically target ASD symptoms. Randomized controlled trials of manualized interventions have shown promise for some young children but data on their ability to translate into meaningful change in low-resource community settings is lacking.  There is a pressing need for (1) enhanced strategies for successful community implementation and (2) better measures of the impact of implemented evidence-based programs for young children with ASD.  This work describes one such pilot program that partners a Part C system with behavioral intervention specialists to not only evaluate the impact of the program, but also to understand whether novel metrics for assessing impact may be helpful in program evaluation.

Objectives:  

• Assess implementation of evidenced based early intervention program
• Evaluate CGI-I measure as indicator of impact of early intervention program

Methods:  

This program evaluation study presents results of use of modified Early Intervention-specific (EI) Clinical Global Impressions of Improvement (CGI-I) scales within the context of a model service delivery program.  58 families were provided with parent education-focused service models through Part C system following a diagnosis of ASD.  All 58 families participate in a brief service model that consisted of 2 home-based evaluation support sessions. 19 of these families participate in an extended service that included 12 additional sessions guided by the Early Start Denver Model (ESDM). Clinicians provided CGI-I ratings at the completion of the brief model for each family and again at the completion of the extended model for families that participated in both. Ratings are on a 7-point Likert scale, with higher scores indicating worse functioning and lower scores indicating improved functioning. The CGI-I rating assesses improvement in the following domains: participation in caregiving routines, participation in play routines, verbal and nonverbal communication skills, social skills, engagement in restricted interests and repetitive patterns of behavior, and demonstration of challenging behavior. 

Results:  

While no significant difference scores were found in pre- and post-service scores on global measures of functioning as indexed by the Vineland Adaptive Behavior Scales-2, both caregivers and clinicians rated children as more improved on the CGI-I for the extended service delivery model, although the clinician reported (T = 7.56, p < .001) difference was larger than that of the caregiver report (T = 2.74, p < .05).  Follow up comparisons will be presented regarding the main domain drivers of global CGI-I change from pre- to post-intervention within the EI CGI-I . 

Conclusions:  

The modified CGI scale was able to capture significant changes in total score in an ASD population between pre- and post-intervention. Similar changes were not documented on a common global measure often used as a proxy of change in model programs.  Future program implementation and evaluation work may benefit by adopting and validated global and domain relevant CGI rating systems.