Assessment of Fidelity in a Summer Program for Social Competency in Youth with ASD

Background: Although social skills interventions (SSIs) have recently been deemed evidenced-based for children with ASD (Reichow & Volkmar, 2010), the mechanisms of effectiveness remain unclear. Treatment fidelity (or adherence to principles and procedures of an intervention) is a common treatment mechanism in youth therapies (Perepletchikova & Kazdin, 2005); while it may be crucial in the study of ASD treatments (Wainer & Ingersoll, 2013), it is rarely examined in relation to outcomes. While multifaceted, fidelity in such interventions can generally be conceptualized as engagement manualized activities (manual fidelity) or adherence to treatment principles (conceptual fidelity; Lerner & Mikami, 2012). Given that friendship-making is a key SSI goal (Kasari et al., 2012), and recent evidence that within-group friendships may be amenable to intervention in some SSIs (e.g., Socio-Dramatic Affective Relational Intervention; SDARI; Lerner & Mikami, 2012), examination of distinct predictive roles of fidelity components, is crucial to understanding its role in SSI efficacy. Likewise, study of fidelity measurement timing (i.e. early in treatment versus over time) is clinically valuable for knowing when fidelity may best be addressed.

Objectives: 1) To demonstrate analysis of manual and conceptual fidelity in predicting outcomes in SDARI. 2) To assess relative role of early fidelity and fidelity change in predicting outcomes.

Methods: Thirty-four youth (9 – 16 years; 23 male) participated in 6-week summer SDARI. They completed blinded sociometric ratings of peers in their group (Coie, Dodge, & Coppotelli, 1982) after the first (early) and last (endpoint) week. Group counselors completed measures of SDARI manual and conceptual fidelity (Lerner & Mikami, 2012) twice/week. Early fidelity was calculated by averaging the two initial sessions; fidelity change was calculated via Empirical Bayes residual of fidelity over time.

Results: Hierarchical Linear Modeling was used to control for nesting within groups, with early and change in manual and conceptual fidelity predicting change in sociometrics. Early (β₀₄ = 1.89, p < .05) and increased (β₀₂ = 90.58, p <.05) manual fidelity predicted increased reciprocated friendships; meanwhile, decreased conceptual fidelity predicted increased reciprocated friendships (β₀₁ = -13.64, p <.05). Early manual fidelity predicted increased social preference (β₀₄ = 1.89, p <.05). Methods for these analyses will be presented.

Conclusions:  Findings suggest that manual fidelity was most effective for friendship formation. Higher early manual fidelity and increased manual fidelity predicted increased reciprocated friendships, indicating that activity adherence could beneficially be promoted throughout SDARI. Results also suggest that children with ASD may benefit from a manualized structure. Higher early manual fidelity also predicted increased social preference, suggesting that content of introductory sessions may have been beneficial for within-group liking. Contrary to hypotheses, decreased conceptual fidelity predicted increased reciprocated friendships, suggesting that decreased adherence to the concepts of SDARI by counselors may have facilitated friendship-making with peers.  For instance, counselors may have observed children begin to demonstrate friendship gains, and then may have taken a more passive approach to sessions over time. Results speak to the importance of further investigation into the role of fidelity throughout the course of SSIs.