A Preliminary Evaluation of the Brief Observation of Social Communication Change (BOSCC) As Candidate Outcome Measure in an Independent Dutch Sample

Background: The Autism Diagnostic Observation Schedule (ADOS) is commonly used as outcome measure. However, it has not primarily been designed for measuring change. A recently developed measure that is aimed to detect change in ASD symptoms, the Brief Observation of Social Communication Change (BOSCC), may be more apt for such purposes. The current study includes a head to head comparison between both measures, comparing ability of detecting behavioral change based on a 10-minute unstructured parent-child interaction (BOSCC) versus a 40-60 minute standardized semi-structured examiner-child interaction (ADOS). 

Objectives: To evaluate the usefulness of the BOSCC in detecting ASD symptom change in comparison to the ADOS, focusing on 1) inter- and intra-rater reliability; 2) construct validity; and 3) sensitivity to capture change.

Methods: Participants encompassed 48 toddlers diagnosed with ASD who were involved in an early intervention study (Oosterling et al., 2010). In that study an intervention was tested in a randomized controlled trial; parents in the experimental group received parent (Focus) training in addition to care-as-usual, whereas parents in the control group received care-as-usual alone. The current study conducted a secondary analysis; applying the BOSCC coding scheme on 96 videotaped parent-child dyads. At baseline and after one year of intervention the ADOS, non-verbal IQ, MacArthur-Bates Communicative Development Inventory (MCDI) and Child Behavior Checklist (CBCL) were also assessed. The Autism Diagnostic Interview (ADI-R) was only applied at baseline. Group-based and individual analyses were done.

Results: 1) Excellent inter-and intra-rater reliability was obtained for total and sub scores of the BOSCC; intraclass correlations were 0.96–0.99 and 0.77–0.98, respectively. This is comparable to the established reliability of the ADOS. 2) With regard to convergent validity, the BOSCC total score showed a moderate Spearman’s correlation with the ADI-R (r_s=0.46), whereas the ADOS total score showed a weak correlation with the ADI-R (r_s=0.39). With regard to discriminant validity, weak correlations were found between the BOSCC and CBCL scores (r_s=-0.04–-0.30), whereas the ADOS showed weak to moderate correlations (r_s=-0.05– -0.41). In contrast, for both the BOSCC and ADOS moderate to strong correlations were found with non-verbal IQ (BOSCC: r_s= -0.44–-0.57; ADOS: r_s=-0.54–-0.60) and MCDI scores (BOSCC: r_s=-0.41–-0.60; ADOS: r_s=-0.50–-0.65). However, overall, construct validity of the BOSCC and the ADOS did not significantly differ. 3) Both the BOSCC and the ADOS total scores were significantly lower at follow-up than at baseline. When considering the clinical reliability of change for each individual separately, using Reliable Change Indexes, the BOSCC was able to capture more reliable change (29% showed significant change on the BOSCC compared to 10% on the ADOS). Change measured by the BOSCC was weakly correlated with change on the ADOS, and with change on other measures.

Conclusions: Our preliminary results indicate that the BOSCC, used in a naturalistic setting to measure change in social communicative behavior, seem to be a promising outcome measure, and has greater potential in measuring individual change as compared to the ADOS. Explanations for findings and recommendations for future research will be discussed.