A Preliminary Evaluation of the Brief Observation of Social Communication Change (BOSCC) As Candidate Outcome Measure in an Independent Dutch Sample
Objectives: To evaluate the usefulness of the BOSCC in detecting ASD symptom change in comparison to the ADOS, focusing on 1) inter- and intra-rater reliability; 2) construct validity; and 3) sensitivity to capture change.
Methods: Participants encompassed 48 toddlers diagnosed with ASD who were involved in an early intervention study (Oosterling et al., 2010). In that study an intervention was tested in a randomized controlled trial; parents in the experimental group received parent (Focus) training in addition to care-as-usual, whereas parents in the control group received care-as-usual alone. The current study conducted a secondary analysis; applying the BOSCC coding scheme on 96 videotaped parent-child dyads. At baseline and after one year of intervention the ADOS, non-verbal IQ, MacArthur-Bates Communicative Development Inventory (MCDI) and Child Behavior Checklist (CBCL) were also assessed. The Autism Diagnostic Interview (ADI-R) was only applied at baseline. Group-based and individual analyses were done.
Results: 1) Excellent inter-and intra-rater reliability was obtained for total and sub scores of the BOSCC; intraclass correlations were 0.96–0.99 and 0.77–0.98, respectively. This is comparable to the established reliability of the ADOS. 2) With regard to convergent validity, the BOSCC total score showed a moderate Spearman’s correlation with the ADI-R (rs=0.46), whereas the ADOS total score showed a weak correlation with the ADI-R (rs=0.39). With regard to discriminant validity, weak correlations were found between the BOSCC and CBCL scores (rs=-0.04–-0.30), whereas the ADOS showed weak to moderate correlations (rs=-0.05– -0.41). In contrast, for both the BOSCC and ADOS moderate to strong correlations were found with non-verbal IQ (BOSCC: rs= -0.44–-0.57; ADOS: rs=-0.54–-0.60) and MCDI scores (BOSCC: rs=-0.41–-0.60; ADOS: rs=-0.50–-0.65). However, overall, construct validity of the BOSCC and the ADOS did not significantly differ. 3) Both the BOSCC and the ADOS total scores were significantly lower at follow-up than at baseline. When considering the clinical reliability of change for each individual separately, using Reliable Change Indexes, the BOSCC was able to capture more reliable change (29% showed significant change on the BOSCC compared to 10% on the ADOS). Change measured by the BOSCC was weakly correlated with change on the ADOS, and with change on other measures.
Conclusions: Our preliminary results indicate that the BOSCC, used in a naturalistic setting to measure change in social communicative behavior, seem to be a promising outcome measure, and has greater potential in measuring individual change as compared to the ADOS. Explanations for findings and recommendations for future research will be discussed.