Measuring Treatment Effect in Children with Autism with the Brief Observation of Social Communication Change (BOSCC)

Background:  

A range of social interaction measures are used when investigating effects of interventions for young children with autism (Bölte & Diehl, 2013). There is a need for a more unified selection of measures to permit comparison across studies. The Brief Observation of Social Communication Change (BOSCC, Lord et al. in press) is being developed as a sensitive measure of treatment effect on core autism features. The requirement of no formal training or certification, and enhancement of coding through decision-trees, should be appealing for researchers. However it is not yet clear how BOSCC will relate to measures more closely tied to the specific intervention target. 

Objectives:  

1) To explore if BOSCC detects the treatment effect previously identified by the joint engagement (JE) coding scheme, and 2) to investigate if there is a relationship in change from pre to post-test for BOSCC and JE coding. 

Methods:  

As part of an RCT (N=61 children with autistic disorder (aged 29-60 months)) testing the effect of an eight-week joint attention intervention (Kaale et al., 2012), 10-minute video-recordings of parent-child play were coded for JE (Bakeman & Adamson, 1984). The same videos were now coded using BOSCC. As for the JE-coding possible treatment effects on the BOSCC-coding were analysed using ANCOVA with BOSCC baseline-scores as covariate. In addition, a reliable change index (RCI; Jacobson & Truax, 1991) was used to compare individual change scores for each measure. The relationship between BOSCC and JE was analysed with Chi-square. Inter-rater reliability for both measures was very good to excellent (BOSCC: weighted Kappa’s=.74-1.0, ICCs=.96-99, JE coding: ICC=.80).   

Results:  

For JE coding there was a statistically significant group difference at post-intervention (F (1, 59)=6.271, p=.015). The children in the intervention group spent on average 12.2% more time in JE with their mothers compared to the control group. For BOSCC coding the two groups were not significantly different (F (1, 59)=2.459, p=.123) in the size of this change. The proportion of children obtaining an RCI-score showed no significant association between the two measure (p=.51). RCI raw-scores are presented in table 1. 

Conclusions:

As reported in Kaale et al. (2012) JE-coding revealed a significant effect of intervention. Still, no significant group effect was found for BOSCC-codings. This indicates that while a short-term intervention may impact on the targeted behaviour, this impact does not necessarily extend to autism symptoms more generally. Also, it is interesting to note that there was no relationship in children’s RCI-scores on BOSCC and JE from baseline to post-test. We discuss results in light of their implications for the design of intervention studies and meaningful measurement of outcome.