Preliminary Reliability and Validity of the Brief Observation of Social Communication Change (BOSCC)
Objectives: To provide preliminary reliability and validity for the BOSCC.
Methods: The BOSCC coding scheme was applied to 177 parent-child free-play videos from 56 toddlers and pre-schoolers with diagnoses of Autism Spectrum Disorder (ASD). All children were using phrase speech or less and were participating in various intervention trials active at two sites. Exploratory Factor Analyses (EFA) were conducted, providing BOSCC domains, as well as intraclass correlations (ICCs) for Test-Retest (TRT) and Inter-rater reliability (IRR). Growth curve models were fit to all available data for each child’s BOSCC score and ADOS Calibrated Severity Score (CSS). Exploratory analyses were also conducted comparing BOSCC scores in children who did and did not demonstrate changes on metrics that have successfully measured change in other research (Wetherby et al., 2014) including the Vineland Adaptive Behaviour Scales- Communication domain (VABS) and Mullen Scales of Early Learning Receptive Language domain (MSEL).
Results: EFA results support a two-factor model, consistent with the social communication and Restricted, Repetitive Behavior (RRB) domains seen in the ASD symptom literature. ICCs computed for TRT and IRR data ranged from 0.79-0.91 and 0.94-0.96, respectively. Results of whole sample (including responders and non-responders) random effects analyses indicate that the average rate of change each month in the ADOS CSS score was 0.055, which corresponded to an effect size of -0.026 per month. The average rate of change in the BOSCC ASD total score was -0.711 corresponding to a substantially larger, though small effect size for the rate of change of -0.068 per month. However, compared to children who did not show improvements on the VABS and MSEL, children who showed improvements (increases) on the on VABS and MSEL also demonstrated significantly larger decreases in BOSCC scores (see Figure 1).
Conclusions: This work provides preliminary evidence for the utility of the BOSCC as a treatment response measure. The BOSCC demonstrates more change than the ADOS CSS over the course of months of intervention. The BOSCC may be an additional tool which is sensitive enough to capture slight changes and also standard and flexible for use across studies. We hope that the BOSCC can aid in identifying efficacious treatments, monitoring progress, and tailoring individualized treatment plans for children with ASD.