Preliminary Reliability and Validity of the Brief Observation of Social Communication Change (BOSCC)

Background: The field of ASD intervention research lacks adequate outcome measures to assess the utility of treatments (Anagnostou et al., 2015). Although standard measures of social communication behaviors exist, these measures are often not sensitive enough to subtle changes over short periods of time (e.g., the Autism Diagnostic Observation Schedule; ADOS), capture only very specific behaviors (e.g., Early Social Communication Scales), or rely on parent report leading to potential reporter bias. To address these limitations, our group developed the Brief Observation of Social Communication Change (BOSCC), an observation-based measure coded by a blind researcher/clinician that is intended to be sensitive to slight changes in social communication and restricted, repetitive behaviors.

Objectives: To provide preliminary reliability and validity for the BOSCC.

Methods: The BOSCC coding scheme was applied to 177 parent-child free-play videos from 56 toddlers and pre-schoolers with diagnoses of Autism Spectrum Disorder (ASD). All children were using phrase speech or less and were participating in various intervention trials active at two sites. Exploratory Factor Analyses (EFA) were conducted, providing BOSCC domains, as well as intraclass correlations (ICCs) for Test-Retest (TRT) and Inter-rater reliability (IRR). Growth curve models were fit to all available data for each child’s BOSCC score and ADOS Calibrated Severity Score (CSS). Exploratory analyses were also conducted comparing BOSCC scores in children who did and did not demonstrate changes on metrics that have successfully measured change in other research (Wetherby et al., 2014) including the Vineland Adaptive Behaviour Scales- Communication domain (VABS) and Mullen Scales of Early Learning Receptive Language domain (MSEL).

Results: EFA results support a two-factor model, consistent with the social communication and Restricted, Repetitive Behavior (RRB) domains seen in the ASD symptom literature. ICCs computed for TRT and IRR data ranged from 0.79-0.91 and 0.94-0.96, respectively. Results of whole sample (including responders and non-responders) random effects analyses indicate that the average rate of change each month in the ADOS CSS score was 0.055, which corresponded to an effect size of -0.026 per month. The average rate of change in the BOSCC ASD total score was -0.711 corresponding to a substantially larger, though small effect size for the rate of change of -0.068 per month. However, compared to children who did not show improvements on the VABS and MSEL, children who showed improvements (increases) on the on VABS and MSEL also demonstrated significantly larger decreases in BOSCC scores (see Figure 1).

Conclusions: This work provides preliminary evidence for the utility of the BOSCC as a treatment response measure. The BOSCC demonstrates more change than the ADOS CSS over the course of months of intervention. The BOSCC may be an additional tool which is sensitive enough to capture slight changes and also standard and flexible for use across studies. We hope that the BOSCC can aid in identifying efficacious treatments, monitoring progress, and tailoring individualized treatment plans for children with ASD.