Improving Impressions of Social Competence: Results from an RCT of the START Social Skills Program for Adolescents with ASD

Background: There has been a growing interest in targeting the social vulnerabilities of adolescents with ASD. Outcomes of these efforts have primarily consisted of a variety of parent and self-report measures (Miller, Vernon, Wu, & Russo, 2014). These measures, while serving as useful indicators of improvement, have notable limitations related to social desirability and other reporting biases (McMahon et al., 2013; Moskowitz, 2006). There is a need to determine if purported social improvements endorsed on parent and self-rating scales are also reflected in impression ratings of social competence made by one’s peers.  Blind peer ratings of social competence (made after watching conversation probes of each participant) present a novel method for understanding if everyday observers are able to discern noticeable improvements in social aptitude. 

Objectives:  The current study measures social skills improvement using social impression ratings following participation in a randomized controlled trial (RCT) of a social intervention for adolescents with ASD.

Methods:  Participants consisted of 35 adolescents with ASD (ages 12-17; 31% female) who were enrolled in an RCT of the Social Tools And Rules for Teens (START) program. Adolescents were randomized to either a treatment group or a waitlist control group. The treatment group completed the 20-week curriculum; each 2-hour session consisted of an individual therapeutic check-in, unstructured socialization with typically developing high school and undergraduate facilitators, structured social activity, discussion of a weekly social topic, and a check-out session with the parents. Two 5-minute video conversations with two unfamiliar similar-aged students (one male and one female who were unaffiliated with the study) were recorded in naturalistic settings at both intake and 20-week time points. Undergraduates unfamiliar with the aims of the project or the disability status of the individuals (total n=131) were recruited to watch these videos and provide social ratings along numerous dimensions, including social skills, comfort, awkwardness, perceived quality of existing relationships. The same undergraduate completed a rating for a given participant at both intake and 20-week time points. Videos were counterbalanced among raters, so a rater was not always viewing videos in the same order. Every participant video was rated five times by different raters to ensure consistency of ratings.

Results: A series of dependent t-tests were performed for both the treatment and waitlist control groups. Pre- and post-intervention ratings demonstrated meaningful change in the treatment group (p=.001), but no significant improvement was noted in the control group. While participants varied in their total change scores over the course of their START program enrollment, 97% of the immediate treatment participants experienced improvement in overall ratings from their pre-intervention to post-intervention (20-week) time point.

Conclusions:  Overall, observer ratings of social impressions provide favorable evidence regarding the social validity and significance of the START program. This method offers a real world indication of improved social competence that is not adequately captured by survey measures alone. The START model appears to positively impact the use of pivotal conversation strategies, which in turn creates more favorable social impression ratings among one’s peers.