Development and Interrater Reliability of a New Measure of Functional Behaviour Skills for Children with Autism Spectrum Disorder

Background:  Autism is a heterogeneous neurodevelopmental disorder that causes children to exhibit a wide range of functional and cognitive deficits. To date, only a portion of children with autism have responded to Intensive Behavioural Intervention (IBI) by making significant gains in their cognitive and language functioning. Many children at the severe end of the autism spectrum are nonverbal and require training for the development of functional skills. While cognitive, language and adaptive behaviour assessments are commonly used to measure the outcomes of IBI, there is a need for an objective observational measure that is sensitive to important changes in functional behavioural skills.

Objectives: The objective of this study was to develop a functional behaviour skills assessment (FBSA) and evaluate its reliability in measuring adaptive skills in children with autism. A standardized videotaped assessment and coding system were created to assess various functional skills. To ensure consistency across multiple trained assessors and raters, interrater reliability of this coding system was calculated. 

Methods:  The FBSA was developed to measure five distinct functional behaviour skills, namely requesting, hand-washing, sitting at the table while eating, responding to name and toileting. Standardized scenarios of the child’s behaviour with their parent were administered in the children’s homes by trained and reliable assessors who were blind to the child’s treatment status. Each FBSA took 1 hour to administer and 1 hour to score. The first of these four skills were videotaped and coded. Toileting skill, however, was not videotaped and scored separately. The presence or absence of aspects of requesting behaviours in a 10-minute period of child-parent interaction was coded using the ABLLS-R requesting (Partington, 2006) criteria. Other videotaped skills were coded using task analyses; specifically the presence or absence of aspects of the functional skill (e.g. did the child turn on the tap during hand-washing). The parent’s use of prompts was also coded behaviours (e.g. did the parent physically prompt the child’s action). Raters were trained on three coding tapes to an 80% rater agreement criterion. These raters then coded 30 videotapes and interrater reliability using Inter-Observer Agreement (IOA) was evaluated on 47% of these.

Results:  Interrater reliability was separately scored for each skill. Response to name was demonstrated  to have the largest agreement between raters with an IOA of 89%. This was followed by requests at 84%, hand-washing at 78%, and sitting at table while eating at 76%. The average IOA across all skills and videos was 84%.

Conclusions:  Our novel FBSA and coding system offers a valuable measure of functional skills for children functioning at the severe end of the autism spectrum. Results demonstrated that our coding system is reliable (greater than 75% interrater reliability), suggesting that consistent objective ratings of functional skills can be achieved. This reliable measurement of functional behavioural skills will allow professionals the opportunity to follow the progress of children with autism to determine appropriate adaptive functional skills treatment.

Partington, J. (2006). The assessment of basic language and learning skills-revised. Pleasant Hill CA: Behavour Analysts Inc.