Interrater Reliability Between Parents' and Preschool Teachers' Ratings of Language in Children with Childhood Autism

Background:

Children with childhood autism often fall below basal levels on standardized direct assessments for language competence (Charman et al., 2003). Several researchers thus suggest the use of report-based language assessments for young children with autism (Charman, 2004; Tager-Flusberg et al., 2009). The centrality of language development in understanding autism, and the increasing numbers of children diagnosed with the disorder early on in preschool years underline the need for reliable, valid and effective measurements (Charman et al., 2003). However, some claim that parent-reports tend to over-estimate the child’s abilities compared to reports from others that know the child well (Tomasello & Mervis, 1994). The accuracy of parent-report assessments has, and still is a topic of intense debate.

Objectives:

The aim of this study was to investigate interrater-reliability between parents’ and preschool teachers’ ratings of language in children autism. 

Methods:  

Parents and preschool teachers of 61 children with a diagnosis of childhood autism (aged 2-4 years) separately filled out the MacArthur-Bates Communicative Development Inventory; Words and Gestures (CDI ­­­– WG: Fenson et al., 2007), a widely used report-based language measure. Interrater reliability between parents’ and preschool teachers’ ratings was estimated for children’s language production and language understanding. Pearson’s rwas used to analyze the correlations between parents’ and preschool teachers’ ratings. The Spearman-Brown formula was then used to estimate the interrater reliability. Although most used for split-half reliability, the Spearman-Brown formula is also applicable to interrater reliability analysis (Fan & Chen, 2000).

Results:

Mean ratings of word production were 121 for parents and 109 for preschool teachers, while mean ratings of word understanding were 182 for parents and 158 for preschool teachers. Analyses showed high correlations (r =.93**) between parents’ and preschool teachers’ ratings on language production. For language understanding, preschool teachers and parents correlated lower, but still high (r = .79**).  Spearman-Brown reliability analyses showed R = .96 for word production, and R = .88 for word understanding.

Conclusions:

The high levels of interrater reliability between parents and preschool teachers obtained in this study strengthen psychometrical properties of the CDI as a reliable measure when assessing language production and –understanding in children with childhood autism. This supports proposals from Charman (2004) and Tager-Flusberg et al. (2009) that report-based assessments should be included when testing language skills in children with autism. The results also suggest that both parents and preschool teachers are useful sources of information. Although parents tended to rate children’s language skills slightly higher compared to preschool teachers, this may just as well be interpreted as contextual differences between home and preschool, as opposed to interpretations of parents’ over-estimation. Based on our findings the CDI may be a valuable and cost-effective alternative to direct testing where the latter is difficult, as often is the case when testing children with autism (Charman, 2004).