Measuring Treatment Outcome in Autism Preschools

Background: As services for young children with autism become more widely available, there is a growing need for ready measures of treatment outcome. Such measures will  demonstrate efficacy to stakeholders including funding agencies and parents, and can provide ongoing, targeted information to treatment providers regarding the progress of each child. A new treatment outcome measure, the Preschool Outcome Questionnaire (POQ), was recently shown to have good reliability and validity, and to be sensitive to change over time, when used in an intensive therapeutic preschool (Barker et al., in press).  The POQ takes just a few minutes to complete for parents and/or treatment staff.  The POQ is not specifically targeted towards measuring autism symptoms, rather it targets more general behavioral symptoms (including attention, motivation, social skills, and behavioral regulation) that are important for all preschool treatment programs. Objectives: This study evaluated the validity of the POQ with regard to an autism preschool population, and the ability of the POQ to detect change (behavioral improvement) on a monthly basis.

Methods: The Giant Steps Autism Preschool has three classrooms of twelve children each, divided according to symptom severity and level of functioning.  The parents of 32 children agreed to participate in the study.  Children were administered the ADOS at baseline.  The POQ is a 35-item, 4-choice Likert scale focused mostly on overt behavior (e.g., “How often does/is your child resist being hugged” “Argue” “Have sleeping problems”). The POQ was administered on a monthly basis for 6 months, along with two other questionnaires that are relevant to behavior in children with autism but are not designed specifically to be sensitive to short-term change (the Social Communication Questionnaire and the BASC-2).

Results: Although the POQ and ADOS utilize different methods and focus on somewhat different behaviors, baseline POQ and ADOS scores demonstrated moderate correlations (>.6 for POQ and ADOS-Total; >.4 for POQ and ADOS-Social). We interpret this to mean that both instruments provide a window on the child’s general functioning.  POQ items demonstrated adequate internal consistency, and as expected captured more improvement in symptoms across the study period than either the SCQ or BASC-2. We discuss additional analyses of the mechanisms underlying the POQ (for example, evidence for stronger validity for the higher-functioning groups, and possibilities for the ideal time period between POQ administrations) and our ongoing efforts to identify a factor structure that fits the autism preschool population. 

Conclusions: There is substantial ongoing discussion about the best ways to diagnose or identify autism in young children. This study takes a different tack, reporting on a new measure (the POQ) that tracks response to intensive preschool treatment programs.  This easy (5-10 minutes) measure can be administered frequently and data from each administration can be used to quickly target children who are not making expected progress.  More work is needed, but the POQ offers tremendous promise for capturing treatment data for reporting to funding agencies and other important stakeholders.