Applying the Theory of Change Approach in a National Autism Charity – Benefits, Challenges and Issues in Selecting Measures.

Background: Large autism charities typically undertake a wide range of activities including human services, information and signposting, training, campaigns and research. These activities may have emerged over a long period, sometimes in response to need or opportunistically due to funding, without a clear strategy or overarching aim. A theory of change is a planning tool that maps a charity’s path from needs and inputs, to activities, outputs and outcomes. A well constructed theory can bring clarity to a charity’s activities and purpose, and identify assumptions about causality, gaps in activities, and suitable performance measures.

Objectives: (1) To produce an overarching theory of change for a leading nationwide autism charity, to help clarify its strategy and how each service or department contributes to its aims (2) to produce a ‘dashboard’ of performance measures aligned to the theory of change, to report to the charity’s CEO, board of trustees, staff, autism community and other stakeholders.

Methods: Workshops and consultation events were held with staff, trustees, autistic people, families, autism professionals, volunteers, donors, elected representatives, journalists and academics. The initial aim was to identify core beliefs about what the charity should stand for. These were translated into an overall aim. A second round of workshops was held with key staff to develop elements of the theory of change relating to autism professionals, education, human services, public awareness, employers and businesses. Autistic people and families were consulted on the outcomes of most importance to them as part of a national online survey in standard and Easy Read formats. A team of government analysts provided volunteer hours to help identify suitable data sources.

Results: N/A

Conclusions: Producing a theory of change proved worthwhile as it helped to clarify aims and accountabilities, and promoted the distinction between activities (e.g. training), outputs (e.g. autism aware professionals) and outcomes directly experienced by autistic people. It also proved useful in making the case for funding for autism services. A theory of change for a national autism charity is likely to be complex, given the range of challenges faced by autistic people across the life course. There are some limitations to this approach: the evidence base for autism interventions is poor, meaning assumptions about causality are weak; and although being understood, appreciated and supported were seen as important by all stakeholders, there is no consensus within the autism community on outcome measures. Using quantitative measures in an autism setting also exposes a number of issues: many autistic people are undiagnosed and so will not be represented in most data sources; measuring the indirect impact of training and campaigns is difficult; direct support for autistic people usually lacks a counterfactual; a person centred approach works against standard measurement; activities and outcomes for the relatively small number of autistic people with the greatest support needs can get ‘lost’ among bigger numbers. However on balance, the process has been positive as it creates an impetus for improving measurement, and challenges the practice among autism charities of reporting mainly activity measures, rather than outcomes.