An Investigation of Parent's Ability to Report Problem Behavior

Background:

Direct observation is a powerful method for problem behavior assessment. However, it can be costly, intrusive, and it may cause behavior reactivity.  An alternative method of observation is video recording, but it can generate a large amount of data which is expensive to manually analyze. A practical approach is selective archiving [Hayes & Abowd, 2008], where a trained observer signals the recording of problem behaviors. For a wider population of observers, for example parents at home, it is initially critical to determine the quality of the data gathered through selective archiving. We report the findings from a study that evaluated the effectiveness of this approach with parents in the home environment. We developed a robust multi-camera system, CRAFT (Continuous Recording and Flagging Technology), which collects several hours of continuous digital video and links it to problem behavior instances identified by parents through a button click on a remote control.  
Objectives:
The aims of the study were to validate parent’s ability to signal their child’s problem behavior and to establish baseline capabilities of parents as data collectors. We also wanted to investigate the requirements and potential capabilities of CRAFT.
Methods:
In this study, we recruited six families, each with a child diagnosed with a developmental disorder. Participant families either belonged to a clinical intervention program or a home-based treatment program of a severe behavior clinic. A trained observer from the clinic visited each home and installed CRAFT. Before the deployment, the trained observer and the parents had a discussion to determine the target problem behaviors that will be the focus for parent identification. Up to four cameras were mounted based on the parent’s ranking of where problem behavior was more likely to happen. A laptop collected the data from the cameras. Parents were instructed to click a button on the remote every time they observed the onset of a problem behavior. After a day, the trained observer went back to collect CRAFT. The observer independently coded the videos in their entirety and identified all instances of target behaviors. Once the ground truth had been established, it was compared to the annotation data obtained from parent “clicks”.
Results:
On average, 2 cameras were deployed in each house for approximately 16 hours. Results show that out of 19 verifiable problem behaviors, parents correctly identified 10 and missed 9 instances of problem behaviors. Out of 28 verifiable parent annotations, 18 were false indications. The precision of parent annotation is 0.35 and recall is 0.53.  
Conclusions:
Results suggest that parents falsely indicate a high occurrence of target behaviors when it is not present and miss true instances. Future research will address the role of parent education by examining the effect that training has on identifying target behaviors.  In addition, given that many severe behaviors occur with high intensity but low frequency, data will be collected over longer periods.