LORIS is a web-based data management system that has been used internationally as the backbone for a number of large-scale neurodevelopmental studies including the NIH MRI study of normal brain development (Das 2012; Evans 2006). Currently, LORIS provides a framework that houses data for more than 10,000 unique subject profiles, with data collected on over 500 behavioral and clinical measures.
Objectives:
Specific features of the database have been customized and further developed in recent years for autism research and deployed in the Infant Brain Imaging Study (IBIS)3 that studies brain behaviour correlates in infants at high risk for autism, and the Autism Spectrum Disorder project for NeuroDevNet that investigates the brain and behavioural development in affected individuals (Wolff 2012). In these longitudinal studies involving data collection across multiple sites, it is critical to provide rigorous and extensive data management systems that facilitate quality checking and validation of collected data. To this end, LORIS incorporates modules that ensure data quality and facilitate the uniformity of data within each project.
Methods:
For both behavioral and neuroimaging modalities, quality control and extensive validation mechanisms provide detailed, timely and informative feedback at the level of each entered variable, and improve data collection going forward.
Results:
First, at the data entry level, restricted data options, strict input controls, and range checking enforce data consistency according to project standards. Automated scoring and normative look-up tables ensure proper scoring, double data entry reduces errors, and strict completion rules ensure the completeness of the data. All of these features guarantee the quality of stored data and reduce the need to revisit data with corrective action.
Once data is entered, separate modules have been designed for data validation and feedback based on automated and manual checks. Through a web-accessible user-friendly module, researchers can quickly review data completeness, integrity, and validity, and flag cases that need further examination. In addition, the statistics module provides a quick overview of the data, while user-defined scatterplots can be displayed for visual verification of the data.
Finally, a separate module has been designed to ensure that the clinical data acquired are validated against a “gold” standard of reliability, as well as for within-project reliability. This reliability module helps with verification and administration of reliability on important clinical measures such as the Autism Observational Scale for Infants (AOSI), the Autism Diagnostic Observation Schedule (ADOS) and others. Through a web-accessible video and document repository, clinicians can download cases to review and provide validation responses directly in database. Enforced by project-defined regulations and thresholds, the reliability module enables monitoring, recording and reporting of within and across site reliability, ensuring that clinical data is valid and reliable across the project.
Conclusions:
In summary, a series of database features within the LORIS data management system have been customized for the use of multisite, longitudinal projects in autism and have demonstrated great benefit. Given the direction of autism research towards multimodal research that takes advantage of newly established networks, such infrastructure is critical to ensure quality of data and facilitate collaborations.