An Online Verbal IQ Test: Development, Validity, and Standardization of an Adaptive Vocabulary Test for Remote Phenotyping of Parents

Background:  Autism Spectrum Disorders (ASD) are defined by a dyad of behaviors, but comorbid intellectual disability (ID, which affects 30-70% of individuals with ASD) can present equal or even greater challenges to achieving successful adaptation and long term outcomes. In behavioral and neuropsychological studies of autism, researchers use IQ as a matching or co-varying variable. Unfortunately, this practice limits a) statistical power to find case-control group differences on measures that correlate with IQ, and b) our ability to understand the impact of autism on cognitive ability.  Because IQ is ~ 50% heritable, offspring typically do not deviate much from average parent IQ.  Thus, matching case and control on parent IQ would allow us to assess the impact of autism on IQ and all measures that correlate with it. However, collecting parent IQ can be impractical with standardized clinician administered tests, since both parents often cannot come to the lab. Requiring IQ assessment of both parents would likely bias the sample and prove expensive and time-intensive for both researchers and participants.

Objectives: To develop an online test of verbal IQ (the Verbal Adaptive Test, VAT) that can be administered remotely or on a lab computer in a fraction of the usual time.

Methods: We developed a multiple-choice vocabulary task and piloted 209 items to 1,599 adult participants recruited online. An Item Response Theory (IRT) model was fit to the data, and individual item performance was assessed across several parameters including difficulty, guessing, loadings on the primary factor, and discrimination, which measures how well an item discriminates between two individuals of similar abilities. 

Results: A three-parameter model demonstrated the best fit to the item response data. Several indices of performance suggested overall desirable item performance. Briefly, 75.1% of items had discrimination >1 (which is considered ‘good’; M = 1.37, SD = 0.62), and 90.3% of items loaded saliently on the primary factor. In addition, the VAT Standard Error of Measurement (SEM) is less than or equal to the average SEM of the WAIS VCI (Wechsler Adult Intelligence Scales Verbal Comprehension Index) across a broad range of abilities (see figure). Initial items targeted IQ scores less than 100, and additional items targeting IQ scores greater than 100 and at both ability extremes are being added. Preliminary simulated data results suggest that the VAT will decrease test administration by more than 40%.

Conclusions: The VAT is an efficient and accurate online verbal IQ test that performs above desired metrics at both the individual item and composite test levels. The VAT also offers precise IQ measurement across a broad ability range including the extremes, which are difficult to measure using tests designed with classical test theory. This feature, combined with decreased administration time, makes the VAT appealing for IQ assessment in individuals with ASD and ID, and their family members.