Internal consistency refers to the interrelatedness of test items. In the case of a multidimensional instrument such as the CAARS 2, the items comprising each scale are related mainly by their theoretical association to the underlying factors being measured. The higher the intercorrelation of the items on a scale, the higher the reliability coefficient, or measure of internal consistency. A common measure of internal consistency is coefficient alpha, which ranges from 0 (poor reliability) to 1 (perfect reliability; Cronbach, 1951). A desirable value for alpha (≥ .80) indicates that items relate strongly to a common construct but are not so highly related as to indicate redundancy among the items. Conventional reliability coefficients (i.e., coefficient alpha) are influenced by how interrelated the items are, the number of items on the scale (John & Benet-Martinez, 2000), the purpose of scale interpretation, and the narrowness of the construct being measured. Coefficient alpha can underestimate internal consistency and often provides the lower bound estimate (Raykov, 1997; Viladrich et al., 2017). Coefficient alpha is a widely reported coefficient in social and behavioral research and is therefore included in the results; however, an alternative metric is also reported for completeness.

Coefficient omega is a measure of internal consistency reliability derived from confirmatory factor analysis (CFA) parameter estimates and designed for congeneric scales. It is an excellent alternative for measuring internal consistency when items vary in the strength of their relationship to the latent constructs, because it makes fewer assumptions about the data than coefficient alpha (for a review, see Graham, 2006; McNeish, 2018). As the CAARS 2 items have four ordered categorical responses, coefficients alpha and omega are calculated based on the polychoric correlation matrix, a technique used to provide more accurate estimates for relations between ordinal items (Zumbo et al., 2007). Coefficient omega is interpreted similarly to coefficient alpha (e.g., values greater than or equal to .80 are desirable.)

Internal consistency estimates for the CAARS 2 are presented in Tables 8.1 to 8.2 for the Normative Sample, and Tables 8.3 to 8.4 for the ADHD Reference Samples (see chapter 7, Standardization, for a description of the Normative and ADHD Reference Samples).

The reliability coefficients presented in these tables indicate that the CAARS 2 has excellent internal consistency on all forms, across all age groups. Across all age groups and genders in the Normative Sample, the median coefficient omega was .93 (ranging from .84 to .98) for scales on the CAARS 2 Self-Report, and .94 (ranging from .86 to .99) for scales on the CAARS 2 Observer. For the ADHD Reference Sample, the median coefficient omega was .92 (ranging from .88 to .95) for scales on the CAARS 2 Self-Report and .94 (ranging from .90 to .97) for scales on the CAARS 2 Observer.

Internal consistency values provide an estimate of a test score’s reliability, but these values also have practical applications through the calculation of SEM, a statistic that quantifies the amount of error in the obtained scores. The formula for SEM is:

SEM values were calculated for the CAARS 2 scale scores using the standard deviation of the reference sample’s T-scores (i.e., SD = 10), along with the omega reliability coefficient. Overall, the median SEM was 2.60 for Self-Report and 2.36 for Observer in the Normative Sample, and 2.85 for Self-Report and 2.39 for Observer in the ADHD Reference Sample (see Tables 8.1 and 8.2 for SEM values for each scale for the Normative Samples, and Tables 8.3 and 8.4 for the ADHD Reference Samples). The low values reported here indicate a very small SEM, or very little error in the estimated true scores.

Practical applications of SEM values for the CAARS 2 include the calculation of confidence intervals (see chapter 4, Interpretation, for a description of how confidence intervals can be used in the interpretation of results). The confidence interval is a margin of error around the obtained score used to arrive at an estimate of a true score (Harvill, 1991). This margin provides a range of scores, at a particular level of probability, within which the true score would be expected to fall. For the CAARS 2, 90% and 95% confidence interval levels were calculated based on the SEM for each scale. (As described in Scoring in chapter 3, Administration and Scoring, confidence intervals around scores based on the Principal Normative Samples are automatically included in the computerized report. To calculate confidence intervals related to the ADHD Reference Samples, see appendix C).