An individual’s education level (EL) can sometimes be considered a proxy for or a contributing factor to one’s socioeconomic status (SES), which is among the sociocultural variables that may influence the fairness of a test. It was expected that the constructs measured on the CAARS 2 would be independent of influence from EL. To test this hypothesis and ensure generalizability of scores from the CAARS 2 Content Scales, individuals in the Self-Report and Observer samples reported the EL of the rated individual using one of five options: No high school diploma (EL 1), High school diploma/GED (EL 2), Some college/university or associate degree (EL 3), Bachelor’s degree (EL 4), or Graduate or professional degree (EL 5; more information about the representativeness of these groups can be found in Education Level in chapter 7, Standardization). For the sake of MI and DTF analyses, EL was re-categorized into two groups comprising individuals with and without post-secondary education (i.e., Group 1 consists of EL 1 and EL 2: N = 1,515 for Self-Report and N = 1,134 for Observer; Group 2 consists of EL 3, EL 4, and EL 5: N = 710 for Self-Report and N = 796 for Observer). Analyses compared mean group differences across all five levels of education.

First, differences in the factor structure across the two EL groups were evaluated with MI. With more stringent models tested at each level, neither the CAARS 2 Self-Report nor the CAARS 2 Observer displayed meaningful deterioration in model fit (see Table 10.17). For Self-Report, some model comparisons were significant using the Satorra-Bentler χ² test (e.g., the intercept model, p < .001); however, the indicators must be considered in tandem, and many other model fit statistics did not show meaningful deterioration of fit. Therefore, the observed change in model fit were minor and not meaningful, such that invariance between the EL groups on the construct assessed by the CAARS 2 can reasonably be assumed. These results support the invariance of the CAARS 2 scale scores across factor model between individuals with and without post-secondary education, meeting the first-step of the criteria for establishing its unbiased and generalizable use across these populations.

Next, differences in the CAARS 2 Content scales’ functioning for the two broad EL groups were explored with DTF. An example of a DTF graph for the comparison of low EL and high EL groups is provided in Figure 10.4. Test functioning curves for each group are depicted with a shaded 95% confidence interval band; the two groups’ curves are almost completely overlapping, demonstrating a lack of differential functioning for the Inattention/Executive Dysfunction scale. Very similar patterns of results were found for all Content scales across both forms. The effect sizes from DTF analyses are summarized in Table 10.18. Results from both Self-Report and Observer show negligible differences between EL groups (e.g., maximum ETSDD = |.05|). The lack of differential functioning of the CAARS 2 between the different EL groups is further evidence of the test’s equivalence across demographic subgroups.

Click to expand

Figure 10.4. Differential Test Functioning by Education Level: Inattention/Executive Dysfunction

Next, observed mean score group differences were compared using data from the original five EL groups (entire Normative Sample). EL was compared via ANCOVA, with covariates to statistically control for the effects of other demographic factors (i.e., gender, language[s] spoken, clinical status, race/ethnicity, and age). Significant ANCOVA results (i.e., p < .01) were followed up with Tukey’s Honestly Significant Difference (HSD) post-hoc test to evaluate pairwise comparisons, alongside estimates of effect sizes for the omnibus test and pairwise differences.

Results of the ANCOVAs for Self-Report are provided in Table 10.19a, with effect sizes of the pairwise comparisons between group means provided in Table 10.19b. Corresponding results for Observer are provided in Table 10.20a and 10.20b.There were no statistically significant effects of EL on the CAARS 2 Observer scales, as evidenced by negligible effect sizes (η² ranged from .00 to .01). A statistically significant difference between EL groups was only observed for the CAARS 2 Self-Report Emotional Dysregulation scale (p = .009), but the size of this effect was not practically significant (partial η² = .01). The post-hoc analysis revealed that for Emotional Dysregulation, the EL 1 group (i.e., individuals without a high school diploma) scored higher than the EL 4 group (i.e., individuals with a Bachelor’s degree); however, the effect size was small (Cohen’s d = 0.26).

Overall, these results support the absence of meaningful differences in the measurement properties of the test across low and high EL groups. Taken together, results from the MI, DTF, and mean group difference analyses indicate that the CAARS 2 can generalize across EL groups for the Content Scales. There was no strong evidence for meaningful differences in terms of latent structure nor in terms of test functioning between the two groups, and scores were not meaningfully different, supporting the unbiased use of the CAARS 2 for individuals with a range of educational backgrounds and levels.