Gender, for the purposes of these fairness-related analyses, is defined as the rated individual’s gender identity. Analyses were conducted to compare males and females on the CAARS 2 in terms of MI, DTF, and mean group differences. The very small sample size for individuals who are non-binary or indicated “Other” for gender (Self-Report N = 11; Observer N = 5) did not allow for meaningful testing. Therefore, when assessing invariance by gender, only males (N = 1,028 for Self-Report; N = 1,021 for Observer) and females (N = 1,186 for Self-Report; N = 1,123 for Observer) were included.

Invariance between males and females for the CAARS 2 was first explored via MI analyses (see Table 10.1). For Self-Report and Observer, there are some models with a significant Satorra-Bentler χ² test (e.g., Strict models); however, the absence of any decline in many other fit statistics, such as CFI and SRMR, suggests invariance was upheld. As more constraints were added throughout the process of testing MI, model fit did not change in a meaningful way, indicating that the factor model is invariant between males and females.

In addition to MI results, DTF analyses were conducted to explore the invariance of the CAARS 2 for males and females through a different framework. An example of a DTF graph for both Self-Report and Observer is provided in Figure 10.1. Test functioning for males and females are depicted, along with a shaded band to display a 95% confidence interval. The two groups’ curves are almost completely overlapping, demonstrating a lack of difference for the Inattention/Executive Dysfunction scale.

The effect size of the DTF analyses for all CAARS 2 Content scales, as measured by the ETSSD, are summarized in Table 10.2. Negligible differences (i.e., ETSSD ≤ |.04|) between males and females were found across the CAARS 2 Content Scales and across both forms, demonstrating invariance by gender.

Click to expand

Figure 10.1. Differential Test Functioning by Gender : Inattention/Executive Dysfunction

To examine observed group differences between gender, a matched sample of males and females were selected at random from the Normative Samples. Individuals were matched by education level (EL), language(s) spoken, clinical status, race/ethnicity, and age. The demographic characteristics of the rated individuals in the matched samples (and their raters, where applicable) are presented in appendix J.

The paired samples of males and females were then compared for significant differences across mean scores. Results of the ANOVAs and descriptive statistics for each scale are presented in Tables 10.3 and 10.4. When comparing ratings of male and female individuals, for Self-Report, there were statistically significant effects observed for the Inattention/Executive Dysfunction, Hyperactivity, and Impulsivity scales, wherein males yielded slightly higher scores than females; however, the effect sizes were negligible to small (Cohen’s d = |0.18 to 0.28|). For the Observer results, the only statistically significant effect observed was on the Negative Self-Concept scale, wherein ratings of females yielded slightly higher scores than males; however, the effect size was small (Cohen’s d = |0.24|). The effect of gender on the remainder of the scale scores was not statistically significant.

Overall, these results support the absence of meaningful gender differences. Taken together, results from the MI, DTF, and mean group difference analyses indicate psychometric equivalence between males and females for the CAARS 2 Content Scales. There was no strong evidence for meaningful differences in terms of latent structure nor in terms of test functioning between the two gender groups, and scores for males and females were not meaningfully different. Although results of these analyses suggest that use of Combined Gender as the primary reference group is appropriate, there may be instances where specific comparisons to a gender group are desired. Gender Specific and Combined Gender normative scoring options are available; please see Scoring and Report Options in chapter 3, Administration and Scoring, for details.