Establishing that the CAARS 2 operates equivalently and produces equally valid scores that can be interpreted in the same way across racial and ethnic groups is a critical component of ensuring that psychometric fairness standards are met. For the U.S. portion of the CAARS 2 Normative Sample, race and ethnicity were categorized according to the U.S. Census Bureau classifications into the following five groups: Hispanic (regardless of race), Black, Asian, White, and Other (includes Native, Multiracial, and other racial identities not otherwise listed). Individuals whose race/ethnicity was classified as Asian or Other were excluded from MI and DTF analyses due to very small sample sizes (for Other in particular, meaningful interpretation of group-level scores is challenging given that this category includes a multitude of race groups); the Asian subgroup was included in the mean group differences comparisons due to an adequate sample size for the analyses. Additionally, race/ethnicity analyses in this section are limited to individuals who live in the U.S., as the Canadian sample sizes lacked statistical power. More details about the correspondence to the U.S. Census classifications can be found in Race/Ethnicity in chapter 7, Standardization. Differences among the U.S. racial and ethnic groups were explored with regard to the CAARS 2 structure and scores. It was expected that there would be negligible differences in terms of race/ethnicity, as the test was designed to minimize the influence of cultural background, with the goal of generalizing to diverse populations, as described in chapter 6, Development.

First, MI was explored within the U.S. subsamples of the CAARS 2 Total Sample. Due to the smaller sample sizes for Black individuals on the CAARS 2 forms (Self-Report N = 186; Observer N = 199), Hispanic individuals (Self-Report N = 229; Observer N = 232) and Black individuals were combined into a larger group (Self-Report N = 415; Observer N = 431), and the combined group was compared to White individuals (Self-Report N = 1,311; Observer N = 1,021) for MI analysis. It should be acknowledged that combining the groups in this way, while necessitated by small samples sizes, may nonetheless limit interpretability of results. It is important for future analyses to explore White vs. Hispanic and White vs. Black comparisons in larger samples to confirm the pattern of results presented within this section.

Table 10.5 presents the MI results for Self-Report and Observer, respectively. For Self-Report, there were no significant decreases in model fit (in particular, the Satorra-Bentler χ² test) when comparing the increasingly strict models tested at each subsequent level. For Observer, although the comparison between strong model (third level of testing) and strict model (fourth level of testing) was significantly different when examined with the Satorra-Bentler χ² test, no other fit statistics showed any meaningful decreases in model fit, indicating that strong invariance is met. This evidence supports the CAARS 2’s ability to measure the construct with the same structure for White individuals as it does for a combined sample of Black and Hispanic individuals.

DTF analyses were examined next to explore the invariance of the CAARS 2 Content Scales across race/ethnicity, specifically between Hispanic and White individuals and between Black and White individuals (note that Black and Hispanic groups were not directly compared, as the investigation here was more concerned with comparisons of historically marginalized groups to a majority group; instead, analyses permitted separating Black and Hispanic groups apart for DTF, rather than combining them as had been done for MI). An example of a DTF graph each for Self-Report and Observer for the comparison of Black and White individuals is provided in Figure 10.2. Test functioning curves for White individuals and Black individuals are depicted, along with a shaded band to display a 95% confidence interval. The two groups’ curves are almost completely overlapping, demonstrating a lack of difference for the Inattention/Executive Dysfunction scale. Similar results were found for all Content Scales across both forms, as well as for the comparisons between Hispanic and White individuals.

The effect size of the DTF statistics measured by the ETSSD are summarized in Table 10.6. The largest effect size across Self-Report and Observer, when comparing either Black or Hispanic to White individuals, was ETTSD = |0.06|, representing a negligible effect size. The differences between groups are negligible, demonstrating a lack of measurement bias between White, Black, and Hispanic groups and reinforcing the generalizability of the CAARS 2.

Click to expand

Figure 10.2. Differential Test Functioning by Race/Ethnicity (Black/White Comparison): Inattention/Executive Dysfunction

To investigate racial/ethnic groups in terms of their mean score differences, subsamples from the U.S. portion of the Normative Sample of Hispanic, Black, and Asian individuals were compared, respectively, to a corresponding subsample of White individuals from the Normative Samples that was matched on gender, education level (EL), language(s) spoken, clinical status, and age. The demographic characteristics of the rated individuals in the matched samples (and their raters, where applicable) are presented in appendix J.

Comparisons between the matched Hispanic and White samples, Black and White samples, and Asian and White samples were analyzed via a series of ANOVAs, with results presented in Tables 10.7 to 10.12. These tables present the means and standard deviations of the CAARS 2 scale scores, along with the significance tests and effect sizes.

When comparing ratings of Hispanic and White individuals, Observer results indicated there were no statistically significant differences observed across all Content scales, with negligible to small effects (median Cohen’s d = |0.10|). For Self-Report, statistically significant differences were observed between Hispanic and White individuals on the Impulsivity and Emotional Dysregulation scales, with small effect sizes (Cohen’s d = 0.41 and 0.42, respectively). These effects could manifest as scores up to 4 points higher for White individuals on the CAARS 2, with the largest effect observed on the Emotional Dysregulation scale (i.e., White individuals rated these items slightly higher than Hispanic individuals).

When comparing ratings of Black and White individuals, Observer results indicated there were no statistically significant differences observed across all Content scales, with negligible to small effects (median Cohen’s d = |0.07|). For Self-Report, statistically significant differences were observed between Black and White individuals on the Inattention/Executive Dysfunction, Hyperactivity, and Negative Self-Concept scales, wherein lower scores were observed for Black individuals than White individuals, with small to medium effect sizes (Cohen’s d ranging from |0.31 to 0.59|). These differences could manifest as scores up to 6 points higher for White individuals than for Black individuals. The largest effect was observed on the Negative Self-Concept scale (i.e., White individuals rated self-concept items as more severe or more frequently problematic, or rated a greater amount of items, than Black individuals).

When comparing ratings of Asian and White individuals across both forms, results indicated there were no statistically significant effects observed across all Content scales. Cohen’s d effect sizes, capturing the size of the difference between group means, demonstrated negligible to medium effects (median Cohen’s d = |0.16|).

Overall, there were small differences, the majority of which were not significant, observed between White individuals and Hispanic, Black, and Asian individuals on the CAARS 2. Taken together, results from the MI, DTF, and mean group difference analyses indicate psychometric equivalence for Hispanic and White individuals, Black and White individuals, and Asian and White individuals. Together with the absence of evidence for measurement bias, there is support for equity in terms of race/ethnic groups for the CAARS 2 and its appropriate use in racially and ethnically diverse populations.