To address equivalence of scores across countries, individuals in the U.S. and Canada were compared on the CAARS 2 Content Scales. Cross-cultural differences were expected to be minimal, and the lack of meaningful differences would support the generalizability and utility of the CAARS 2 in both countries.

The invariance of the factor structure was compared based on the individual’s country of residence. Results of the MI between the U.S. (Self-Report N = 1,881; Observer N = 1,821) and Canada (Self-Report N = 344; Observer N = 329) are found in Table 10.13.

Across the CAARS 2 Self-Report and Observer scales, there were no statistically significant reductions in the Satorra-Bentler χ² test or any declines in model fit statistics when comparing different levels of invariance. Because the CAARS 2 meets the most stringent level of invariance tested in terms of country of residence, these results support generalizability of the CAARS 2 Self-Report and Observer forms to individuals who live in the U.S. or Canada regarding the factor model.

Next, DTF was evaluated with regard to country of residence. DTF results are presented graphically in Figure 10.3 for the Inattention/Executive Dysfunction scale for both Self-Report and Observer, as an example of the trends seen across all scales and forms. Test functioning curves for individuals in the U.S. and Canada are depicted, along with a shaded band to display a 95% confidence interval; the two groups’ curves are almost completely overlapping, demonstrating a lack of differential functioning between groups. Effect sizes of the DTF analyses for all forms are presented in Table 10.14. Negligible differences (i.e., ETSSD ≤ |.05|) between countries were found, which further supports the generalizability of the CAARS 2 Content Scales to U.S. and Canadian populations alike.

Click to expand

Figure 10.3. Differential Test Functioning by Country of Residence: Inattention/Executive Dysfunction

To examine observed group differences between countries, a subsample of the individuals from the U.S. was randomly selected to match the sample of Canadians from the Normative Sample. Individuals were matched by gender, education level (EL), language(s) spoken, clinical status, and age (see appendix J for demographic characteristics of the samples).

The paired U.S. and Canadian samples were then compared for significant differences across mean scores. Results of the ANOVAs and descriptive statistics for each scale are presented in Tables 10.15 and 10.16. For Observer, there were no statistically significant effects of country of residence. For Self-Report, only Hyperactivity scores were observed to be statistically significantly different between the countries, and the size of the difference was small (Cohen’s d = 0.41). Ratings of individuals from the U.S. and Canada resulted in very similar mean scores as described by Cohen’s d (median Cohen’s d = 0.15 and 0.02 for Self-Report and Observer, respectively). These results indicate that country of residence (specifically, U.S. vs. Canada) had no significant effect on the CAARS 2 Content Scale scores.

Taken together, results from the MI, DTF, and mean group difference analyses indicate psychometric equivalence between individuals from the U.S. or Canada alike on the CAARS 2 Content Scales. There was no evidence for meaningful differences in terms of latent structure nor in terms of test functioning between the groups, and scores were not meaningfully different.