• Sample
• Reliability
  • Internal Consistency
  • Test Information
• Validity
  • Content and DSM Symptom Scales
  • CAARS 2–ADHD Index
  • Associated Clinical Concern Items and Impairment & Functional Outcome Items
  • Validity Scales
• Summary

Sample

A total of 307 Canadians completed the CAARS 2 Self-Report, and 215 Canadians completed the CAARS 2 Observer. Data were cleaned prior to analysis, based on data quality metrics, including indicators of careless responding (e.g., random responding) and response acquiescence (e.g., an irregular number of consecutive item responses). Data for 33 individuals were removed from the Self-Report sample and those for 20 individuals were removed from the Observer sample. The final samples included 274 individuals for Self-Report and 195 for Observer. Refer to appendix J for the demographic characteristics for the individuals being rated and for the Observers.

Reliability

The reliability of the French version of the CAARS 2 was assessed via internal consistency estimates and test information functions (see Internal Consistency and Standard Error of Measurement and Test Information in chapter 8, Reliability, for more details). Coefficients alpha and omega were used as estimates of internal consistency and information functions were generated using the mirt package in R (Chalmers, 2012).

Internal Consistency

Table 13.1 presents alpha and omega coefficients for the French version of the CAARS 2. Internal consistency estimates were excellent for both the Self-Report and Observer form; the median coefficient omega value across scales for the Self-Report was .94 (ranging from .92 to .97) and for the Observer was .96 (ranging from .89 to .98). These estimates are comparable to those found for the English version from the Normative Samples (see Tables 8.1a, 8.1b, 8.2a, and 8.2b in chapter 8, Reliability), as well as to values derived from the English version completed by individuals in the current sample (estimates are not reported as all scales were within .01 of each other across language versions). Overall, results show that the French version of the CAARS 2 is internally consistent and comparable to the English version.

Test Information

Figure 13.1 shows the test information functions for the CAARS 2 by Content and DSM Symptom Scale. As can be seen in the figure, the French versions of the scales display high information across the relevant range of the ability scale (approaching and exceeding 1.5 standard deviations above the mean). Further, the peaks of the information curves are broad with a wide area beneath them, implying the precision of measurement remains consistent across the relevant range of the scales. The peaks of the information functions are also equal to or greater than a value of 10, indicating very high precision and excellent reliability for the French version of the CAARS 2. These information functions are similar in overall shape and magnitude to the English version (see Test Information in chapter 8, Reliability).

Click to expand

Figure 13.1. Test Information Functions by Scale: CAARS 2 French Translation

a) Inattention/Executive Dysfunction

b) Hyperactivity

c) Impulsivity

d) Emotional Dysregulation

e) Negative Self-Concept

f) DSM ADHD Inattentive Symptoms

g) DSM Hyperactive/Impulsive Symptoms

h) DSM ADHD Total Symptoms

Validity

An important goal of this study was to ensure that the CAARS 2 item content was parallel across the English and French versions of the forms. As described in Creation of Translated Forms, a cultural translation was used to ensure the meaning of each item was captured in the translation, rather than just the literal wording. The expectation was that the French translated items should perform similarly to their English counterparts. Evidence of validity was explored for the Content Scales in terms of consistency in the factor structure across languages (tested via measurement invariance [MI] methods). Given the within-subjects design (in which individuals completed the CAARS 2 in both languages), a within-subjects MI approach was taken (Liu et al., 2017; however, results are also available for a between-subjects approach involving the Normative Samples; see appendix N for these results). After invariance was established, correlations were computed, and mean scale scores were compared for both the Content and DSM Symptom Scales to examine whether scores were consistent between the French and English versions. The CAARS 2–ADHD Index, Associated Clinical Concern Items, Impairment & Functional Outcome Items, and Validity Scales were analyzed for differences between the French and English translations. Results for all analyses are reported in the following sections.

Content and DSM Symptom Scales

The factor structure of the Content Scales was compared via a within-subjects MI approach (note that this differs from the methodology described in appendix M), in accordance with recommendations from Liu et al. (2017). Conducting MI with this approach involves testing the following four steps, in order:

1. Configural Invariance: Language versions have the same factor structure.
   
   
2. Weak Invariance: Language versions have the same factor structure and factor loadings,
   
   
3. Strong Invariance: Language versions have the same factor structure, factor loadings, and item thresholds.
   
   
4. Strict Invariance: Language versions have the same factor structure, factor loadings, item thresholds, and item residuals.
   
   

Results of the MI investigations are presented in Table 13.2 and Table 13.3. Overall, the CAARS 2 Content Scales were found to be invariant across the French and English versions, as evidenced by non-decreasing CFI values and nonsignificant Satorra-Bentler chi-square tests (Satorra & Bentler, 2001). As part of the modeling procedure, some steps required partial invariance adjustments to result in nonsignificant chi-square tests, but the adjustments were infrequent and did not compromise the overall comparability of the scales between the two languages (Dimitrov, 2010). The results provide strong evidence for the validity of the CAARS 2 French translation as a parallel measure to the English version.

Results from the correlation and mean group difference analyses are presented in Table 13.4 and Table 13.5. Corrected correlations (Sackett et al., 2000) revealed statistically significant and very strong relationships across scales between the French and English versions of the CAARS 2, with a median correlation of .91 for the Self-Report (range .90 to .95) and .89 for the Observer (range .86 to .91). Further, Welch’s paired t-tests (Welch, 1947) showed no statistically significant differences between scales when comparing obtained scores on the two language versions (p < 0.01); Cohen’s d effect sizes were also negligible (maximum Cohen’s d = 0.14). Taken together, the strong correlations between scales across language versions, and the lack of statistical and practical differences in mean scale scores, indicates that similar scores can be expected between the French and English versions. These results provide evidence for the validity of the French version of the CAARS 2.

CAARS 2–ADHD Index

The CAARS 2–ADHD Index was examined to confirm that it performed similarly across the French and English versions (see chapter 12, CAARS 2–ADHD Index, for more information on the development, scores, and psychometric properties of the CAARS 2–ADHD Index). The probability scores for the French version of the CAARS 2–ADHD Index were compared to the English version using the Wilcoxon Signed Rank Test (note that this non-parametric approach was favored, as the probability score does not follow assumptions of normality; Wilcoxon, 1945); an effect size, r, is also provided which can be interpreted using the correlation guidelines provided in this chapter (Rosenthal, 1991).

The difference in probability scores between the French and English versions was not statistically significant, and effect sizes were very weak (Self-Report: V = 7029, p = .107, r = -.10; Observer: V = 3252, p = .259, r = -.08). Thus, the CAARS 2–ADHD Index operates similarly in English as it does in French, adding to the validity evidence for the French translation.

Associated Clinical Concern Items and Impairment & Functional Outcome Items

The Associated Clinical Concern Items and Impairment & Functional Outcome Items of the CAARS 2 were also examined to ensure language versions operated similarly. To gauge this, the proportion of individuals with concordant item elevations or endorsements across language versions was calculated (see Associated Clinical Concerns: Item Selection and Scoring and Impairment & Functional Outcome Items: Item Selection and Scoring in chapter 6, Development for more information on how endorsed and elevated responses were determined for items in these scales). Item responses were considered concordant across language versions if an individual’s ratings for a given item were either elevated/endorsed or not elevated/endorsed across both languages. Conversely, if an item was elevated/endorsed on one language, but not the other, the person was considered to have discordant item elevations/endorsements. McNemar’s test (McNemar, 1947) with a continuity correction was used (yields a chi-square test statistic) to ensure item elevations/endorsements were not more frequent on one language version than the other (analyses were conducted with the stats package in R).

Results of the item-level analyses are presented in Table 13.6. For the Associated Clinical Concern Items and the Impairment & Functional Outcome Items, the percentage of individuals with concordant item endorsements or elevations was very high, with agreement above 90% for nearly all items on both forms. McNemar’s tests also showed that item elevations/endorsements were not significantly more frequent on one language version than the other (p > .01), with the exception of a single item for Self-Report (though it still displays 89.4% concordance). Taken together, the results demonstrate that elevations and endorsements on the Associated Clinical Concern Items and the Impairment & Functional Outcome Items are highly similar between the French and English versions, supporting the validity of the French translation.

Validity Scales

The CAARS 2 Validity Scales were examined to ensure that they operated similarly in the French and English versions. For both the Negative Impression Index and the Inconsistency Index, the proportion of individuals with concordance across the French and English versions for scale elevations (that is, raw scores that exceeded the cut-off) was compared (details provided in Response Style Analysis: Item Selection and Score Creation in chapter 6, Development).

As can be seen in Table 13.7, the proportion of individuals with concordant elevations were very high (above 90%) for both the Negative Impression Index and Inconsistency Index. Further, McNemar’s tests indicated that scoring above the cut-off on one language version was not statistically significantly more likely than scoring above the cut-off on the other (p > .01). Taken together, both Validity Scales operated similarly in the French and English versions of the CAARS 2, contributing supporting evidence for the validity of the French language version

Summary

The reliability and validity of the French (Canada) version of the CAARS 2 was examined in a translation study where individuals completed both the French and the English versions consecutively (with order counterbalanced across individuals). Both the Self-Report and Observer forms displayed excellent internal consistency and high levels of measurement precision for all Content and DSM Symptom Scales, with coefficients and information functions comparable to those of the English version for both the current sample, as well as the normative sample. These results provide strong evidence for the reliability of the French version of the CAARS 2.

Further, it was demonstrated that the French version of the Content Scales were invariant from the English scales, indicating the measurement models of both the French and English versions of the Content Scales are statistically similar. Examination of obtained scores also supported the finding of high scale correlations and no mean differences between language versions for the Content Scales and DSM Symptom Scales. Analyses for the ADHD Index, Associated Clinical Concern Items, Impairment & Functional Outcome Items, and Validity Scales also showed high concordance on scale/item-level endorsements and on elevations across language versions. Taken together, these findings provide strong evidence for the validity of the French version of the CAARS 2 and justify expectations that scores generated from both the French and English forms should be highly similar.