• Sample
• Reliability
  • Internal Consistency
  • Test Information
• Validity
  • Content and DSM Symptom Scales
  • CAARS 2–ADHD Index
  • Associated Clinical Concern Items and Impairment & Functional Outcome Items
  • Validity Scales
• Summary

Sample

A total of 328 Americans completed the CAARS 2 Self-Report, and 316 Americans completed the CAARS 2 Observer. Data were cleaned prior to analysis, based on data quality metrics, including indicators of careless responding (e.g., random responding) and response acquiescence (e.g., an irregular number of consecutive item responses), 45 individuals were removed from the Self-Report sample and 86 individuals were removed from the Observer sample. The final samples included 283 individuals for Self-Report and 230 for Observer. Refer to appendix J for the demographic characteristics for the individuals being rated and for the Observers.

Reliability

Reliability for the Spanish version of the CAARS 2 was examined with internal consistency estimates and test information functions (see Internal Consistency and Standard Error of Measurement and Test Information in chapter 8, Reliability, for more details). Coefficients alpha and omega were used as estimates of internal consistency and test information functions were generated using the mirt package in R (Chalmers, 2012).

Internal Consistency

Table 13.8 presents alpha and omega coefficients for the Spanish version of the CAARS 2. Internal consistency estimates were excellent for both the Self-Report and Observer form; the median coefficient omega value across scales for Self-Report was .93 (ranging from .90 to .98) and for the Observer was .97 (ranging from .90 to .98). These estimates are comparable to those found for the English version from the Normative Samples (see Tables 8.1a, 8.1b, 8.2a, and 8.2b in chapter 8, Reliability), as well as to values derived from the English version completed by individuals in the current sample (estimates are not reported as all scales were within .03 of each other across language versions). Overall, results show that the Spanish version of the CAARS 2 is internally consistent and comparable to the English version.

Test Information

Figure 13.2 shows the test information functions for the CAARS 2 by Content and DSM Symptom Scale. As can be seen in the figure, the Spanish versions of the scales display high information across the relevant range of the ability scale (approaching and exceeding 1.5 standard deviations above the mean). Further, the peaks of the information curves are broad with a wide area beneath them, implying the precision of measurement remains consistent across the relevant range of the scales. The peaks of the information functions are also equal to or greater than a value of 10, indicating very high precision and excellent reliability for the Spanish version of the CAARS 2. These information functions meet similar criteria to the English version (see Test Information in chapter 8, Reliability).

Click to expand

Figure 13.2. Test Information Functions by Scale: CAARS 2 Spanish Translation

a) Inattention/Executive Dysfunction

b) Hyperactivity

c) Impulsivity

d) Emotional Dysregulation

e) Negative Self-Concept

f) DSM ADHD Inattentive Symptoms

g) DSM Hyperactive/Impulsive Symptoms

h) DSM ADHD Total Symptoms

Validity

Evidence of validity was first explored for the Content Scales in terms of consistency in the factor structure across languages (explored using a within-subjects measurement invariance approach as per Liu et al., 2017; see also appendix N for an alternative approach that compares this study’s sample to the English version from the Normative Samples). After MI was established, correlations were computed, and mean scale scores were compared for both the Content and DSM Symptom Scales to examine whether scores were consistent between the Spanish and English versions. The CAARS 2–ADHD index, the Associated Clinical Concern Items and Impairment & Functional Outcome Items, and the Validity Scales were analyzed for differences between the Spanish and English translations. Results for all analyses are reported in the following sections.

Content and DSM Symptom Scales

First in the series of analyses, results of the MI investigations are presented in Tables 13.9 and 13.10. Overall, the CAARS 2 Content Scales were found to be invariant across the Spanish and English versions, as evidenced by non-decreasing CFI values and nonsignificant Satorra-Bentler chi-square tests (Satorra & Bentler, 2001). As part of the modeling procedure, some steps required partial invariance adjustments to result in nonsignificant chi-square tests, but the adjustments were infrequent and did not compromise the overall comparability of the scales between the two languages (Dimitrov, 2010). The results provide strong evidence for the validity of the CAARS 2 Spanish translation as a parallel measure to the English version.

Results from the correlation and mean group difference analyses are presented in Table 13.11 and Table 13.12. Corrected correlations (Sacket et al., 2000) revealed statistically significant and strong or very strong relationships across scales between the Spanish and English versions of the CAARS 2, with a median correlation of .93 for the Self-Report (range = .88 to .96) and .84 for the Observer (range = .79 to .84). Further, Welch’s paired t-tests (Welch, 1947) revealed almost no statistically significant differences between scales when comparing obtained scores on the two language versions (p < 0.01); Cohen’s d effect sizes were also negligible (maximum Cohen’s d = 0.19). Taken together, the strong correlations between scales across language versions, and the lack of a statistically significant difference for almost all scale scores (with all scales being practically insignificant) indicates that similar scores can be expected between the Spanish and English versions. These results provide evidence for the validity of the Spanish version of CAARS 2.

CAARS 2–ADHD Index

The CAARS 2–ADHD Index was examined to confirm it performed similarly across the Spanish and English versions (see chapter 12, CAARS 2–ADHD Index, for more information on the development, scores, and psychometric properties of the ADHD Index). The probability scores for the Spanish version of the CAARS 2–ADHD Index were compared to the English version using the Wilcoxon Signed Rank Test (Wilcoxon, 1945); an effect size is also provided, r, which can be interpreted using the correlation guidelines provided in this chapter (Rosenthal, 1991).

The difference in probability scores between the Spanish and English versions was not statistically significant, and effect sizes were very weak (Self-Report: V = 3681, p = .723, r = -0.01; Observer: V = 3003, p = .856, r = -0.02). Thus, the CAARS 2–ADHD Index operates similarly in English as it does in Spanish, adding to the validity evidence for the Spanish translation.

Associated Clinical Concern Items and Impairment & Functional Outcome Items

The Associated Clinical Concern Items and Impairment & Functional Outcome Items of the CAARS 2 were also examined to ensure language versions operated similarly. To gauge this, the proportion of individuals with concordant item elevations or endorsements across language versions was calculated and McNemar’s tests with a continuity correction were performed (see Associated Clinical Concern Items and Impairment & Functional Outcome Items in the French Translation section for more information on these statistics). The Associated Clinical Concerns: Item Selection and Scoring and Impairment & Functional Outcome Items: Item Selection and Scoring sections in chapter 6, Development, also provides more information on how endorsed and elevated responses were determined for items in these scales.

Results of the item-level analyses are presented in Table 13.13. For the Associated Clinical Concern Items and the Impairment & Functional Outcome Items, the percentage of individuals with concordant item endorsements or elevations was very high, with agreement above 90% for nearly all items on both forms. McNemar’s tests also showed that item elevations/endorsements were not significantly more frequent on one language version than the other (p > .01), with the exception of a single item on the Impairment & Functional Outcome Items (though it still displays 91.2% concordance). Taken together, the results demonstrate that elevations and endorsements on the Associated Clinical Concern Items and the Impairment & Functional Outcome Items are highly similar between the Spanish and English versions, supporting the validity of the Spanish translation.

Validity Scales

The CAARS 2 Validity Scales were examined to ensure that they operated similarly in the Spanish and English versions. For both the Negative Impression Index and the Inconsistency Index, the proportion of individuals with concordance across the Spanish and English versions for scale elevations (that is, raw scores that exceed the cut-off) was compared (details provided in the Response Style Analysis: Item Selection and Score Creation section in chapter 6, Development).

As can be seen in Table 13.14, the proportion of individuals with concordant scale elevations was very high (above 90%) for both the Negative Impression Index and Inconsistency Index. Further, McNemar’s tests indicated that scoring above the cut-off on one language version was not statistically significantly more likely than scoring above the cut-off on the other (p < .01). Taken together, both Validity Scales operated similarly in the Spanish and English versions of the CAARS 2 Self-Report and Observer, contributing supporting evidence for the validity of the Spanish language version.

Summary

The reliability and validity of the Spanish (North America) version of the CAARS 2 was examined in a translation study where individuals completed both the Spanish (North America) and the English version consecutively (with order counterbalanced across individuals). Both the Self-Report and Observer form displayed excellent internal consistency and high levels of measurement precision for all Content and DSM Symptom Scales, with coefficients and information functions comparable to those of the English version for both the current sample, as well as the normative sample. This provides strong evidence of the reliability of the Spanish version of the CAARS 2.

Further, it was demonstrated that the Spanish version of the Content Scales were invariant from the English scales, indicating the measurement models of both the Spanish and English versions of the Content Scales are statistically similar. Examination of obtained scores also supported the finding of high scale correlations and no practical mean differences between language versions for the Content Scales and DSM Symptom Scales. Analyses for the ADHD Index, Associated Clinical Concern Items and Impairment & Functional Outcome Items, and Validity Scales also showed high concordance on scale/item-level endorsements and elevations across language versions. Taken together, these findings provide strong evidence for the validity of the Spanish version of the CAARS 2 and justify expectations that scores generated from both the Spanish and English forms should be highly similar.