Conners 4 Manual Chapter 12: Fairness

Fairness

view all chapter tables | print this section

• Gender
• Race/Ethnicity
• Country of Residence
• Parental Education Level

To provide evidence that the Conners 4–ADHD Index provides a fair and unbiased measurement for diverse populations, differences between demographic groups were examined. Measurement invariance was explored via differential test functioning (DTF), with a visual inspection of test characteristic curves for each group (see chapter 10, Fairness, for a detailed explanation and sample graphs). A summary of corresponding effect sizes for the differences in test characteristic curves is provided throughout this section, presented as the estimated test score standardized difference (ETSSD).

Mean differences between the classification categories of the probability score were explored in terms of effect size of the difference between the groups, via Cliff’s d (see chapter 10, Fairness, for a detailed explanation of these methods). Given the multidimensional nature of ADHD (as evidenced by the factor structure presented in Internal Structure in chapter 9, Validity), and that the Conners 4–ADHD Index comprises items from various content domains, measurement invariance testing with nested confirmatory factor analyses (CFAs) was not practical, as the Conners 4–ADHD Index was not built to mirror the full multidimensional structure. The items on the Conners 4–ADHD Index are treated as key indicators that can reflect one’s group membership (i.e., likelihood of resembling scores from the ADHD Reference Sample) and represent a unidimensional measure of ADHD-related symptoms; therefore, DTF alone was a sufficient method of evaluating invariance. Prior to conducting the DTF analyses, CFAs were conducted to ensure the statistical requirement for unidimensionality was met. Results supported a 1-factor model for all rater forms (Parent: CFI = .990, RMSEA = .076 [95% CI = .072, .080], SRMR = .027; Teacher: CFI = .980, RMSEA = .089 [95% CI = .085, .093], SRMR = .052; Self-Report: CFI = .958, RMSEA = .088 [95% CI = .082, .093], SRMR = .059).

Because invariance testing relies on modeled data (i.e., estimating the population, rather than describing the sample), larger sample sizes are required, and a greater range of responses is desired. Therefore, the Total Sample (as described in the Standardization Phase in chapter 6, Development) is used for these invariance analyses, because it includes a considerable number of individuals from the general population, as well as individuals with clinical diagnoses (which will extend the variability of responses).

To examine the generalizability of the obtained scores, the effects of demographic group membership were analyzed by a comparison of effect sizes between response distributions of the groups. Group differences were calculated on subsets of the Normative Samples (see appendix F for demographic characteristics of the samples used in this section).

Gender

The invariance between males and females for the Conners 4–ADHD Index was explored via DTF analyses using the Total Samples for each rater form. The effect size of the DTF statistic measured by the ETSSD was 0.01 for Parent, 0.03 for Teacher, and -0.02 for Self-Report. Effects of this magnitude are all trivial in size. These results from the DTF analyses indicate that the Conners 4–ADHD Index demonstrates psychometric equivalence across males and females, as there was no evidence for meaningful differences in test functioning between the two groups.

To further explore potential gender differences, observed group differences between scores on the Conners 4–ADHD Index were also investigated. These group differences were analyzed using a matched sample (matched by age, language[s] spoken, clinical status, race/ethnicity, and PEL [for Parent and Self-Report only]; refer to Table F.36 and Table F.37 in appendix F for demographic characteristics of the matched samples). Differences are presented as Cliff’s d values, comparing the frequency of each probability score category within each gender group. Results of these analyses are presented in Table 12.27 for all forms. Effect sizes were negligible for Parent and Self-Report (Cliff’s d = -.03 for Parent and .05 for Self-Report). For Teacher, the difference was a small effect (Cliff’s d = -.25). Teacher ratings resulted in male students being more likely to fall into the High or Very High category than female students; however, the overall distribution of probability scores only revealed a small difference.

Overall, the small differences observed between gender groups add support for the generalizable use of the Conners 4–ADHD Index across males and females. Together with the DTF results, there is evidence for equitable measurement by gender when using the Conners 4–ADHD Index.

Race/Ethnicity

Invariance between Hispanic and White youth and between Black and White youth for the Conners 4–ADHD Index was explored via DTF analyses using the Total Sample for each rater form (note that the sample size for Asian youth was too small to permit DTF). When comparing Hispanic and White youth, the ETSSD effect size was 0.01 for Parent, 0.02 for Teacher, and 0.02 for Self-Report. When comparing Black and White youth, effect sizes were -0.01 for Parent, 0.08 for Teacher, and -0.01 for Self-Report. All effects are trivial in size, indicating no evidence of DTF by race/ethnicity. These results demonstrate a lack of measurement bias between White and Black, and White and Hispanic groups, reinforcing the generalizability of the Conners 4–ADHD Index and demonstrating psychometric equivalence for White and Hispanic and White and Black youth.

To further explore potential race/ethnicity differences, observed group differences between scores on the Conners 4–ADHD Index were also investigated. These group differences were analyzed using a matched sample from the Normative Sample (matched on age, gender, language[s] spoken, clinical status, and PEL [PEL only for Parent and Self-Report]; refer to Table F.38 to Table F.42 in appendix F for demographic characteristics of the matched samples). Results of these analyses for all forms are presented in Table 12.28 for White/Hispanic comparison, Table 12.29 for White/Black comparison, and Table 12.30 for White/Asian comparison. Effects were negligible when comparing ratings of Hispanic and White youth (Cliff’s d ranging from -.03 to .05) for all three rater types. When comparing ratings of Black and White youth, effects were negligible for Parent (Cliff’s d = .05) and Self-Report (Cliff’s d = -.06). Teacher ratings of Black students were slightly higher than teacher ratings of White students; however, the effect size was small (Cliff’s d = .18). Finally, when comparing ratings of Asian and White youth, effects were negligible for Parent (Cliff’s d = -.03) and Self-Report (Cliff’s d = -.11). Teacher revealed slightly lower ratings for Asian, compared to White youth; however, the effect size was small (Cliff’s d = -.18).

Overall, the small differences observed between race/ethnicity groups add support for the fair and generalizable use of the Conners 4–ADHD Index across these populations, and, together with the DTF results, provide evidence for equitable measurement by race/ethnicity when using the Conners 4–ADHD Index.

Country of Residence

To address equivalence of scores across countries, youth in the U.S. and Canada were compared on the Conners 4–ADHD Index. Cross-cultural differences were expected to be minimal, and the lack of meaningful differences would support the generalizability and utility of the Conners 4–ADHD Index for use in both the U.S. and Canada. The invariance by country of residence for the Conners 4–ADHD Index was explored via DTF analyses using the Total Samples for each rater form. The effect size of the DTF statistic was negligible for all forms (ETSSD = -0.01 for Parent, -0.05 for Teacher, and -0.01 for Self-Report; negative ETSSD values indicate higher expected scores for Americans than Canadians even when matched on the construct being measured). Trivial differences were found (i.e., effect sizes at or below |.05|), which demonstrates the invariance of the Conners 4–ADHD Index across countries and support its generalizability to U.S. and Canadian populations.

To further explore potential country of residence differences, the Conners 4–ADHD Index was also investigated in terms of observed group differences between scores. These group differences were analyzed using a matched sample (matched on age, gender, language[s] spoken, clinical status, and PEL [PEL only for Parent and Self-Report]; refer to Table F.43 and Table F.44 in appendix F for demographic characteristics of the matched samples). Group differences were analyzed via Cliff’s d effect size values, which compared the percentage of the sample scoring within each category of the Conners 4–ADHD Index probability score between the two countries. Results of these analyses are presented in Table 12.31 for all forms. Effect sizes were negligible for all comparisons (Cliff’s d = -.06 for Parent, Cliff’s d = -.03 for Teacher, and Cliff’s d = .02 for Self-Report).

Overall, the very small differences observed between the country of residence groups add support for the fair and generalizable use of the Conners 4–ADHD Index with individuals from the U.S. and Canada, and together with the DTF results, provide evidence for the equitable measurement of individuals from American and Canadian populations using the Conners 4–ADHD Index.

Parental Education Level

Parental education level (PEL) can sometimes be considered a proxy for, or a contributing factor to, one’s socioeconomic status (SES), with SES being a characteristic upon which fairness could unduly vary. It was expected that the constructs measured on the Conners 4 would be independent of influence from PEL. To test this hypothesis and ensure generalizability of scores from the Conners 4–ADHD Index, individuals in the Parent and Self-Report samples reported the PEL of the rated individual from one of five options: No high school diploma (PEL 1), High school diploma/GED (PEL 2), Some college or associate’s degree (PEL 3), Bachelor’s degree (PEL 4), and Graduate or professional degree (PEL 5). Within the normative samples, the proportion of parents in each of these groups matched recent U.S. and Canadian census values (more information can be found in Parental Education Level in chapter 7, Standardization).

Equivalence across the PEL groups was investigated with DTF analyses using the Total Samples for the Parent and Self-Report. For the sake of DTF analyses, which require binary variables, PEL was re-categorized into two groups: parents without post-secondary education (PEL 1 and PEL 2; N = 855 for Parent and N = 530 for Self-Report) and parents with post-secondary education (PEL 3, PEL 4, and PEL 5; N = 2,385 for Parent and N = 1,057 for Self-Report). The DTF results revealed trivial effect sizes (ETSSD = 0.01 for Parent, and -0.02 for Self-Report). The trivial values observed in this analysis support the invariance of the Conners 4–ADHD Index with regard to the educational background of the parent of the youth being rated.

To further explore potential PEL differences, the Conners 4–ADHD Index was also investigated for potential group differences in terms of the frequency distribution of scores for each group. These group differences were analyzed using the entire Normative Samples. While previous sections of this chapter looked at group differences based on matched subsamples of the Normative Sample, it was not possible to create demographically matched groups for the PEL analysis due to the small sample sizes of the PEL groups. Given the lack of differences found for other demographic variables (as evidenced by results earlier in this section), results for the PEL analysis are unlikely to be influenced by the inclusion of covariates. Differences are presented as Cliff’s d values, which compare the percentage of the sample scoring within each category of the Conners 4–ADHD Index probability score between the PEL groups. Results of these analyses are presented in Table 12.32 (a to b) for Parent and Self-Report. Effect sizes were negligible for all comparisons (Cliff’s d ranges from -.04 to .04 for Parent and -.06 to .09 for Self-Report).

Overall, the very small differences observed between PEL groups demonstrate a lack of influence of parental education level on the Conners 4–ADHD Index. This finding adds support for the fair and generalizable use of the Conners 4–ADHD Index, and together with the DTF results, provides evidence for equitable measurement across PEL groups when using the Conners 4–ADHD Index.

< Back

Next >