Manual

Conners 4 Manual

Chapter 10: Country of Residence


Country of Residence

view all chapter tables | print this section

To address the equivalence of scores across countries, youth in the U.S. and Canada were compared on the Conners 4. Cross-cultural differences were expected to be minimal, and the lack of meaningful differences would support the generalizability and utility of the Conners 4 in both countries.

The invariance of the factor structure was compared based on the youth’s country of residence. Results examining MI between the U.S. (N = 2,838 for Parent and N = 2,563 for Teacher) and Canada (N = 376 for Parent and N = 301 for Teacher) are found in Tables 10.23 and 10.24. MI for Self-Report was not explored due to the small sample size for Canadians (N = 199), as the large number of items on the scales (N = 60 Content Scale items on Self-Report) and ordinal nature of the items requires a more substantial sample size.

For Parent, the results (as seen in Table 10.23) indicate no meaningful change across models. The Satorra-Bentler χ2 test was significant for the intercept model, but other fit statistics did not decline, indicating no meaningful change and therefore no compromise regarding invariance. Across the Conners 4 Teacher scales (as seen in Table 10.24), there were no statistically significant reductions in the Satorra-Bentler χ2 test or any declines in the majority of model fit statistics when comparing different levels of invariance. Because the Conners 4 meets the most stringent level of invariance tested in terms of country of residence, these results support generalizability of the Conners 4 Parent and Teacher forms to both American and Canadian youth regarding factor structure, loadings, thresholds, and intercepts.


Click to expand

Table 10.23. Measurement Invariance by Country of Residence: Conners 4 Parent

Scales

Model

χ2

df

RMSEA

CFI

TLI

SRMR

Comparison

Satorra-Bentler χ2

df

CFI

Content Scales

Configural

10963.89***

3274

.038

.973

.972

.040

Threshold

10978.90***

3333

.038

.973

.973

.040

configural v. threshold

64.08

59

.000

Loading

10879.70***

3386

.037

.974

.974

.040

threshold v. loading

61.18

53

.001

Intercept

10756.99***

3439

.036

.975

.975

.040

loading v. intercept

76.58*

53

.001

Impairment & Functional Outcome Scales

Configural

2117.22***

298

.062

.983

.980

.044

Threshold

2142.16***

317

.060

.983

.981

.044

configural v. threshold

20.86

19

.000

Loading

2064.74***

333

.057

.984

.983

.044

threshold v. loading

16.32

16

.001

Intercept

1972.51***

349

.054

.985

.985

.044

loading v. intercept

16.93

16

.001

DSM Oppositional Defiant Disorder Symptoms Scale

Configural

835.04***

70

.082

.983

.978

.038

Threshold

868.99***

80

.078

.982

.980

.038

configural v. threshold

11.99

10

.001

Loading

805.67***

89

.071

.984

.984

.038

threshold v. loading

7.84

9

.002

Intercept

730.66***

98

.063

.986

.987

.038

loading v. intercept

7.33

9

.002

DSM Conduct Disorder Symptoms Scale

Configural

1018.57***

180

.054

.970

.966

.087

Threshold

1034.48***

193

.052

.970

.968

.087

configural v. threshold

11.96

13

.000

Loading

965.43***

207

.048

.973

.973

.087

threshold v. loading

12.39

14

.003

Intercept

857.32***

219

.043

.978

.978

.088

loading v. intercept

12.89

12

.005

Note. N = 2,838 Americans; N = 376 Canadians. RMSEA = Root mean square error of approximation; CFI = Comparative Fit Index; TLI = Tucker-Lewis Index; SRMR = Standardized root mean square residual; ∆CFI = change in CFI. *p < .05, **p < .01, ***p < .001.


Click to expand

Table 10.24. Measurement Invariance by Country of Residence: Conners 4 Teacher

Scales

Model

χ2

df

RMSEA

CFI

TLI

SRMR

Comparison

Satorra-Bentler χ2

df

CFI

Content Scales

Configural

12248.98***

3274

.044

.971

.970

.052

Threshold

12231.60***

3333

.043

.972

.971

.052

configural v. threshold

60.41

59

.001

Loading

12087.58***

3386

.042

.972

.972

.052

threshold v. loading

66.61

53

.000

Intercept

11803.73***

3439

.041

.973

.974

.052

loading v. intercept

69.07

53

.001

Impairment & Functional Outcome Scales

Configural

1426.14***

106

.093

.981

.976

.054

Threshold

1466.93***

118

.089

.980

.978

.054

configural v. threshold

13.63

12

.001

Loading

1410.26***

128

.084

.981

.981

.054

threshold v. loading

9.72

10

.001

Intercept

1350.45***

138

.078

.982

.983

.054

loading v. intercept

11.42

10

.001

DSM Oppositional Defiant Disorder Symptoms Scale

Configural

992.76***

70

.096

.986

.982

.036

Threshold

989.52***

80

.089

.986

.985

.036

configural v. threshold

10.18

10

.000

Loading

926.14***

89

.081

.987

.987

.036

threshold v. loading

8.44

9

.001

Intercept

852.16***

98

.073

.989

.990

.036

loading v. intercept

11.36

9

.002

DSM Conduct Disorder Symptoms Scale

Configural

835.26***

130

.062

.967

.960

.109

Threshold

853.18***

142

.059

.967

.963

.109

configural v. threshold

10.14

12

.000

Loading

790.80***

154

.054

.970

.970

.109

threshold v. loading

10.97

12

.003

Intercept

747.30***

165

.050

.973

.974

.109

loading v. intercept

14.39

11

.003

Note. N = 2,563 Americans; N = 301 Canadians. RMSEA = Root mean square error of approximation; CFI = Comparative Fit Index; TLI = Tucker-Lewis Index; SRMR = Standardized root mean square residual; ∆CFI = change in CFI. *p < .05, **p < .01, ***p < .001.


Next, differential test functioning was evaluated with regard to country of residence. DTF results are presented graphically in Figure 10.3 for the Inattention/Executive Dysfunction scale, as an example of the trends seen across all scales and rater forms. Test functioning curves for youth in the U.S. and Canada are depicted, along with a shaded band to display a 95% confidence interval; the two groups’ curves are almost completely overlapping, demonstrating a lack of difference between groups. Results in terms of effect sizes of the DTF analyses for all forms are presented in Table 10.25. Negligible differences between countries were found (i.e., effect sizes at or below |.06|), which further supports the generalizability of the Conners 4 to U.S. and Canadian populations alike.


Figure 10.3. Differential Test Functioning by Country of Residence: Inattention/Executive Dysfunction

a) Parent

Parent

b) Teacher

Teacher

c) Self-Report

Self-Report


Click to expand

Table 10.25. Differential Test Functioning Effect Sizes by Country of Residence

Scale

Parent

Teacher

Self-Report

Content Scales

Inattention/Executive Dysfunction

−.01

.03

.00

Hyperactivity

.03

.01

.02

Impulsivity

.00

.01

−.04

Emotional Dysregulation

.01

−.01

.05

Depressed Mood

.01

−.06

−.05

Anxious Thoughts

−.01

−.01

.03

Impairment & Functional Outcome Scales

Schoolwork

.01

−.04

.00

Peer Interactions

.03

.01

.00

Family Life

−.02

.01

DSM Symptom Scales

Oppositional Defiant Disorder Symptoms

.01

−.01

−.02

Conduct Disorder Symptoms

−.01

.03

.01

Note. Values presented are expected test score standardized differences (ETSSD); guidelines for interpreting |ETSSD|: negligible effect size < 0.20; small effect size = 0.20 to 0.49; medium effect size = 0.50 to 0.79; large effect size ≥ 0.80. Positive ETSSD values indicate that Canadian youth received higher scores than American youth who had the same level of the construct being measured.


To examine observed group differences between countries, a subsample of the youth from the U.S. were randomly selected to match the sample of Canadians from the Normative Samples. Youth were matched by gender, PEL (PEL was matched for Parent and Self-Report only), language(s) spoken, clinical status, and age. Refer to Table F.43 in appendix F for the demographic characteristics of the American and Canadian paired samples for all Conners 4 forms. Details about the raters for the Parent and Teacher samples are provided in appendix F in Table F.44.

The paired samples of Americans and Canadians were then compared for significant differences across mean scores. Results of the ANOVAs and descriptive statistics for each scale are presented in Tables 10.26 to 10.28. Across all rater forms and all scales, there were no statistically significant effects of country of residence. Ratings of youth from the U.S. and Canada resulted in very similar mean scores as described by Cohen’s d; all reported effect sizes were negligible to small (with Cohen’s d ranging from 0.00 to |0.31|). These results indicate that country of residence (specifically, U.S. vs. Canada) had no significant effect on the Conners 4 scale scores. Along with the results of MI and DTF demonstrating equivalence, there is evidence that the Conners 4 is appropriate for use with American and Canadian populations alike.



Click to expand

Table 10.27. Group Differences by Country of Residence: Conners 4 Teacher

Scale

U.S.
(N = 119)

Canada
(N = 119)

Cohen’s d

F
(1, 236)

p

η2

Content Scales

Inattention/Executive
Dysfunction

M

49.2

48.6

0.06

0.21

.648

.00

SD

10.2

9.4

Hyperactivity

M

50.3

49.0

0.14

1.12

.291

.00

SD

9.8

9.1

Impulsivity

M

49.4

48.5

0.10

0.57

.453

.00

SD

9.2

9.0

Emotional Dysregulation

M

49.3

48.9

0.04

0.12

.734

.00

SD

8.6

9.4

Depressed Mood

M

50.1

49.5

0.06

0.21

.648

.00

SD

9.4

10.8

Anxious Thoughts

M

50.6

49.1

0.15

1.33

.250

.01

SD

9.5

10.2

Impairment & Functional Outcome Scales

Schoolwork

M

48.3

48.7

−0.04

0.10

.752

.00

SD

10.4

9.2

Peer Interactions

M

49.1

49.0

0.01

0.01

.927

.00

SD

9.3

8.9

DSM Symptom Scales

Oppositional Defiant Disorder Symptoms

M

48.9

48.9

0.00

0.00

.998

.00

SD

8.2

8.7

Conduct Disorder Symptoms

M

49.5

50.4

−0.08

0.42

.518

.00

SD

8.6

13.5

Note. Guidelines for interpreting η2: negligible effect size < .01; small effect size = .01 to .05; medium effect size = .06 to .13; large effect size ≥ .14. Guidelines for interpreting Cohen’s |d|: negligible effect size < 0.20; small effect size = 0.20 to 0.49; medium effect size = 0.50 to 0.79; large effect size ≥ 0.80. A positive Cohen’s d value indicates that ratings of American youth resulted in higher scores than ratings of Canadian youth.



< Back Next >