Conners 4 Manual Chapter 9: Internal Structure

Internal Structure

view all chapter tables | print this section

• Content Scales
• Impairment & Functional Outcome Scales

The internal structure of the Conners 4 Content Scales and Impairment & Functional Outcome Scales were explored, both as a step in the creation of the final forms (see chapter 6, Development, for more details about item and scale selection), and as a step in providing evidence for the validity of the measurement of the intended constructs. The extent to which items interrelate and conform to the theoretical framework can provide evidence for intended interpretation and use of the instrument (AERA, APA, & NCME, 2014). The structure of the Conners 4 was investigated through confirmatory factor analysis (CFA), and alternative and competing measurement models were tested to determine the best fit of the data for the Conners 4 Parent, Teacher, and Self-Report.

Content Scales

The underlying relationships of the Conners 4 Content Scales (i.e., Inattention/Executive Dysfunction, Hyperactivity, Impulsivity, Emotional Dysregulation, Depressed Mood, and Anxious Thoughts) were investigated to provide evidence of the internal structure of the Conners 4. The nature of the multidimensionality of constructs measured in the Conners 4 have been subject to some debate in the literature (e.g., Burns et al., 2001; Conners, 2008; Martel et al., 2016; Nichols et al., 2017; Parke et al., 2015), specifically regarding the separation or unification of Inattention and Executive Dysfunction, as well as whether Hyperactivity and Impulsivity are distinct. To address these considerations, the following models were tested that varied the combination of these factors:

• 5-factor model

  1. Inattention/Executive Dysfunction

  2. Hyperactivity/Impulsivity

  3. Emotional Dysregulation

  4. Depressed Mood

  5. Anxious Thoughts

• 6-factor model

  1. Inattention/Executive Dysfunction

  2. Hyperactivity

  3. Impulsivity

  4. Emotional Dysregulation

  5. Depressed Mood

  6. Anxious Thoughts

• 7-factor model

  1. Inattention

  2. Executive Dysfunction

  3. Hyperactivity

  4. Impulsivity

  5. Emotional Dysregulation

  6. Depressed Mood

  7. Anxious Thoughts

The following criteria for goodness-of-fit statistics were used to evaluate these models:

• Comparative Fit Index (CFI, Bentler,1990): ≥ .90 for acceptable fit and ≥ .95 for good fit (Hu & Bentler, 1999; McDonald & Ho, 2002).

• Tucker-Lewis Index (TLI; Tucker & Lewis, 1973): ≥ .90 for acceptable fit and ≥ .95 for good fit (Hu & Bentler, 1999; McDonald & Ho, 2002).

• Root mean square error of approximation (RMSEA; Browne & Cudeck, 1992): ≤ .08 for acceptable fit and ≤ .06 for good fit.

• Standardized root mean square residual (SRMR; Bentler, 1995): ≤ .08 for good fit.

CFI and TLI range from 0 to 1, with higher values indicating greater fit; conversely, RMSEA and SRMR range from 0 to 1, with lower values indicating better fit. The models were evaluated for statistically significant differences, given their nested structure. A scaled χ² difference statistic with a conservative statistical significance level of p ≤ .01 was deemed meaningful for comparing models, as there were multiple comparisons to be examined and χ² is known to be sensitive to large sample sizes (Tanaka, 1987). The difference in CFI was also evaluated, such that CFI had to improve by more than .01 to be considered a meaningful difference between models (Cheung & Rensvold, 2002). In addition to comparing nested models, the results of each model were evaluated by examining overall fit indices, factor loadings, and correlations among factors. When examining intercorrelations, correlation at or above .95 indicates that the factors are not meaningfully distinct, and parsimony should be favored (i.e., in this case, the selection of the model in which those factors are combined, rather than separated). Additionally, confidence intervals for the inter-factor correlations were examined, and intervals that do not include a value of 1 are understood to indicate distinct constructs (Brown, 2006).

Analyses were conducted with the Total Samples, including all available data from the clinical and general population groups (N = 3,257 for Parent; N = 2,870 for Teacher; and N = 1,589 for Self-Report; see Standardization Phase in chapter 6, Development, for details about these samples), using correlated-factor models with robust estimation methods for ordinal items via the lavaan package in R (Rosseel, 2012). As can be seen in Table 9.1, results for these competing models for Parent, Teacher, and Self-Report all demonstrated strong fit and performed similarly to one another. The fit indices met or exceeded typical guidelines for good fit. Model fit improved (i.e., CFI and TLI increased, and SRMR and RMSEA decreased) as more factors were added to the model.

While all models displayed good fit to the data, a series of analyses were conducted to determine which model had the best fit across Conners 4 Parent, Teacher, and Self-Report forms. Results of the χ² difference test for nested models can be seen in Table 9.2. All model comparisons displayed statistically significant differences (p < .01), indicating that a greater number of factors did significantly improve fit, yet models with additional factors did not show a meaningful gain in CFI (ΔCFI < .01 for all comparisons). Therefore, further investigation was warranted to determine the most appropriate model for the data.

Inspection of the inter-factor correlations of the models was the final step in this series of analyses. In the 7-factor model, Inattention and Executive Dysfunction were correlated at or above the recommended threshold for meaningfully distinct factors (Parent r = .961, Teacher r = .978, Self-Report r = .950). Given this finding, the 7-factor model was rejected, as separating Inattention and Executive Dysfunction was not supported by empirical evidence. All goodness-of-fit statistics and the χ² difference test indicated that the 5-factor model performed worse than the 6-factor model; therefore, the 6-factor model was inspected further. Examining the inter-factor correlations of the 6-factor model, as seen in Tables 9.3 to 9.5, it can be determined that Hyperactivity and Impulsivity are strongly correlated but are not entirely overlapping constructs (Parent r = .855, Teacher r = .943, Self-Report r = .829). In addition, the confidence intervals for these correlations did not include a correlation of 1, providing further evidence of their distinctions. Therefore, the 5-factor model was also rejected. In the end, the 6-factor model performed best for the Conners 4 Content Scales across Parent, Teacher, and Self-Report.

The factor loadings of the final 6-factor models were inspected, and all loadings (see Tables 9.6 to 9.8) were positive, statistically significant, and exceeded a typical minimum threshold (loading ≥ .32, although all loadings except for one exceeded a higher threshold of .40; Tabachnick & Fidell, 2007). The strength of this model provides strong evidence for the structural validity of the Conners 4 Content Scales.

Impairment & Functional Outcome Scales

Similar to the Conners 4 Content Scales, the internal structure of the Conners 4 Impairment & Functional Outcome Scales was also explored. Three models were compared for Parent and Self-Report: a 1-factor model for the broad construct of impairment; a 3-factor model that distinguishes the three domains of school, peer, and family settings; and a 3-factor model that also includes a method factor to account for the reverse-scored nature of some of the items (i.e., the common method of responding given the valence of the items). Three items on each form are reserved-scored (note that in the tables within this section, item stems are used that align the item’s valence with the rest of the scale; see appendix C for full item text). For the Teacher form (which does not include items about Family Life), a broad 1-factor model of impairment was compared against a 2-factor model that includes Schoolwork and Peer Interactions, which was then compared against a 2-factor model that included a method factor. Model fit and model comparisons were evaluated against the same criteria described in the Content Scales section above.

Analyses were conducted with the Total Samples, including all available data from the clinical and general population groups (N = 3,257 for Parent; N = 2,870 for Teacher; and N = 1,589 for Self-Report; see Standardization Phase in chapter 6, Development, for details about these samples), using the lavaan package in R (Rosseel, 2012); results are presented in Table 9.9. The results for Parent and Self-Report demonstrate that the 3-factor model outperformed the 1-factor model for all rater forms. The goodness-of-fit statistics indicate better fit for the 3-factor model, as well as a significant difference between the models (see Table 9.10), as evidenced by a statistically significant χ² difference test (p < .001 for Parent and Self-Report) and a meaningful change in CFI (ΔCFI=.064 for Parent, and .078 for Self-Report, both substantially larger than the typical guideline of ΔCFI=.010 [Cheung & Rensvold, 2002]). Results showed a similar pattern for Teacher, with the 2-factor model outperforming the 1-factor model (χ² difference p < .001, ΔCFI=.053). Therefore, the 1-factor model of general impairment was rejected for all forms.

Next, the 3-factor model with a method factor for Parent and Self-Report and the 2-factor model with a method factor for Teacher were examined. Although the goodness-of-fit statistics show a strong model fit for these alternative models, there does not appear to be a significant improvement in fit when the method factor is added. Given the limited improvement in fit, the 3-factor model for Parent and Self-Report and the 2-factor model for Teacher were selected as the optimal solutions.

Correlations between the factors ranged from .616 to .822 across the scales and rater forms, demonstrating positive and moderate to very strong relationships, as seen in Table 9.11. Detailed inspection of the selected models revealed strong fit in terms of factor loadings (see Tables 9.12 to 9.14). The factor loadings were statistically significant (p < .01), positive, and clearly associated with the intended factor (loadings ranged from .35 to .94 for Parent [all loadings except for one were above .40], .54 to .93 for Teacher, and .54 to .85 for Self-Report).

The strength of these final models provides strong evidence for the internal structure of the Conners 4 Impairment & Functional Outcome Scales.

< Back

Next >