Conners 4 Manual Chapter 9: Relation to Criterion Variables

Relation to Criterion Variables

view all chapter tables | print this section

• Clinical Group Differences
• ADHD and General Population Comparisons
• Depression and General Population Comparisons
• Generalized Anxiety Disorder and General Population Comparisons
• Disruptive Disorders and General Population Comparisons
• ADHD and Depression/Anxiety Comparisons
• Classification Accuracy
• ADHD vs. General Population Classifications
• Depression, Generalized Anxiety Disorder, or Oppositional Defiant Disorder vs. General Population Classifications

Clinical Group Differences

To provide evidence of the criterion-related validity of the scores from the Conners 4, differences among test scores from various relevant clinical groups and the general population were explored. Mean score differences were compared within the following groups:

• Youth diagnosed with ADHD Inattentive Presentation (ADHD Inattentive), ADHD Hyperactive/Impulsive Presentation (ADHD Hyperactive/Impulsive), ADHD Combined Presentation (ADHD Combined), and youth from the general population.

• Youth diagnosed with a depressive disorder, referred to as the Depression group (diagnostic groups include Major Depressive Disorder and Persistent Depressive Disorder) and youth from the general population.

• Youth diagnosed with Generalized Anxiety Disorder and youth from the general population.

• Youth diagnosed with a Disruptive, Impulse-Control, or Conduct Disorder, referred to as the Disruptive Disorders group (diagnostic groups include Oppositional Defiant Disorder, Conduct Disorder, and Intermittent Explosive Disorder) and youth from the general population.

• Youth diagnosed with ADHD (all presentations) and youth diagnosed with Depression or Anxiety (diagnostic groups include Major Depressive Disorder, Persistent Depressive Disorder, Generalized Anxiety Disorder, Panic Disorder, Separation Anxiety, and Social Anxiety Disorder).

All scale-level analyses were conducted via a series of analysis of variance tests (ANOVA; conducted in R via the stats package, version 3.6.1; R Core Team, 2013). Given the large number of comparisons conducted, a conservative significance level (p < .01) was adopted to determine statistical significance. Effect sizes, as measured by eta-squared (η²; see Cohen, 1973, for detailed interpretation) and by Cohen’s d, are provided for all analyses.

The magnitude of the differences between T-scores for each set of groups was explored using Cliff’s d for the Critical & Indicator Items pertaining to self-harm and sleep problems. The Severe Conduct Critical Items were omitted from the analyses because these items are included in the DSM Conduct Disorder Symptoms Scale analyses. For all Critical & Indicator Item analyses, Self-Harm Critical Items are considered endorsed by the rater at an item response of 2 or higher for all raters. For the Sleep Problems Indicator, the “having trouble sleeping” item is considered to be endorsed by the rater at an item response of 2 or higher for Parent and Self-Report (this item does not appear on Teacher), and the“appearing/feeling tired” item is considered to be endorsed by the rater at an item response of 2 or higher for Parent and Teacher, and a response of 3 for Self-Report (see chapter 6, Development, for more information on these cut-offs for endorsement).

ADHD and General Population Comparisons

Mean score differences between youth from the general population and those with ADHD (including those diagnosed with ADHD Inattentive, ADHD Hyperactive/Impulsive, and ADHD Combined) were compared across Conners 4 Content Scale scores, Impairment & Functional Outcome Scale scores, and DSM Symptoms Scale scores. Item-level ratings across groups were also explored for the Self-Harm Critical Items and Sleep Problems Indicator. To facilitate comparisons between groups, a subsample of the general population sample was selected to match the demographics of the full ADHD sample (i.e., unifying ADHD Inattentive, ADHD Hyperactive/Impulsive, and ADHD Combined), in terms of age, gender, race/ethnicity, and parental education level (PEL; note that PEL is matched for Parent and Self-Report only, as this information was not provided by Teacher raters). Many of the youth in the ADHD sample were also diagnosed with co-occurring disorders (46.6% of the Parent sample, 51.3% of the Teacher sample, and 43.7% of the Self-Report sample had a co-occurring disorder). The most common co-occurring diagnoses in the sample were depressive disorders, anxiety disorders, and disruptive disorders, while other diagnoses were less commonly reported (e.g., Autism Spectrum Disorder, Learning Disorders). The demographic characteristics of the General Population and ADHD groups and their raters are presented in appendix F in Tables F.12 to F.16.

As described in chapter 1, Introduction, the Conners 4 includes scales that measure

• symptoms related to ADHD (i.e., Inattention/Executive Dysfunction, Hyperactivity, Impulsivity, Emotional Dysregulation, and the three DSM ADHD Symptoms scales);

• impairments related to ADHD (i.e., Schoolwork, Peer Interactions, and Family Life); and

• symptoms related to commonly co-occurring disorders (i.e., Depressed Mood, Anxious Thoughts, DSM Oppositional Defiant Disorder Symptoms, and DSM Conduct Disorder Symptoms).

It was expected that large differences would be observed between the ADHD and General Population groups on the scales that measure specific ADHD symptoms and impairments. Additionally, due to the high level of co-occurring disorders in youth with ADHD in general (see Gnanavel et al., 2019 for a review), and high proportion observed in the study’s ADHD groups, it was expected that compared to the General Population, youth with ADHD would have higher scores on the scales related to commonly co-occurring disorders. While all ADHD presentations were expected to display higher scores than the General Population across scales, it was also expected that when comparing scores between ADHD presentations, youth with ADHD Hyperactive/Impulsive Presentation and ADHD Combined Presentation would exhibit comparatively higher scores on scales indexing hyperactive and/or impulsive symptoms than youth diagnosed with ADHD Inattentive Presentation.

Comparisons between these four groups were analyzed with a series of ANOVAs, and significant omnibus F-tests were followed up with Tukey’s honestly significant difference (HSD) post-hoc tests for pairwise comparisons. Results are displayed in Tables 9.25 to 9.27. Mean scale scores are also depicted graphically in Figures 9.1 to 9.3. As expected, significant differences were observed for nearly all scale level comparisons across Parent, Teacher, and Self-Report. The size of these differences varied and are reported in terms of η² and Cohen’s d effect size values.

Symptoms Related to ADHD. First, results were reviewed for the scales that measure ADHD symptoms (i.e., Inattention/Executive Dysfunction, Hyperactivity, Impulsivity, Emotional Dysregulation, and the three DSM ADHD Symptom Scales). Effect sizes for the omnibus F-tests were all large on the Conners 4 Parent (η² ranged from .28 to .54) and were medium to large for the Conners 4 Teacher (η² ranged from .10 to .23) and the Conners 4 Self-Report (η² ranged from .07 to .24). Across all three rater-types, the largest effects were observed on the Inattention/Executive Dysfunction scale. In looking at specific pairwise comparisons, results of the ADHD and General Population comparisons were first explored, followed by an exploration of the differences between the ADHD groups.

When comparing the individual ADHD groups to the General Population group, for Conners 4 Parent, Conners 4 Teacher, and Conners 4 Self-Report, all three ADHD groups scored significantly higher than the General Population on the majority of these scales, with medium to large effect sizes for the significant effects (Parent Cohen’s d = 0.85 to 2.39, Teacher Cohen’s d = 0.65 to 1.23, Self-Report Cohen’s d = 0.57 to 1.36). The moderate to large systematic differences in scale scores between the General Population and the ADHD groups provide strong evidence to support the validity of the Conners 4, as they demonstrate that the Conners 4 scores successfully distinguish the profiles of youth with and without ADHD.

Significant differences between presentations of ADHD (i.e., Inattentive vs. Hyperactive/Impulsive vs. Combined) were also observed. As expected, when scores from the Hyperactivity, Impulsivity, DSM ADHD Hyperactive/Impulsive Symptoms, and DSM Total ADHD Symptoms scales were examined for the Conners 4 Parent and Conners 4 Teacher, the ADHD Combined and ADHD Hyperactive/Impulsive groups did not differ significantly from each other. However, both groups scored higher than the ADHD Inattentive group, with small to large effect sizes for the significant effects (Parent Cohen’s d ranged from 0.42 to 1.27, Teacher Cohen’s d ranged from 0.47 to 0.89). A similar, though slightly less pronounced, pattern was observed on the Conners 4 Self-Report. Specifically, the ADHD Hyperactive/Impulsive group had significantly higher scores than the ADHD Inattentive group on the Hyperactivity and DSM ADHD Hyperactive/Impulsive Symptoms scales with medium effect sizes (Cohen’s d = 0.73 and 0.74, respectively). These results provide evidence to support the validity of the Conners 4 scores, as clear differences between ADHD groups emerged and were in line with expectations.

Impairment & Functional Outcome Scales. Next, results for the Impairment & Functional Outcome Scales (Schoolwork, Peer Interactions, and Family Life [for Parent and Self-Report only]) were examined. All ANOVAs were statistically significant, and effect sizes were all large for the Conners 4 Parent (η² ranged from .31 to .37), and medium to large for the Conners 4 Teacher (η² ranged from .13 to .18) and Conners 4 Self-Report (η² ranged from .09 to .16). While there were significant effects in all areas of impairment, the largest differences were on the Schoolwork scale across all three rater types.

All three ADHD groups scored significantly higher than the General Population group on all scales, with medium to large effect sizes (Parent Cohen’s d = 1.09 to 1.74, Teacher Cohen’s d = 0.50 to 1.06, Self-Report Cohen’s d = 0.52 to 0.93). These results provide evidence to demonstrate that the Conners 4 Impairment & Functional Outcome Scales distinguish effectively between the levels of impairment displayed by youth with and without ADHD.

No significant differences between presentations of ADHD (i.e., Inattentive vs. Hyperactive/Impulsive vs. Combined) were observed on the Schoolwork scale for the three rater types. However, parents and teachers indicated higher levels of impairment in Peer Interactions for the ADHD Hyperactive/Impulsive and ADHD Combined groups than the ADHD Inattentive group, with small effect sizes (Parent Cohen’s d = 0.45, Teacher Cohen’s d = 0.32 to 0.41). Additionally, parents indicated higher levels of impairment in Family Life for the ADHD Combined group than the ADHD Inattentive group, with a small effect size (Cohen’s d = 0.48). For Self-Report, there were no significant differences between presentations of ADHD on the Peer Interactions or Family Life scales.

Commonly Co-occurring Symptom Scales. Results were examined for the scales that assess commonly co-occurring symptoms (i.e., Depressed Mood, Anxious Thoughts, DSM Oppositional Defiant Disorder Symptoms, and DSM Conduct Disorder Symptoms). The effect sizes were medium to large for the Conners 4 Parent (η² ranged from .06 to .23), and small to medium for the Conners 4 Teacher (η² ranged from.03 to .09) and Conners 4 Self-Report (η² ranged from .01 to .09). The ADHD groups tended to score higher than the General Population group on these scales; however, not all comparisons were statistically significant. For the Depressed Mood and Anxious Thoughts scales, the majority of the ADHD groups scored significantly higher than the General Population group, with small to large effect sizes for the significant effects (Parent Cohen’s d = 0.78 to 1.01, Teacher Cohen’s d = 0.38 to 0.48, Self-Report Cohen’s d = 0.54 to 0.76). For the DSM Oppositional Defiant Disorder Symptoms scale, while all ADHD groups scored higher than the General Population on the Parent form, only the ADHD Hyperactive/Impulsive and ADHD Combined groups scored higher than the General Population group on the Teacher and Self-Report forms, with moderate to large effect sizes for the significant effects (Parent Cohen’s d = 0.84 to 1.36, Teacher Cohen’s d = 0.65 to 0.86, Self-Report Cohen’s d = 0.50 to 0.75). Fewer significant differences and smaller effect sizes were found on the DSM Conduct Disorder Symptoms scale. This pattern was expected, as very few youth in the ADHD sample were diagnosed with Conduct Disorder. Very few differences were found between presentation of ADHD on these scales.

Critical & Indicator Items. Finally, differences in item ratings between groups were explored for the Self-Harm Critical Items¹ (see Table 9.28) and Sleep Problem Indicator (see Table 9.29).

Parents of youth with ADHD were more likely to endorse the Self-Harm Critical Items than were parents of youth in the General Population group (up to 3.6 times more likely), while teacher and self-report ratings did not differentiate between the two groups to the same extent. For the Self-Report, although both groups endorsed these items at a similar rate, alarmingly, this rate was quite high for all groups (up to 18.1% of the sample). Negligible differences were found between presentations of ADHD for Parent, Teacher, and Self-Report.

Ratings of youth in the ADHD group generally showed higher endorsement for sleep-related problems than the General Population group. Endorsement of the “Having trouble sleeping” item was more likely in the ADHD group than in the General Population group (up to 5.4 times more likely for Parent, up to 2.2 times more likely for youth reporting about themselves; note that this item is not included on the Teacher form). All three raters were more likely to indicate that youth in the ADHD group appeared or felt tired compared to youth in the General Population group (up to 6.5 times more likely for Parent, up to 2.9 times more likely for Teacher, and nearly twice as likely for Self-Report). Small differences were observed between the presentations of ADHD (see Table 9.29).

Summary. Taken together, these results provide strong evidence for the validity of the Conners 4 to not only distinguish between youth with ADHD and those from the general population, but also to distinguish between ADHD presentations. It is worth noting that the effect sizes for Conners 4 Parent exceed those for the Conners 4 Teacher and Self-Report (that is, there are larger differences between youth with and without ADHD when examining parent-reported ratings relative to teacher- or self-reported ratings). This trend is consistent with the patterns of results from different rater types presented in previous literature (see Narad et al., 2015).

Depression and General Population Comparisons

Mean score differences between the General Population and Depression groups (i.e., Major Depressive Disorder or Persistent Depressive Disorder) were compared across Conners 4 Content Scale scores, Impairment & Functional Outcome Scale scores, and across DSM Symptom Scale scores. Demographically matched subsamples were selected to facilitate each set of comparisons (matched in terms of age, gender, race/ethnicity, and PEL [matched for Parent and Self-report only]). The youth included in the Depression group did not have co-occurring ADHD (i.e., youth with both ADHD and Depression were excluded from these analyses); however, many youth in the sample did have other co-occurring diagnoses. The demographic characteristics of the matched General Population and Depression groups are presented in appendix F in Table F.17 and the demographic characteristics of the parent and teacher raters are presented in Table F.18.

Scores for the Depression group were expected to be higher than scores from the General Population group on the Depressed Mood scale and on scales where there is symptom overlap between Depression and ADHD (e.g., Inattention/Executive Dysfunction, DSM ADHD Inattentive Symptoms, Emotional Dysregulation). Additionally, given that depression and anxiety tend to be highly co-occurring among clinical groups (and there were, in fact, high levels of co-occurring anxiety within the Depression group; see appendix F, Table F.17), the Depressed group was expected to have higher scores on the Anxious Thoughts scale than the General Population group. Finally, it was expected that the Depression group would have higher scores on the Impairment & Functional Outcome Scales than the General Population group, as evidence of impairment is part of the diagnostic criteria for depressive disorders.

Results from the ANOVAs are found in Tables 9.30 to 9.32. As expected, the largest effects were found on the Depressed Mood scale (Parent Cohen’s d = 1.37, Teacher Cohen’s d = 1.62, Self-Report Cohen’s d = 2.09). Additionally, large effects were observed on Anxious Thoughts (Parent Cohen’s d = 1.01, Teacher Cohen’s d = 1.16, Self-Report Cohen’s d = 1.65) and Emotional Dysregulation (Parent Cohen’s d = 0.95, Teacher Cohen’s d = 0.93, Self-Report Cohen’s d = 1.09), and medium to large effects were found on Inattention/Executive Dysfunction (Parent Cohen’s d = 0.72, Teacher Cohen’s d = 1.04, Self-Report Cohen’s d = 0.91), and the DSM ADHD Inattentive Symptoms scale (Parent Cohen’s d = 0.76, Teacher Cohen’s d = 1.07, Self-Report Cohen’s d = 0.72). Finally, impairment was noted by all three raters, with Peer Interactions having the largest effect sizes across all raters (Parent Cohen’s d = 0.97, Teacher Cohen’s d = 1.20, Self-Report Cohen’s d = 1.24). Significant differences between the Depression and General Population groups were also found for the Schoolwork scale for Teacher and Self-Report, with large effect sizes (Teacher Cohen’s d = 0.96, Self-Report Cohen’s d = 0.80). Finally, significant differences between the Depression and General Population groups were also found for the Family Life scale, with scores from parents and youth in the Depression group indicating significantly more impairment than the General Population group in a family setting (Parent Cohen’s d = 0.76, Self-Report Cohen’s d = 0.80). This pattern of observations reveals that, as expected, there were significant differences in scale scores between the General Population and youth with Depression, with these differences being most salient on the scales pertinent to the symptoms of depression as well as those pertinent to commonly co-occurring diagnoses, such as anxiety.

Differences in item ratings between the Depression and General Population groups were explored for the Self-Harm Critical Items (see Table 9.33) and the Sleep Problems Indicator (see Table 9.34). Ratings of youth in the Depression group showed higher levels of endorsement for the Self-Harm Critical Items than the General Population group (up to 4.5 times more likely for Parent, up to 2.0 times more likely for Teacher, and up 3.3 times more likely for Self-Report). Ratings of youth with Depression showed higher endorsement for the Sleep Problems Indicator than ratings of youth from the General Population (up to 5.0 times more likely for Parent, 4.3 times more likely for Teacher, and up 2.0 times more likely for Self-Report).

Generalized Anxiety Disorder and General Population Comparisons

Mean score differences between the General Population and Generalized Anxiety Disorder (GAD) groups were compared across Conners 4 Content Scale scores, Impairment & Functional Outcome Scale scores, and DSM Symptom Scale scores. While individuals with other forms of anxiety were recruited as part of the broader clinical sample (e.g., Social Anxiety, Separation Anxiety, Panic Disorder), only those diagnosed with GAD were included in this sample as the majority of the items (4 out of 5 for Parent and Teacher; 5 out of 6 for Self-Report) pertain to GAD symptoms (see appendix C for item content). Demographically matched subsamples were selected to facilitate each set of comparisons (matched in terms of age, gender, race/ethnicity, and PEL [PEL was matched for Parent and Self-Report only]). The GAD group excluded youth with co-occurring ADHD but included other co-occurring disorders. The demographic characteristics of the matched General Population and GAD groups are presented in appendix F in Table F.19, and the demographic characteristics of the parent and teacher raters are presented in Table F.20.

Scores from the GAD group were expected to be higher than scores from the General Population group on the Anxious Thoughts scale, as well as on scales where there is symptom overlap between anxiety and ADHD (e.g., Inattention/Executive Dysfunction, DSM ADHD Inattentive Symptoms, Emotional Dysregulation). Additionally, given that depression and anxiety tend to be highly co-occurring among clinical groups (and there were, in fact, high levels of co-occurring depression within the GAD group; see Table F.19), the GAD group was expected to have higher scores on Depressed Mood scale than the General Population group. Finally, it was expected that the GAD group would score higher than the General Population on the Impairment & Functional Outcome Scales as impairment is part of the diagnostic criteria for GAD.

Results from the ANOVAs are found in Tables 9.35 to 9.37. As expected, the largest effect size was found on the Anxious Thoughts scale across all raters (Parent Cohen’s d = 1.42, Teacher Cohen’s d = 1.59, Self-Report Cohen’s d = 1.05). Additionally, small to large effects were found on Depressed Mood (Parent Cohen’s d = 1.15, Teacher Cohen’s d = 0.72, Self-Report Cohen’s d = 0.78), Emotional Dysregulation (Parent Cohen’s d = 1.15, Teacher Cohen’s d = 0.73, Self-Report Cohen’s d = 0.46), Inattention/Executive Dysfunction (Parent Cohen’s d = 0.73, Teacher Cohen’s d = 0.62, Self-Report Cohen’s d = 0.61), and the DSM ADHD Inattentive Symptoms scale (Parent Cohen’s d = 0.67, Teacher Cohen’s d = 0.61, Self-Report Cohen’s d = 0.47). Finally, there was impairment noted by all three raters, with Peer Interactions having the largest effects across all raters with medium to large effect sizes (Parent Cohen’s d = 0.75, Teacher Cohen’s d = 0.98, Self-Report Cohen’s d = 0.58). Significant differences between the GAD and General Population groups were also found for the Family Life scale for parent raters only, with scores from parents of the GAD group indicating significantly more impairment than the General Population group in a family setting (Cohen’s d = 0.87). In terms of impairment with Schoolwork, there were significant effects for Parent (Cohen’s d = 0.42) and for Teacher (Cohen’s d = 0.58). Overall, as expected, the most prominent differences between youth with GAD and youth from the general population were found on the scales that were most relevant to the symptoms of anxiety and extended to scales tapping into commonly co-occurring diagnoses, such as depression, along with ratings of impairment. These results lend evidence to the validity of the Conners 4.

Differences in item ratings between the GAD and General Population groups were explored for the Self-Harm Critical Items (see Table 9.38) and the Sleep Problems Indicator (see Table 9.39). Parents of youth with GAD were more likely to endorse the Self-Harm Critical Items than were parents of youth in the General Population (up to 2.0 times more likely), while teacher and self-reported ratings did not differentiate between the two groups to the same extent. Although the differences between the General Population and GAD groups were not large for the Self-Report, there was a very high rate of endorsement (up to 35.1%) of these items, indicating a high level of self-harming thoughts and behaviors occurring in both groups. Generally, ratings of youth in the GAD group showed higher endorsement for sleep-related problems than the General Population group. Endorsement of the “Having trouble sleeping” item was more likely in the GAD group than in the General Population group (3.3 times more likely for Parent). All three rater types were more likely to indicate that youth in the GAD group were tired compared to youth in the General Population group (3.7 times more likely for Parent, 5.0 times more likely for Teacher, and 3.0 times as likely for Self-Report).

Disruptive Disorders and General Population Comparisons

Mean score differences between the General Population group and the Disruptive, Impulse-Control, or Conduct Disorders group (i.e., youth diagnosed with Oppositional Defiant Disorder [which constituted most of the youth in these samples], Conduct Disorder, and/or Intermittent Explosive Disorder; termed Disruptive Disorders group for this section) were compared across Conners 4 Content Scale scores, Impairment & Functional Outcome Scale scores, and DSM Symptom Scale scores. Demographically matched subsamples were selected to facilitate each set of comparisons (matched in terms of age, gender, race/ethnicity, and PEL [PEL was matched for Parent and Self-Report only]). Due to the high rate of co-occurrence between ADHD in the Disruptive Disorders groups, the Disruptive Disorders groups did not exclude youth with co-occurring ADHD. The demographic characteristics of the matched General Population and Disruptive Disorders groups are presented in appendix F in Table F.21, and the demographic characteristics of the parent and teacher raters are presented in Table F.22.

Given that the majority of the Disruptive Disorders group included youth with co-occurring ADHD, it was expected that scores from the Disruptive Disorders group would be higher than scores from the matched General Population group on most scales. With that said, it was expected that the largest differences would emerge on scales that tap into impulsivity, emotional regulation, and/or behavioral problems, including the Impulsivity and Emotional Dysregulation Content Scales, as well as the DSM Oppositional Defiant Disorder Symptoms and DSM Conduct Disorder Symptoms scales. Finally, it was expected that scores from the Disruptive Disorders group would be higher on the Impairment & Functional Outcome Scales than scores from the General Population group, as youth with Disruptive Disorders often have severe impairments in multiple domains (e.g., Counts et al., 2005).

Results from the ANOVAs are found in Tables 9.40 to 9.42. As expected, large effects were found on the Impulsivity (Parent Cohen’s d = 1.61, Teacher Cohen’s d = 1.10, Self-Report Cohen’s d = 1.27) and Emotional Dysregulation (Parent Cohen’s d = 2.11, Teacher Cohen’s d = 1.61, Self-Report Cohen’s d = 1.04) scales, as well as the DSM Oppositional Defiant Disorder Symptoms scale (Parent Cohen’s d = 1.94, Teacher Cohen’s d = 1.42, Self-Report Cohen’s d = 1.20). Medium to large effects were found on the DSM Conduct Disorder Symptoms scale (Parent Cohen’s d = 0.65, Teacher Cohen’s d = 0.96), while there was no statistically significant difference between Self-Report scores. As expected, large effects were also found for all Impairment & Functional Outcome Scales (Parent median Cohen’s d = 1.61, Teacher median Cohen’s d = 1.10, Self-Report median Cohen’s d = 1.04) indicating a higher level of impairment across the school, peer, and family domains for the Disruptive Disorders group relative to the General Population group.

Differences in item ratings between the Disruptive Disorders and General Population groups were explored for the Self-Harm Critical Items (see Table 9.43) and the Sleep Problems Indicator (see Table 9.44). Ratings of youth in the Disruptive Disorders group generally showed higher levels of endorsement for the Self-Harm Critical Items than the General Population group (up to 4.0 times more likely for Parent, up to 3.0 times more likely for Teacher, and up 2.5 times more likely for Self-Report). Ratings of youth with Disruptive Disorders showed higher endorsement for the Sleep Problems Indicator than ratings of youth from the General Population (up to 4.6 times more likely for Parent, 15.2 times more likely for Teacher, and 3.0 times more likely for Self-Report).

ADHD and Depression/Anxiety Comparisons

To consider how well the Conners 4 is able to distinguish youth with ADHD from youth with other clinical disorders, the Conners 4 Content Scale scores, Impairment & Functional Outcome Scale scores, and DSM Symptom Scale scores were compared between youth with ADHD but no co-occurring Depression (i.e., Major Depressive Disorder or Persistent Depressive Disorder) or Anxiety (i.e., Generalized Anxiety Disorder, Panic Disorder, Social Anxiety, or Specific Phobias) and youth with either Depression or Anxiety but no co-occurring ADHD. To facilitate comparisons between the ADHD group and the Depression/Anxiety group, the groups were not matched based on demographic characteristics, as finding an exact match would have resulted in too few cases to make meaningful comparisons between groups. The demographic characteristics of the ADHD and Depression/Anxiety groups are presented in appendix F in Table F.23 and the demographic characteristics of the parent and teacher raters are presented in Table F.24.

It was expected that the ADHD group and the Depression/Anxiety groups would display distinct symptom profiles. Given that Inattention/Executive Dysfunction is a core feature of ADHD, it was expected that youth with ADHD would exhibit higher ratings than youth with Depression/Anxiety, although elevated scores should be observed for both groups. The ADHD group was expected to score higher than the Depression/Anxiety group on the Conners 4 Content Scales that are highly specific to the symptoms of ADHD (e.g., Hyperactivity, Impulsivity) whereas the Depression/Anxiety group was expected to be similar to the ADHD group on scales where there is symptom overlap with ADHD (e.g., Emotional Dysregulation) and to exceed the ADHD group on scales closely related to the symptoms of depression or anxiety (e.g., Depressed Mood or Anxious Thoughts). In terms of the Impairment & Functional Outcome Scales, it was expected that the ADHD group and Depression/Anxiety group would exhibit similar scores, given that ADHD, Depression, and Anxiety are each associated with functional impairment. With respect to the DSM Symptom Scales, it was expected that the ADHD group would score higher than the Depression/Anxiety group on the ADHD Inattentive Symptoms, ADHD Hyperactive/Impulsive Symptoms, and Total ADHD Symptoms scales, given that these scales capture core symptoms of ADHD. Finally, it was expected that the ADHD group would score higher on the DSM Oppositional Defiant Disorder Symptoms and DSM Conduct Disorder Symptoms scales, as youth with ADHD are more likely to exhibit co-occurring externalizing symptoms than youth with Depression/Anxiety, and for the Conners 4, these particular ADHD samples included youth with co-occurring disruptive disorders (9% to 12%).

Results from the ANOVAs are found in Tables 9.45 to 9.47 and are depicted graphically in Figures 9.4 to 9.6. As expected, there were medium to large effects across forms on the Inattention/Executive Dysfunction, Hyperactivity, and Impulsivity scales (Parent Cohen’s d = 0.88 to 1.10, Teacher Cohen’s d = 0.49 to 0.73, Self-Report Cohen’s d = 0.57 for all three scales), with the ADHD group scoring higher than the Depression/Anxiety group. Also, as expected, the ADHD group and the Depression/Anxiety group scored similarly on Emotional Dysregulation with no significant differences between groups across forms. There were medium to large effects across rater types for the Depressed Mood and Anxious Thoughts scales (Parent Cohen’s d = -0.60 to -0.94, Teacher Cohen’s d = -0.65 to -0.79, Self-Report Cohen’s d = -0.63 to -0.83); as expected, the Depression/Anxiety group scored higher than the ADHD group on these scales.

In terms of the Impairment & Functional Outcome Scales, small to large differences were observed on the Schoolwork scale for all raters (Parent Cohen’s d = 0.80, Teacher Cohen’s d = 0.37, Self-Report Cohen’s d = 0.59), with the ADHD group reporting more impairment at school than the Depression/Anxiety group. For the Peer Interactions scale, a small difference emerged on Self-Report (Cohen’s d = -0.36) with the Depression/Anxiety group scoring higher than the ADHD group. For the Family scale, a small difference emerged on Parent (Cohen’s d = 0.36) with the ADHD group scoring higher than the Depression/Anxiety group.

For the DSM Symptom scales, as expected, there were medium to large effects for the ADHD Inattentive Symptoms, Hyperactive/Impulsive Symptoms, and Total ADHD Symptoms scales, with medium to large effect sizes (Parent Cohen’s d = 0.96 to 1.06, Teacher Cohen’s d = 0.49 to 0.68, Self-Report Cohen’s d = 0.57 to 0.64) and the ADHD group scoring higher than the Depression/Anxiety group. Fewer differences were found between the groups on the DSM Oppositional Defiant Disorder Symptoms and DSM Conduct Disorder Symptoms scales.

Differences in item ratings between the ADHD and Depression/Anxiety groups were explored for the Self-Harm Critical Items (see Table 9.48) and the Sleep Problems Indicator (see Table 9.49). Parents, teachers, and youth from the Depression/Anxiety group were more likely to endorse the Self-Harm Critical Items than those in the ADHD group. For the Sleep Problems Indicator, parents and youth from both the ADHD and Depression/Anxiety groups were about equally as likely to endorse the “Having trouble sleeping” item; however, for all three rater groups, there was a higher level of endorsement for the “Appearing/feeling tired” item for the Depression/Anxiety group compared to the ADHD group.

Classification Accuracy

Classification accuracy is an index of validity akin to predictive validity evidence in that it considers the portion of respondents that are correctly classified into their respective groups (e.g., General Population vs. ADHD) based on their performance on a criterion measure (Jenkins et al., 2007). To evaluate the classification accuracy of the Conners 4, this section used binary classification modelling to predict the accuracy with which the Conners 4 scales² correctly classify youth from the General Population sample and youth from the clinical diagnostic groups. These diagnostic groups included youth diagnosed with ADHD, a depressive disorder (i.e., the Depression group), Generalized Anxiety Disorder, and Oppositional Defiant Disorder.

For the Conners 4 Parent, Teacher, and Self-Report, the binary classification modelling consisted of two sets of analyses for each form: (a) binomial logistic regression, followed by (b) the creation of confusion matrices and the derivation of classification accuracy statistics. Logistic regression is a statistical approach used to predict a dichotomous dependent variable from one or more predictor variables. In the current application, general population versus clinical diagnostic group membership (ADHD, Depression, Generalized Anxiety Disorder, or Oppositional Defiant Disorder) represented the dichotomous dependent variable of interest, and the Conners 4 scale T-scores represented the predictor variables of interest. The second set of analyses involved using the classification(s) predicted by the logistic regression models and actual group memberships to construct confusion matrices. Using the approach outlined by Kessel and Zimmerman (1993), the confusion matrices were then used to derive classification accuracy statistics (see Response Style Analysis: Item Selection & Score Creation in chapter 6, Development, for a detailed explanation of these statistics).

A common approach to deriving classification accuracy statistics is to utilize matched samples with an equal number of individuals in each class (general population vs. clinical diagnostic group) to control for the influence of confounding variables. The approach outlined below employed samples matched on key demographic characteristics. Utilizing matched samples cultivates a base rate of 50%, or an equal likelihood that the individual being rated could belong to one class or the other (see Janes & Pepe, 2008, for a review). However, the real-world prevalence of a given clinical disorder can be higher or lower than 50% and thus the subsequent positive predictive value (PPV; i.e., the probability that an individual with a positive assessment result actually has a clinical disorder) and negative predictive value (NPV; i.e., the probability that an individual with a negative assessment result actually does not have a clinical disorder) will vary as a function of real world prevalence (see Robinson et al., 2016 and Lavigne et al., 2016 for reviews). The prevalence (or base rate of a given disorder in the clinician’s referral population) can vary widely depending on the purpose of the evaluation and the setting, and predictive ability will vary based on prevalence. For example, in a screening setting you might expect the prevalence of ADHD to be around 10% or less; whereas, in a clinically referred sample, a prevalence of approximately 50% may be more likely, rising to 60%–80% in an ADHD-specific clinical practice. Accordingly, the classification accuracy statistics of the Conners 4 scales (assuming a 50% base rate) are presented in the following sections, and the PPV and NPV based on varying base rates are provided in subsequent tables for reference.

ADHD vs. General Population Classifications

To consider how well the Conners 4 distinguishes between youth from the general population and those diagnosed with ADHD, three pairs of demographically matched samples were created. General Population samples consisting of youth who had not received any clinical diagnoses were matched to each of the following groups: (a) youth diagnosed with ADHD Primarily Inattentive or ADHD Combined, (b) youth diagnosed with ADHD Primarily Hyperactive/Impulsive or ADHD Combined, and (c) youth diagnosed with any of the three ADHD presentations (Inattentive, Hyperactive/Impulsive, or Combined). Youth were matched on the following key demographic variables: age, gender, race/ethnicity, and PEL (PEL was matched for Parent and Self-Report only). Demographic characteristics of the matched General Population and ADHD samples and their raters are presented in appendix F in Tables F.25 to F.29, respectively.

The matched samples were submitted to a series of logistic regressions to explore how well the scores predicted group membership. For all analyses, the Impairment & Functional Outcome Scales were included in the analyses as these scales were designed specifically to measure the impairment in school,social, and home life as outlined in the DSM³ for a thorough ADHD diagnosis. Specifically, logistic regressions were used to predict the following:

• How well the specific Conners 4 Content Scales (i.e., those that assess ADHD symptoms, including Inattention/Executive Dysfunction, Hyperactivity, Impulsivity, and Emotional Dysregulation) together with the Impairment & Functional Outcome Scales distinguished between youth from the General Population and those diagnosed with ADHD (all presentations).
• How well the DSM Total ADHD Symptoms scale together with the Impairment & Functional Outcome Scales distinguished between youth from the General Population and those diagnosed with ADHD (all presentations).
• How well the DSM ADHD Inattentive Symptoms scale together with the Impairment & Functional Outcome Scales distinguished between youth from the General Population and those diagnosed with ADHD Predominantly Inattentive or Combined Presentations.
• How well the DSM Hyperactive/Impulsive Symptoms scale together with the Impairment & Functional Outcome Scales distinguished between youth from the General Population and those diagnosed with ADHD Predominantly Hyperactive/Impulsive or Combined Presentations.

The classification accuracy statistics are summarized in Tables 9.50 to 9.55. Moderate to high levels of classification accuracy were demonstrated for both the combination of the Conners 4 Content Scales and Impairment & Functional Outcome Scales (overall correct classification rate = 84.9% for Parent, 74.3% for Teacher, 75.4% for Self-Report) and the combination of the Conners 4 DSM ADHD Symptoms Scales and Impairment & Functional Outcome Scales (overall correct classification rate = 83.9% to 88.1% for Parent, 69.3% to 71.8% for Teacher, 68.0% to 74.6% for Self-Report). The high degree of correct classification across raters provides evidence of validity for the Conners 4 by demonstrating the accuracy with which groups can be identified using relevant collections of scales, and therefore supports the accuracy of insights gleaned from using the Conners 4 in this manner.

Depression, Generalized Anxiety Disorder, or Oppositional Defiant Disorder vs. General Population Classifications

To consider how well specific predictor scales distinguish between youth from the general population and youth with depression, Generalized Anxiety Disorder (GAD), or Oppositional Defiant Disorder (ODD), demographically matched samples were created. The General Population sample consisted of youth who have not received a clinical diagnosis. The Depression sample consisted of youth diagnosed with either Persistent Depressive Disorder or Major Depressive Disorder, the GAD sample consisted of youth diagnosed with GAD, and the ODD sample consisted of youth diagnosed with ODD. Youth were matched on the following key demographic variables: age, gender, race/ethnicity, and PEL (PEL was matched for Parent and Self-Report only). Youth with co-occurring ADHD were included in the Depression, GAD, and ODD samples. This approach differs from the approach that was taken in the Clinical Group Differences section of this chapter where youth with co-occurring ADHD were omitted from the Depression and GAD versus General Population comparisons. In the Clinical Group Differences section, the objective was to consider the full profile of Conners 4 scale scores, which would be influenced by the presence of co-occurring ADHD. For Classification Accuracy, the objective was to consider whether the targeted Conners 4 scales could correctly classify individuals with and without depression, GAD, or ODD, a state which should not be affected by the presence or absence of co-occurring ADHD. The demographic characteristics of the matched General Population and clinical samples and their raters are presented in appendix F in Tables F.30 to F.35. The targeted predictor scales were: Depressed Mood for the Depression sample, Anxious Thoughts for the GAD sample, and the DSM Oppositional Defiant Disorder Symptoms Scale for the ODD sample.

The matched samples were submitted to a logistic regression to explore how well the predictor scales distinguished between youth from the General Population and youth from the clinical groups. The classification accuracy statistics are summarized in Tables 9.56 and 9.57 and demonstrate moderate to strong classification results (overall correct classification for Depressed Mood: Parent 83.6%, Teacher 79.2%, Self-Report 73.8%; overall correct classification for Anxious Thoughts: Parent 76.4%, Teacher 74.5%, Self-Report 67.9%; overall correct classification for DSM Oppositional Defiant Disorder Symptoms: Parent = 82.1%, Teacher = 79.3%, Self-Report = 71.2%). These promising results of the classification of youth with various clinical disorders, relative to undiagnosed youth, lends evidence to the validity of the Conners 4 as an effective tool to distinguish meaningfully different clinical groups.

< Back

Next >