Psychometric Properties of the Metacognitions Questionnaire-30 (MCQ-30) in Older Norwegian Adolescents

In the Self-regulatory executive function (S-REF) model [46,44], the mechanism proposed to underly psychopathology is a particular negative and rigid thinking style named the Cognitive Attentional Syndrome (CAS; [43]). The CAS includes top-down directed self-regulatory strategiesin the form of perseverative thinking patterns (e.g., worry/rumination), strategic attention towards threats (i.e., “threat monitoring”) and maladaptive coping behaviors (e.g., thought control, avoidance). The CAS is considered to result from dysfunctional beliefs about cognition (i.e., metacognitive beliefs), for example “worrying is uncontrollable” or “I need to control my thoughts” [43]. The clinical implication that naturally follows from the model is that psychological disorders are maintained by a common set of metacognitive beliefs and strategies, and that treatment that effectively modify these factors will be successful for a range of emotional disorders and comorbid symptoms [43].

To empirically test the role of dysfunctional metacognitions to emotional distress and disorders, valid and reliable assessment tools are needed. Since the publication of the S-REF model [46] several have been developed (see Capobianco and Nordahl [8] for an overview), but the most used and empirically supported is the metacognitions questionnaire (MCQ: [10]) and its 30-items shortened version (MCQ-30; [45]). The MCQ-30 assesses five subscales of metacognitive beliefs: (1) positive beliefs about worry; (2) negative beliefs about uncontrollability of thoughts and danger of worry; (3) cognitive confidence; (4) beliefs about the need to control thoughts; and (5) cognitive self-consciousness. The MCQ-30 has good psychometric properties in non-clinical (e.g., [19, 34]) and clinical (e.g., [13, 16]) adult populations. It has been important in establishing the role of dysfunctional metacognitive beliefs across a range of symptom domains and emotional disorders [7, 8, 15, 21, 38].

Future directions for research on the role of metacognitions in psychopathology include evaluating their role in the mental health of children and adolescents [8]. The prevalence of emotional disorders increases in adolescence [14], also at the symptoms level [26, 36]. Thus, there is a need to have psychometric sound and measurement invariant assessment tools of metacognitions that can be used in adolescents for research and clinical purposes. The application of the original MCQ-30 developed for adults in adolescence enables direct comparison of dysfunctional metacognitive beliefs with adult populations. Further, to be positioned to properly examine the role of metacognitions in the development of emotional disorder through adolescence, a strong psychometric measure of metacognitions is needed.

Myers et al. [27] conducted a systematic review of the MCQ and its derivates in children and adolescents and concluded that further psychometric research is needed to advance research on the role of metacognitions in youth mental health. More recently, Thingbak and colleagues [39] reviewed and synthesized the empirical research on the associations between metacognitive beliefs and anxiety and depression in youth. They identified 32 papers with a total of 9887 participants aged 7–18 years and concluded that metacognitive beliefs, positively correlate with anxiety and depression in youths and that clinical groups endorsed dysfunctional metacognitive beliefs more strongly compared to non-clinical groups (all except the domain for positive beliefs about worry). However, they identified a problem with moderate to high between-study heterogeneity in these results which they partly attributed to the use of different adaptations and versions of the MCQ which has not been appropriately anchored or psychometrically evaluated. Hence, the need for psychometric evaluation of measures assessing metacognitions is still warranted to secure that the same concepts are being investigated and to reduce error of measurement.

Declarative metacognitive knowledge is considered well-developed in older adolescents, [33]. Applying the original adult-version of MCQ-30 in this age group is therefore most likely suitable. Further, using the original MCQ-30 without modifications has the advantage of allowing us to compare results across late adolescence and adulthood, and help overcome some of the issues associated with MCQ derivates [27]. To the best of our knowledge, only one study has evaluated the psychometric properties of the MCQ-30 in adolescents. Li et al. [23] included 2827 adolescents (aged 11 to 18 years) in China and found support for the proposed five-factor structure of the MCQ-30. The factors were found to be internally consistent, positively correlated with symptoms of anxiety and depression, and measurement invariant on three levels (configural, metric and scalar) across several demographic variables and high/low levels of anxiety/depression. Because there are differences between Eastern and Western cultures (e.g., collectivistic vs. individualistic), which may also affect adolescent development and functioning, it is important to test whether the MCQ-30 instrument is valid across such differing societies.

With an aim to facilitate research on the role of dysfunctional metacognitions in adolescents, we set out to evaluate the psychometric properties of the MCQ-30 in older Norwegian adolescents aged 16 to 18 years. We hypothesize that the original five-factor solution will provide a good fit to the data and that measurement invariance (across sex and anxiety levels) will be displayed. We further expect the MCQ-30 subscales to show adequate internal consistency and to find support for convergent validity.

Statistical analyses were conducted using JASP 0.18.3. To evaluate the proposed five-factor structure of the MCQ-30, a confirmatory factor analysis (CFA) was performed. We used robust estimation and the diagonal weighted least squares (DWLS) estimator recommended for ordinal data (Li [22]), combined with listwise deletion for missing data. In JASP, listwise deletion is compatible with selecting a DWLS estimator, and considered appropriate as the missingness at the item level in our data was below 0.4%.

According to recommendations for assessing model fit [18], we included the following fit indices: the Comparative Fit Index (CFI; [3]), the Tucker-Lewis Index (TLI; [40]), the Root Mean Square Error of Approximation (RMSEA; [35]) and the Standardized Root Mean Square Residual (SRMR; [4, 20]). Recommended cut-off values for CFI and TLI are 0.90 for acceptable model fit and 0.95, indicating very good model fit. Furthermore, recommended cut-off values for RMSEA are below or close to 0.06, and SRMR should be less than 0.08 [18].

Secondly, measurement invariance was tested on four different levels for both groups (configural, metric, scalar and strict). Measurement invariance can indicate whether the same construct is being measured across some specified groups or over time which is particularly important in developmental research [24]. The MCQ-30 structure was tested for measurement invariance across sex (male and female) and severity of anxiety symptoms (two groups consisting of those below versus at or above the cut-off score of 25 on SCARED). Configural invariance measure is an indication of whether the overall factor structure is the same between groups; metric invariance is a test of the item factor loadings between groups; and the scalar invariance indicates the similarity of item thresholds between investigated groups. Strict invariance indicates that loadings, measurement intercepts and measure residual variances are equal across groups [25]). When testing for measurement invariance, examining the change in Comparative Fit Index (CFI; [3]) and the Root mean square error of approximation (RMSEA; [35]) between models (ΔCFI, ΔRMSEA) are recommended as suitable measures (Chen [11]). Criteria for assessing invariance can vary depending on sample size or unequal sample size between groups, but a ΔCFI value less than 0.01 and a ΔRMSEA less than 0.015 are commonly applied criteria of non-invariance [23, 32].

Thirdly, means with standard deviations on the MCQ-30 total score and the five subscales, and their internal consistencies (Cronbach`s alpha and McDonald’s Omega Coefficient) were calculated.

Finally, convergent validity was tested by examining bivariate correlations between MCQ-30 subscales, anxiety (SCARED) and depression (SMFQ). Positive relationships between the five factors and symptoms of anxiety and depression were used as indicators of convergent validity.

Results from measurement invariance testing of the five-factor model across two groups of sex (male and female) and severity of anxiety (below versus above SCARED cut-off) are presented in Table 1.

The testing of configural, metric, scalar and strict measurement invariance showed that the factor structure, the factor loadings and the indicator thresholds of the MCQ-30 were the same, across sex and anxiety level.

To test convergent validity, each of the five MCQ-30 subscales were correlated with anxiety (SCARED) and depression (SMFQ) (presented in Table 2). All five subscales of the MCQ-30 were significantly and positively correlated with anxiety and depression, showing moderate to strong associations. Negative beliefs about uncontrollability of thoughts and danger of worry had the strongest relationship to anxiety and depression symptoms among the metacognitive belief factors.

The aim of the present study was to evaluate the psychometric properties of the MCQ-30 in a sample of Norwegian adolescents. As hypothesized, the fit indices indicated that the five-factor model had an excellent fit, with the exception of a significant chi-square. A significant chi-square indicates a poor fitting model, but this statistic is very sensitive to sample sizes and models with larger numbers of observed variables [17]. Four levels of measurement invariance were tested and in sum the MCQ-30 five-factor structure were found to have measurement equivalence across sex (male/female) and across two levels of anxiety. The factors demonstrated acceptable to good reliability, and significant and positive relationships with anxiety and depressive symptoms indicating convergent validity.

The current results are in line with a study of Chinese adolescents [23], as well as previous findings from Norwegian adults [16, 29]. Finding good psychometric properties of the MCQ-30 in a Scandinavian/Western adolescent sample indicates that the measure is relatively culturally independent, although future studies from other parts of the world are needed to substantiate such claim.

The original MCQ-30 with its proposed five-factor structure holds in a group of 16- to 18-year-olds, which suggest that the measure can be used in studies of dysfunctional metacognitions in this age-group and is applicable in clinical practice. The findings justify the use of MCQ-30 in observational and clinical studies. This is important because longitudinal studies in normally developing adolescents are needed to gain insight into the development of metacognitive beliefs across adolescence, and we need intervention studies examining the effect of metacognitive therapy and other methods that may affect metacognitions. This type of research relies on access to validated measurement tools, such as the MCQ-30.

In the present sample, one of the items (MCQ12; “I monitor my thoughts”) showed a low loading to its suggested factor (CSC). The same finding has been reported in two previous studies evaluating the MCQ-30 using network analysis in Norwegian adults [2, 29]) The aim of the current study was to test if the established factor structure was confirmed in the present sample, which it did. If this was not the case, it would have been appropriate to go with exploratory approaches and potentially modifying the measure. However, a modification would not allow for comparing findings across studies, which would represent a serious limitation. While metacognitive beliefs assessed with the MCQ-30 theoretically can be distinguished in subcategories, it is expected that factors and its items will be interconnected. Using a network approach to evaluate the structure of the MCQ-30 and not relying on a common factor model, [29] found that the MCQ12 item shared positive edges with both the CSC and the NC cluster and argue “I monitor my thoughts” are conceptually linked to more general beliefs about the importance and power of thoughts as assessed with NC. One option for further research on the MCQ-30 in adolescents is to explore its structure and centrality of specific metacognitions taking interdependence and mutual influence between the items and factors into account.

Measurement invariance (MI) tests assess the psychometric equivalence of a construct across groups and support for measurement invariance is a prerequisite to evaluate for example group differences or to assess the relations between theoretical concepts over time (e.g., [31]. We tested four levels of MI (i.e., configural, metric, scalar, strict) and found support for measurement invariance across sex (i.e., male and female) and anxiety level (below versus above clinical cut-off on the SCARED). This finding indicates that the MCQ-30 assess the same constructs in females and males, and among those below and above a cut-off for clinical anxiety which means that we can apply the measure to assess gender differences and compare individuals with different levels of anxiety (i.e., the five MCQ subscales does not depend on anxiety level). This result is in line with the study by [23] who also reported that the adult version of the MCQ-30 demonstrated measurement invariance (configural, metric, scalar) in adolescents. Adding to the [23] study, we also tested and found support for strict invariance, meaning that the sum of specific variance and error variance is similar across the tested groups.

Consistent with findings from a Chinese study [23], the internal consistency for the MCQ-30 total scale and the five subscales ranged from acceptable to very good in this sample. The previous review of the psychometric properties of MCQ derivates in youth reported mixed results for internal reliability, especially for the NC subscale [27]. While the inconsistent findings could result from small samples sizes, as pointed out by Li et al. [23], another explanation is that the MCQ derivates are not well enough adjusted. Our results indicate that applying the original MCQ-30 in older adolescents is one way to overcome this issue and that it provides a reliable measurement of metacognitive factors.

Our results support convergent validity of the MCQ-30 factors as they all showed positive and significant correlations with anxiety and depression. In line with metacognitive theory (Wells and Matthews [46]) and the recent review of metacognitions and their associations with anxiety and depression in youth [39], the factor negative beliefs about uncontrollability of thoughts and danger of worry (NEG) showed the strongest correlation with symptoms. This observation aligns with the suggestion that negative metacognitive beliefs are the strongest correlate of emotional distress symptoms due to its close relationship with the CAS. In turn the CAS is considered the more proximal influence on emotional distress symptoms compared to metacognitive beliefs [43].

There are several limitations associated with our study that should be acknowledged. We relied on a convenience sample, which limits the possibility to generalize to other samples. Support of measurement invariance is restricted to two grouping variables, and analysis using depression as a grouping variable was not possible due to a low number of individuals reporting elevated depression. The findings are limited to older adolescents, and further investigation on younger adolescents will be useful in exploring validity of the measure for an even wider application. Furthermore, there is a need to evaluate the psychometrics of the MCQ-30 in clinical samples. Moreover, we used only one assessment point, and there is a need to assess the test-retest reliability of the factors, to evaluate whether they are responsive to therapeutic interventions, and to test measurement invariance over time.

In conclusion, the original MCQ-30 can be applied to Norwegian adolescents aged 16–18 years old to assess dysfunctional metacognitive beliefs. This finding enables further research on the role for dysfunctional metacognitions which in the long run has the potential to advance formulation and interventions to address psychological vulnerability and disorders in adolescents.