Psychometric Properties of the Spanish Version of the Cognitive Flexibility Inventory

Introduction

Cognitive flexibility (CF) has been defined by many authors as the ability that enables people to effectively adjust their cognitive process to changing environmental demands (Deák, 2003; Dennis & Vander Wal, 2010). Thus, high cognitive flexibility is essential to successfully challenge and restructure maladaptive beliefs into a more adaptive thinking style (Dennis & Vander Wal, 2010). In recent years, this ability has been considered an essential mechanism associated with mental health (Bonanno & Burton, 2013; Kashdan & Rottenberg, 2010; Wersebe et al., 2018). However, one of the major problems of this construct has been the lack of consensus about its definition and measurement. For example, the same term has been used to refer to different constructs due to the different psychological traditions that consider cognitive flexibility a focus of study (Ionescu, 2012, 2017).

A first example of that is the definition of CF as a cognitive mechanism of executive functioning (Geurts et al., 2009; Miyake et al., 2000). According to this view, CF has been generally measured using neurocognitive tests, such as the Wisconsin Card Sorting Test, the Stroop Colour Test, or the Trail Making Test, which assess the flexible/perseverative response pattern of an individual when conducting tasks that require shifting mental sets in response to stimuli. In addition to these, experimental psychologists have also developed task-switching variants of these neuropsychological tests that involve switching between different mental operations in response to task cues (Allport et al., 1994; Rogers & Monsell, 1995; Waugh et al., 2011).

Another conceptualization of CF refers to the rigidity or the ability to adapt our thoughts and behaviours to better respond to the context (Deák, 2003; Garcia-Garcia et al., 2010). In fact, CF is a core mechanism of change in therapies such as the Cognitive Behavioural Therapy (Beck et al., 1979), as well as in more modern, transdiagnostic treatments, such as the Unified Protocol (Boisseau et al., 2010), which propose that CF would be a transdiagnostic process underlying psychopathology. When assessing this conceptualization of CF, authors usually rely on self-report measures, such as the Cognitive Flexibility Scale (CFS; M. M. Martin & Rubin, 1995) and the Cognitive Flexibility Inventory (Dennis & Vander Wal, 2010), which have become the most popular. Compared to the majority of neuropsychological tests, these self-report measures are significantly faster to administer and score and are less sensitive to practice effects (McCaffrey et al., 1992). This view of CF as an ability to flexibly change one’s thoughts and obtain different interpretations of a situation when needed is also more consistent with the perspective of modern psychological treatments (Boisseau et al., 2010).

The CFS is one of the first self-report measures of CF. The CFS evaluates CF linked to interpersonal communication competence, so the authors proposed that cognitive flexibility should be considered an essential component of this interpersonal ability (M. M. Martin & Anderson, 1998; M. M. Martin & Rubin, 1995). This, however, has been argued to be a limitation to the study of CF since it is not clear whether communication flexibility can be compared to the CF required to change and replace maladaptive cognitions effectively (Dennis & Vander Wal, 2010). As a consequence, Dennis and Vander Wal (2010) developed the Cognitive Flexibility Inventory, an alternative measure of CF that can be administered for research and clinical purposes to evaluate an individual’s ability to think adaptively and flexibly when facing stressful life events.

The Cognitive Flexibility Inventory (Dennis & Vander Wal, 2010) is a brief self-report measure composed of two subscales: Alternatives and Control. The Alternatives subscale measures the tendency to perceive multiple alternative explanations of life events and generate multiple solutions to difficult situations (e.g., “I consider multiple options before making a decision”), whereas the Control subscale was developed to test the tendency to perceive difficult situations as controllable (e.g., “I feel I have no power to change things in difficult situations”). The original English scale (Dennis & Vander Wal, 2010) supported this two-factor structure and had good evidence of internal consistency (> .84) and test-retest reliability (r = .81). The scale has now been adapted into Chinese (Wang et al., 2016), Japanese (Oshiro et al., 2016), Iranian (Shareh et al., 2014), Russian (Kurginyan & Osavolyuk, 2018), and Italian (Portoghese et al., 2020). However, some inconsistences have been found in the latent structure in the cross-cultural adaptations. For example, a three-factor structure (i.e., Control, Alternatives, and Alternatives for human behaviours) was reported in the Iranian version (Shareh et al., 2014). In addition, while the Russian (Kurginyan & Osavolyuk, 2018) and the Italian (Portoghese et al., 2020) adaptations replicated the original two-factor structure of the scale, they differed in the number of items that composed the two-factor structure. Specifically, item 1 (“I am good at ‘sizing up’ situations”) was problematic in both adaptations. This item loaded into the Control subscale in the Russian version (Kurginyan & Osavolyuk, 2018), while it had high cross-loading into both factors in the Italian version, which led the authors to remove this item from their adaptation of the inventory (Portoghese et al., 2020).

Regarding the construct validity of the Cognitive Flexibility Inventory according to external criteria, the different versions of the inventory have shown that CF is positively associated with the use of more adaptive coping efforts and less maladaptive strategies (Kurginyan & Osavolyuk, 2018). In addition, research with the Cognitive Flexibility Inventory has indicated that depressed individuals perform poorly in CF (Dennis & Vander Wal, 2010; Shareh et al., 2014), which supports the idea that CF might be an underlying mechanism in emotional disorders such as depression (Boisseau et al., 2010). Also, Demirtas (2020) showed that cognitive flexibility positively correlated with self-efficacy and mental well-being.

Research has also tested the construct validity of the Cognitive Flexibility Inventory in relation to other measures of CF, such as the Attributional Styles Questionnaire (ASQ; Peterson et al., 1982). Surprisingly, though, these studies showed a negative association between the Cognitive Flexibility Inventory and the ASQ, thus suggesting that greater CF on the Cognitive Flexibility Inventory would be related to less cognitive flexibility on the ASQ (Dennis & Vander Wal, 2010; Kurginyan & Osavolyuk, 2018). In this regard. some studies have used the ASQ as a measure of CF using the intra-individual standard deviation for the stable and global items associated with the six negative events (Fresco et al. 2017). However, previous validations considered extreme scores as indicative of less CF for the operationalization of CF (Dennis & Vander Wal, 2010; Kurginyan & Osavolyuk, 2018), which might not be adequate as a measure of CF and might explain the surprising findings reported in past research.

Although evidence of construct validity and reliability of the Cognitive Flexibility Inventory has been studied in several languages, an adaptation for the Spanish population is still missing. The present study will adapt the Cognitive Flexibility Inventory into Spanish and examine its psychometric properties in an Spanish sample to provide a self-report measure to evaluate the flexibility required to successfully modify and restructure maladaptive beliefs with a more adaptive thinking. Because there is no evidence about the structure of the Cognitive Flexibility Inventory in the Spanish population and considering the aforementioned inconsistencies in the internal structure of this measure in prior cross-cultural validations, an exploratory structural equation modelling (ESEM) will be used to investigate the internal structure of the scale. In addition, sources of construct validity of the scale, together with estimates of its internal consistency and stability over time (test-retest), will be examined. Empirical evidence supporting the two-factor solution obtained in the original version of the scale Dennis & Vander Wal, 2010 is expected to be found. Also, significant positive associations (i.e., convergent construct validity) between the Cognitive Flexibility Inventory and the use of adaptive coping strategies and favorable attributional styles (a construct closely related to flexibility in cognitive thinking) are expected. Lastly, a negative relationship between CF and depressive symptomatology, in line with previous research is expected.

Method

Participants

Participants were 300 undergraduate students from Spain (mean age = 22.66, SD = 4.92, 76% females). The sample size was determined according to the graded scale of sample sizes for scale development proposed by Guadagnoli & Velicer (1988), which recommends a minimum of 300 participants for obtaining satisfactory results in scale validation. The study was advertised using printed posters at the author’s university. Once the participants contacted the researchers by email, they were provided with the web link to a Qualtrics survey with all the questionnaires. Participants completed all the measures on the same administration.

The Cognitive Flexibility Inventory was administered again one month after the initial administration to assess test-retest reliability. This second administration was completed by a subsample of 85 participants (mean age = 23.81, SD = 4.81, 80% females, n = 68), who had visited the laboratory to participate in another study where they completed the CFI first, followed by other measures. As a token of appreciation for their participation in the study, all participants received a compensation of 10 euros. In addition, participants who completed the second assessment received an additional 10 euros.

The present study was approved by the ethical committee of the Jaume I University (CD/37/2021) and it was performed in accordance with the Declaration of Helsinki (1964). All the participants gave their written informed consent to participate in the study.

Instruments

Spanish Version of the Cognitive Flexibility Inventory (Dennis & Vander Wal, 2010; Appendix A and B)

This is a 20-item questionnaire composed of two subscales: Alternatives (13 items) and Control (7 items). The Alternatives subscale evaluates the ability to find multiple explanations for a problem and to produce different solutions in front of difficult situations. The Control subscale measures the tendency to perceive situations as controllable. Items are scored using a 7-point Likert scale, which ranges from 1 (strongly disagree) to 7 (strongly agree). Higher scores in both scales reflect greater CF. Internal consistency for the Alternatives and Control subscales in previous research were .86 and .91, respectively (Dennis & Vandel Wal, 2010). The translation into the Spanish language was conducted following the back translation method (Brislin, 1973). First, two of the authors of the present study, who are psychologists and native Spanish speakers and proficient in English, translated the English version into Spanish. These authors were familiar with the target language and culture and intent of the scale. Second, a native-English-speaking professional translator who was not familiar with the questionnaire back-translated the questionnaire from Spanish to English. Then, the original and back-translated versions were compared, with a focus on on appropriateness, clarity, and meaning of each item. Any discrepancies identified between the original and the back-translated items were discussed and resolved between all the study authors. For example, in item 1, the term “sizing up” was translated as “evaluando” which equates to “evaluating” in English. Likewise, in item 9, the word “troublesome” was translated as “problemático”, which means “problematic” in English.

The Attributional Styles Questionnaire (ASQ; Peterson et al., 1982; Sanjuán & Magallares, 2008)

This questionnaire consists of twelve hypothetical life events (6 positive and 6 negative). Individuals are asked to describe a plausible cause if these events happened to them. These causes are rated on a 7-point scale following these anchors: (1) internal or external, (2) stable or unstable, and (3) global or specific. Two composite indexes can be obtained based on the scores of the positive and negative situations, one for positive attributional style and one for negative attributional style, respectively. Reliability coefficients (Cronbach’s alpha) in the present study for positive and negative events were .81 and .74, respectively.

The Cognitive Emotion Regulation Questionnaire (CERQ; Dominguez-Sánchez et al., 2013; Garnefski et al., 2001)

It is a 36-item scale that evaluates the cognitive aspects of emotion regulation. Specifically, the questionnaire comprises 9 subscales, each consisting of different ways in which an individual can think after experiencing threatening or stressful events: Acceptance, Positive reappraisal, Positive refocusing, Putting into perspective, Catastrophizing, Self-blame, Other-blame, Rumination, and Refocusing on planning. Each scale consists of four items measured on a 5-point scale, ranging from 1 (almost never) to 5 (almost always). Higher scores indicate more frequent use of that strategy. Reliability coefficients (Cronbach’s alpha) of the nine CERQ subscales in this study ranged from .62 to .88.

The Beck Depression Inventory-II (BDI-II; Beck et al., 1996; Sanz et al., 2003)

This inventory measures the severity of the depression symptomatology during the last week. The BDI-II is composed of 21 items that reflect cognitive, behavioural, emotional, and somatic symptoms common to depression. Items are rated on a 3-point scale. Higher scores indicate higher severity of the depression symptomatology. The Cronbach’s alpha of the scale in the present study was .92.

Data Analysis

To investigate the internal structure of the Cognitive Flexibility Inventory we used an exploratory structural equation modeling (ESEM) analysis with two latent factors (to replicate the original structure) using Mplus 8.4. We used a weighted least square mean and variance adjusted (WLSMV) estimator due to the non-normal distribution of the data. To investigate the fit of the data to the models, we used the comparative fit indexes (CFI), the Tucker-Lewis index (TLI), and the root mean square error of approximation (RMSEA) indices. For interpretation purposes, CFI and TLI >.90 and >.95 indicate acceptable and optimal fits, respectively (Marsh et al., 2004). RMSEA values ≤.10 indicate an acceptable fit (Weston & Gore, 2006). Confidence intervals (95%) were also calculated and reported. Cronbach’s alpha and omega were used to examine the internal consistency of our adaptation of the Cognitive Flexibility Inventory. Cohen’s d was calculated to compare the effect sizes of the means obtained in each subscale by sex. In addition, differential item functioning (DIF) was calculated with the purpose of detecting whether items of the Cognitive Flexibility Inventory function differently depending on sex. To test for differential item functioning (DIF) in responses based on sex, we followed a series of steps to calculate a single covariate MIMIC model. These steps included: (1) confirmatory factor analysis (CFA) for the total sample, (2) building the MIMIC model without direct effects, and (3) if the modification indices suggested significant direct effects, the model was further tested with these additions, following the approach proposed by Jones (2006). Also, intra-class correlation coefficients (ICC; CI 95%) were used to explore the temporal stability of the Cognitive Flexibility Inventory in a one-month retest. Finally, sources of construct validity were analysed by computing Pearson correlations between the Cognitive Flexibility Inventory and measures of cognitive emotion regulation (CERQ), attributional styles (ASQ), and depression symptomatology (BDI-II). According to Cohen (1992), correlation values ≥.10, .30 and .50 are considered small, medium, and large effect sizes, respectively. Reliability and construct validity analyses were conducted using SPSS v.26. The datasets generated during and/or analysed during the current study are available in https://osf.io/3fbx5/?view_only=119640dc32bf47b2b508718f180cf165.

Results

Descriptive Analysis of the Items

The distributional properties of the 20 items of the Spanish version of the Cognitive Flexibility Inventory (i.e., means, standard deviations, skewness, and kurtosis) are presented in Table 1. Specifically, skewness ranged from -1.41 to 0.58, and kurtosis ranged from -1.20 to 3.53. Thus, we obtained asymmetric values for items 3, 13, 15 and 18. Item 3 revealed a leptokurtic distribution.

Structural Validity Evidence

Structure analyses displayed acceptable/optimal fit indices for the tested model (CFI = .938, TLI = .922, RMSEA = .098, CI 90% [.090, .106]). As shown in Table 2, all the items significantly loaded into one of the two factors proposed by the original version of the Cognitive Flexibility Inventory with loadings over .40. Item 15, which originally belonged to the Control scale, had cross-loadings in both factors, but had a higher loading into the Alternatives factor. Then, a new ESEM analysis was carried out to test the new structure based on 19 items (i.e., without Item 15). Results showed good fit indices (CFI = .948, TLI = .934, RMSEA = .094, CI 90% [.085, .103]), and also salient loadings of each item for corresponding sub-scale (Alternatives, > .39, Control, >.66; see Table 2). Therefore, the following analyses were conducted based on the 19-item structure. Results showed a significant correlation between both factors (r = .134).

Reliability Evidence and Descriptive Coefficients

Cronbach’s alpha and omega indices for each factor are presented in Table 3. All coefficients, both in the total sample and across sex, were above .86. Cohen’s d results show no sex differences between the Cognitive Flexibility Inventory subscales. However, significant gender-related effects were observed in the analysis of differential item functioning (DIF). Specifically, DIF by gender was found for item 10 (i.e., “Soy bueno poniéndome en la piel de otra gente”), with fit indices CFI = .937, TLI = .922, and RMSEA (90% CI) = .095 [.087, .103], indicating that females scored higher than males on this item. After accounting for this direct effect in the MIMIC model for CFI (CFI = .941, TLI = .927, RMSEA 90% CI = .092 [.084, .100]), no other significant effects were observed. One-month retest correlations ranged from .67 to .90.

Construct Validity Evidence

To evaluate the construct validity of the Spanish version of the Cognitive Flexibility Inventory, bivariate correlations were conducted with measures of emotion regulation, attributional styles, and depression (Table 4). The results showed that the two subscales were generally positively associated with the use of adaptive cognitive emotion regulation strategies and inversely related to maladaptive forms of emotion regulation. Specifically, the Alternatives subscale was positively associated with Positive refocusing, Positive reappraisal, Putting into perspective, Rumination, and Refocusing, while negatively linked to Catastrophizing, and Other-blame. The Control subscale was positively associated with Positive reappraisal, Positive refocusing, and refocusing on planning, while negatively associated with Rumination, Catastrophizing and Self-blame.

Finally, both the Alternatives and the Control subscales were positively associated with the positive-composite attributional style but negatively linked to the negative-composite attributional style and the severity of depression symptoms. The strength of the correlations was generally small, except for a moderate correlation between the Alternatives scale and the CERQ-RP.

Discussion

This study aimed to adapt the Cognitive Flexibility Inventory into the Spanish language and examine its psychometric properties in a sample of Spaniards. Following the original developments by Dennis and Vander Wal (2010), we found an acceptable model fit for a two-factor structure of the inventory. However, our analyses suggested a modified two-factor version composed of 14 items in the Alternatives subscale and 6 items in the Control subscale. Similar to previous adaptations of the Cognitive Flexibility Inventory (Kurginyan & Osavolyuk, 2018), a problematic item in the scale was found. Specifically, item 15 (“I am capable of overcoming the difficulties in the life that I face”) loaded on the Alternatives factor as opposed to the Control factor, which is inconsistent with the distribution proposed in the original English version (Dennis & Vander Wal, 2010). After a discussion about the meaning of Item 15 in the Spanish version and the theoretical implications of changing the structure of the Cognitive Flexibility Inventory based on analytical evidence only, the authors of the present study removed this item from the scale. Consequently, we obtained a 19-item version of the Cognitive Flexibility Inventory for Spanish speakers, which showed better fit indices than the 20-item form, as well as reliability and validity evidence.

It is important to note that item 15 is the only direct item that originally belonged to the Control subscale, whereas all items in the Alternative subscale are direct. Evidence has suggested that there is a possible acquiescence bias when items are positively worded, which can affect the reliability and dimensionality of the test by secondary sources of variance when combining direct and reverse items (Checa & Espejo, 2018; Suárez-Alvarez et al., 2018). In fact, the inclusion of positive and negative items when measuring two poles of similar constructs may result in a two-factor structure in which positive and negative items load on different factors (Brown, 2003; Spector et al., 1997). In this regard, it would be advisable that future studies explore the psychometric properties of the Cognitive Flexibility Inventory by controlling for these plausible response style biases combining positive and negative items for both the Alternatives and Control subscales (i.e., rephrasing half of the items in both scales).

Encouragingly, our analyses showed evidence of reliability for both subscales, with reliability coefficients across dimensions above .86. These findings are similar to those obtained by other adaptations of the Cognitive Flexibility Inventory, which support the reliability of the internal consistency of the scale (Gülüm & Dağ, 2012; Kurginyan & Osavolyuk, 2018; Oshiro et al., 2016; Portoghese et al., 2020). Another interesting finding was that we did not find differences in CF between men and women, which is consistent with previous research (M. M. ). In addition, retest analyses also showed evidence about the internal stability of the Cognitive Flexibility Inventory, thus supporting the idea that CF – as evaluated with this inventory – is likely to be relatively stable over time. This is important for clinical purposes, as it suggests that cognitive inflexibility should be a therapeutic target as it might remain relatively stable unless addressed with psychological treatment.

The findings of this study also provided evidence of the construct (convergent/divergent) validity of the Cognitive Flexibility Inventory in relation to other psychological constructs (emotion regulation, attributional style, and depressive symptoms). Previous validation studies of the Cognitive Flexibility Inventory also evidenced that CF is positively associated with the use of adaptive coping styles and negatively related to the severity of depression symptomatology (Dennis & Vander Wal, 2010; Kurginyan & Osavolyuk, 2018). The present study and previous results are consistent with the idea that cognitive inflexibility might be a transdiagnostic process associated with the use of certain emotion regulation strategies and a better emotional adaptation to face difficulties, which is in line with the theoretical model of psychopathology proposed by the Unified Protocol for the transdiagnostic treatment of emotional disorders (Boisseau et al., 2010).

Interestingly, an unexpected positive association was found between CF and rumination. Specifically, higher cognitive flexibility was associated with higher rumination, which has been considered a maladaptive strategy of emotion regulation (Aldao et al., 2010). Rumination can be defined as a repetitive thinking about the reasons and consequences of one’s problems without taking action (Nolen-Hoeksema et al., 2008). This construct consists of two components: brooding and reflection. The former increases depressive feelings by comparison of one’s current situation with some unachieved standard, whereas the latter appears to be related to engaging in cognitive problem solving to alleviate one’s depressive symptoms (Bastin et al., 2014; Treynor et al., 2003). Therefore, and taking into account that our sample is composed of healthy individuals, engaging in adaptive reflection for responding to contextual demands could be adaptive (L. L. Martin & Tesser, 1996), and it could help understand the positive association between CF and rumination. In any case, this interesting finding should be better explored in further research.

Regarding the association between the Cognitive Flexibility Inventory subscales and the measure of attributional styles, we found that the Cognitive Flexibility Inventory was directly associated with the positive-composite score of the ASQ and inversely associated with its negative-composite scale. Therefore, we found that individuals with higher CF tend to perceive that the causes of positive events are stable, controllable, and dependent on themselves, whereas high scores in the Cognitive Flexibility Inventory were associated with a view of negative events as more dependent on causes that are external, unstable, and context-specific. Past research combined positive and negative attributional scores to obtain a total measure of CF (Dennis & Vander Wal, 2010; Kurginyan & Osavolyuk, 2018; Teasdale et al., 2001). However, the ASQ was not developed to be a measure of CF, so the total score may not be appropriate and suitable for this purpose (Fresco et al., 2007). This might explain why past research has obtained unexpected findings when exploring the relationship between the scores obtained with the Cognitive Flexibility Inventory and the ASQ (Dennis & Vander Wal, 2010; Kurginyan & Osavolyuk, 2018), which was interpreted as evidence that CF in the Cognitive Flexibility Inventory was associated with more inflexibility in the ASQ. Based on the results of this study, it is not recommended to use a global computation based on extreme scores on the ASQ as a measure of CF, as it may lead to unexpected findings.

The present study might have made important contributions to the literature on CF. However, a number of limitations should also be considered. For instance, while this instrument might be especially useful for clinical practice, the validation was conducted among college students. Previous validations of the Cognitive Flexibility Inventory have also been conducted with non-clinical samples (Dennis & Vander Wal, 2010; Gülüm & Dağ, 2012; Kurginyan & Osavolyuk, 2018; Oshiro et al., 2016; Portoghese et al., 2020; Shareh et al., 2014), which might be a good starting point for further research. However, it would be recommended to test the validity and reliability of this measure in a clinical sample, including individuals suffering from affective disorders characterized by rigid thinking (e.g., depression, generalized anxiety disorder, or obsessive-compulsive disorder), and particularly to compare CF in clinical and non-clinical populations. In addition, further studies are needed to confirm the current structure of the Spanish version of the Cognitive Flexibility Inventory using confirmatory factor analyses in larger samples.

Despite these limitations, the results of the present study might be important for research and clinical purposes. In particular, this study showed evidence of the validity and reliability of the Spanish adaptation of the Cognitive Flexibility Inventory, enabling their use in clinical and research contexts. This might be an important contribution for professionals interested in a wide range of different disorders, as this instrument could help assess the ability to cognitively adapt and face adverse events, which is a core trait to predict greater therapeutic success in a wide range of psychotherapeutic approaches.