Revisiting the School Burnout Inventory: Further Evidence on Its Factorial Validity, Invariance, and Nomological Network in a Sample of Spanish Adolescents

Introduction

Traditionally, burnout refers to the reactions of stress and exhaustion in the workplace, specifically in the human services context (Maslach & Jackson, 1981; Schaufeli et al., 2002). More recently, the study of this concept was extended to the schools and students. Academic life, like a work, demands tasks, responsibilities and have achievement pressures as attendance or exams (Lee et al., 2010; Salmela-Aro, Savolainen, et al., 2009), a reason why students may experiment burnout as a response of inefficiency and a decrease of personal resources in comparison with the external demands (Parker & Salmela-Aro, 2011; Salmela-Aro, Kiuru, et al., 2009; Salmela-Aro et al., 2017). Burnout is a cumulative process that can have serious consequences affecting students’ achievement and later academic career (Farina et al., 2020; Gabola et al., 2021) and their mental and physical health (Fiorilli et al., 2014; Walburg, 2014). Therefore, an adequate measurement of burnout in the academic context is important if any action has to be taken to improve the wellbeing and academic conditions of the students.

Academic burnout was defined as a multidimensional construct with three correlated dimensions: emotional exhaustion, cynicism, and inadequacy (Maslach & Schaufeli, 1993; Salmela-Aro, Kiuru, et al., 2009). The first dimension, emotional exhaustion, refers to the feeling of chronic stress, fatigue, and overburdened, resulting from the higher levels of tasks and schoolwork; the second dimension, cynicism, is the indifference or negative attitude toward school and school relationships; and the third dimension, inadequacy, refers to the feelings of lack of success and competence as a student and poor academic performance (Farina et al., 2020; Gabola et al., 2021; Kiuru et al., 2008; Salmela-Aro, Kiuru, et al., 2009; Salmela-Aro & Read, 2017).

One of the most used instruments to measure school burnout is the School Burnout Inventory (SBI) developed by Salmela-Aro, Kiuru, et al. (2009), an instrument that is an adaptation of the Bergen Burnout Indicator 15 (BBI-15) to the academic context. The SBI is an instrument that can assess the burnout syndrome with nine items taping the three academic burnout dimensions: emotional exhaustion, cynicism, and sense of inadequacy (Maslach et al., 2001; Salmela-Aro, Kiuru, et al., 2009).

The SBI was originally validated in samples of secondary/high school students. Salmela-Aro, Kiuru, et al. (2009) firstly validated the SBI in a sample of Finnish and Swedish students. They tested the factor structure of the scale with a confirmatory factor analysis and proved that the three-factor solution fitted the data once they remove one of the inadequacy items, e.g., “I feel that I have less and less to give in my schoolwork.” The model worked with two different structures: as a three-factor correlated structure or as a second-order structure with three first-order factors and a second-order factor labeled burnout. This is an evidence that the three theoretical dimensions (exhaustion, cynicism, and inadequacy) are closely related but different constructs.

Since this initial validation almost all researches have used 9 items: four for the dimension exhaustion, three for cynicism, and two for inadequacy. This 9-item SBI has been validated in different samples of secondary school students and adapted to different languages, always replicating the same structure, in Italy (De Stasio et al., 2014; Farina et al. 2020; Fiorilli et al., 2014), Turkey (Secer et al., 2013), in France (Meylan et al., 2015), Switzerland and Italy (Gabola et al., 2021), Chile recently (Carmona-Halty et al., 2022), Germany (Hoferichter et al., 2022), and Russia (Bochaver & Mikhaylova, 2023). In the German validation, they additionally tested a bifactorial model that included a general burnout factor plus three unrelated sub-dimensions (extenuation, cynicism, and inadequacy) in which all items are loaded and they showed that the items of inadequacy and cynicism load more strongly onto a general factor. Although Hoferichter et al.’s (2022) study shows that it makes theoretical sense to test the bifactor model, in our study we decided not to replicate this structure because the inadequacy factor consists only of two items, so the bifactor model is not identified (B. O. Muthén, 2008).

Regarding the validation in a Spanish sample, Moyano and Riaño-Hernández (2013) found that the factorial structure proposed by Salmela-Aro, Kiuru, et al. (2009) was also confirmed, but they adapted the scale including several changes: one of the items of cynicism (“I feel a lack of motivation in my schoolwork and often think of giving up”) was split in two items, one exhaustion item was removed (“I brood over matters related to my schoolwork a lot during my free time”), and a cynicism item was added in relation to the original version of the scale. Additionally, the response scale was also modified, shifting from an intensity scale (from 1 = completely disagree to 6 = completely agree) to a frequency scale (1 = almost never to 6 = almost always). Hence, there is a need to validate the original version scale in Spanish adolescent samples while also considering in this study to analyze its invariance based on the gender and educational level of the participants.

Another critical aspect of psychometric instruments is the reliability. Some results of the validation studies that are shown in Table 1 reported an internal consistency adequate in both the general scale and the three dimensions. However, in other studies the reliability of some dimensions was lower, especially in the inadequacy dimension, as evidenced in the Spanish adaptation initially developed by Moyano and Riaño-Hernández (2013).

In terms of nomological validation, the SBI has shown evidence of having a broad nomological net of variables. For example, the scale and the three dimensions have been positively and significantly correlated with poor mental health outcomes in students, such as depressive symptoms (De Stasio et al., 2014; Fiorilli et al., 2017; Samela-Aro, Kiurum et al., 2009; Secer et al., 2013), anxiety (Moyano & Riaño-Hernández, 2013), and negative emotions (Carmona-Halty et al., 2022), and negatively with academic engagement (Carmona-Halty et al. 2022; De Stasio et al., 2014; Fiorilli et al., 2017; Hoferichter et al. 2022) and academic achievement (Meylan et al., 2015; Samela-Aro, Kiuru, et al., 2009).

Regarding gender differences in burnout, the literature contends that the exposure to school burnout school is different for boys and girls (Gabola et al., 2021). Girls tend to perform better at school than boys (Pomerantz et al., 2002), allocating greater importance to academic achievement compared to boys (Berndt & Miller, 1990), but they also have more exhaustion symptoms (Herrmann et al., 2019; Pomerantz et al., 2002). However, some studies did not support gender differences in any of the dimensions of burnout (Fiorilli et al., 2014; Gabola et al., 2021). Additionally, just a few studies have tested gender measurement invariance of the SBI. Some of the studies have established configural, metric, and scalar invariance of the scale (Carmona-Halty et al., 2022; Fiorilli et al., 2014; Herrmann et al., 2019), but some of them report that they may found invariance only because of the small sample size (Fiorilli et al., 2014).

Furthermore, cross-sectional and longitudinal studies have found that school burnout increases in the transition across school stages (Farina et al., 2020; Lee et al., 2013; Salmela-Aro, Savolainen, et al., 2009) due to the higher demands and stress, especially in the cynicism and inadequacy dimensions (Salmela-Aro & Tynkkynen, 2012). This may lead not only to mean differences but also to a lack of measurement invariance across school stages. However, there is a lack of studies to test the measurement invariance of the SBI across educational stages in the Spanish educational system, and in particular in the transition from Compulsory to Post-Compulsory Secondary Education.

In summary, even though the psychometric properties of the SBI have been tested in multiple studies, the original scale, without modifications, has never been properly tested in Spanish adolescent samples. In addition to it, generally the measurement invariance properties remain unclear and have not been specifically tested in our context regarding adolescents’ gender and educational level. Thus, the aim of this paper is to analyze SBI scale’s psychometric properties in a sample of Spanish adolescent students, focusing on two issues: the scale’s factorial structure and the invariance properties between gender and educational stages.

Method

Sample and Procedure

The total sample is composed of 503 Spanish adolescents aged from 11 to 19 years old (M = 14.60, SD = 1.83), 56% being girls and 44% boys. Regarding their educational stage, 71.4% were in Compulsory Secondary Education (CSE) and 28.6% were in Post-compulsory Secondary Education – PSE. Secondary Education (12-18 years old) is structured in Spain in two stages: CSE (7th-10th grade, 12-16 years) and PSE (11th-12th grade, 17-18 years).

Approval of the authors’ Institutional Ethics Committee was guaranteed (code number H1523870265031). Likewise, the study was carried out with the authorization of the Department of Education of the Valencian Government (Spain) to access the schools and carry out the study. Participation was voluntary and written informed consent from the participants’ parents was obtained prior to the start of the study. The questionnaires were administered in groups during regular school hours by school psychologists familiar with the different instruments.

Instruments and Measures

The School Burnout Inventory (Salmela-Aro, Kiuru, et al., 2009)

This is a scale that measures three components of school burnout with nine items: (1) exhaustion at school (e.g., “I feel overwhelmed by my schoolwork”); (2) cynicism toward the meaning of school (e.g., “I am not motivated to do my school-work and often think of giving up”); and (3) sense of inadequacy at school (e.g., “I usually have feelings of inadequacy about my schoolwork”). The response scale is a 6-point Likert scale from 1 = strongly disagree to 6 = strongly agree.

The General Health Questionnaire (GHQ-12; Lobo & Muñoz, 1996)

Students’ well-being was assessed with the Spanish adaptation of this questionnaire. The GHQ-12 consists of 12 items that assess mental health problems over the past few weeks using a 4-point Likert scale (0 to 3). In this study, higher scores indicate better mental health. Its adequate internal consistency has been shown in previous research and in this study (.86 and .81, respectively).

Motivated Strategies for Learning Questionnaire (MSLQ; Albert, 2017; Pintrich & de Groot, 1990)

School learning affective-motivational dimensions were assessed by two subscales from the Spanish adaptation of the MSQL for secondary education. Items are rated on a seven-point scale from 1 (never) to 7 (always):

Self-Efficacy for Learning

It evaluates the student’s confidence about his/her ability to perform academic tasks. It consists of seven items (e.g., “I’m confident I can understand the most complex material in this course”). Both in previous research (α = .86) and in this study it showed adequate internal consistency (α = .87)

Test Anxiety

This test evaluates the cognitive and emotional components of test anxiety with three items (e.g., “When I take tests, I think of the consequences of failing”). It showed satisfactory internal consistency in the previous research (α = .70), as well as in this study (α = .73).

Schoolwork Engagement

This was assessed through the Spanish adaptation of the Schoolwork Engagement Inventory (SEI; Salmela-Aro & Upadaya, 2012; García-Ros et al., 2018). It has nine items measuring Energy (e.g., “At school, I am bursting with energy”), Dedication (e.g., “I am enthusiastic about my studies”), and Absorption (e.g., “Time flies when I am studying”) with regard to schoolwork, providing an overall schoolwork engagement score that is used in this study. The responses are rated on a five-point scale ranging from 1 (never) to 5 (always). The SEI has also shown adequate internal consistency both in previous research (α = .83) and in this study (α = .82).

Academic Achievement

This is defined as the average grade obtained by the participants at the end of the academic year in which the study was carried out, taking into account all subjects. It was provided by the administrative services of the schools participating in the study.

Statistical Analyses

We estimated a series of competitive confirmatory factor analyses (CFA) models to assess the factor structure of the SBI scale (Salmela-Aro, Kiuru, et al., 2009). First, a baseline model was tested (Model 1) with unidimensional structure. Model 2 hypothesized three related factors of School Burnout: Emotional Exhaustion (four items; EXH1, EXH2, EXH3, EXH4), Cynicism (three items CYN1, CYN2 and CYN3), and Inadequacy (two items INAD1 and INAD2). After testing this model, Model 3 was tested, with Exhaustion, Cynicism, and Inadequacy as three first-order factors and the Overall School Burnout as a second-order factor. Model fit of the CFAs was examined using several statistics and fit indexes with the cut-off criteria by Marsh et al. (2004): chi-square statistic (χ²), comparative fit index (CFI) (values equal or higher than .90), and root mean square error of approximation (RMSEA) and squared root mean residual (SRMR) (values equal or lower than .08 are deemed adequate). All confirmatory models were estimated using MPlus 8.7 (L. K. Muthén & Muthén, 1998-2017) and employing the Robust Maximum Likelihood estimation (MLR).

Additionally, gender and educational stage (CSE vs PSE) invariance was evaluated using multigroup CFA and three levels of equivalence metric invariance (equal loadings across groups), scalar invariance (equal intercepts across groups), and structural invariance (equal variances and covariances of latent factors across groups). Furthermore, to test the relative improvements in fit between different models, we used CFI differences (ΔCFI) with differences greater than .01 or lower than .05 (Little, 1997) indicating a significant deterioration of the model (Cheung & Resvold, 2002). In addition, we reported χ² difference tests to test model fit differences between one model and the previous equivalent model with fewer restrictions, a non-significant chi-square differences suggesting multi-group equivalence or invariance (Cheung & Rensvold, 2002). These analyses were conducted using the R software employing the package Lavaan (Rosseel, 2012).

For reliability analysis, Cronbach’s alpha and McDonald’s omega coefficients were calculated for the overall scale and the subdimensions. Finally, to test the nomological validity of the SBI, Pearson correlations were calculated to examine the relationship between the Overall Burnout scale and the three dimensions (Exhaustion, Cynicism, and Inadequacy) and the student academic achievement and well-being, self-efficacy for learning, test- anxiety, and schoolwork engagement criteria.

Results

Factor Structure and Reliability

According to previous research on the evidence of the factor structure of the SBI, we tested three models: Model 1 that assumes that there is one latent factor underlying all the SBI items, Model 2 that includes three related factors, and Model 3 that explains three first-order factors by a second-order factor. Figure 1 shows the structures of the models tested.

The results of the confirmatory factor analyses are shown in Table 2. Both Model 2 and Model 3 presented better fit index indicators than Model 1, highlighting that the scale is adequate when considering the three dimensions (exhaustion, cynicism, and inadequacy) separately as first-order factors or combined into a single second-order factor of global School Burnout.

Regarding to reliability, in this study, omega coefficients and Cronbach’s alphas were ω = .79, α = .80 for the total scale and ω = .66, α = .65 for exhaustion, ω = .80, α = .80 cynicism and ω = .53, α = .53 for inadequacy.

Invariance Measurement

Our second aim was to test the comparability of the SBI across gender groups and school stages, so we tested configural, metric, and scalar invariance. Model fit results are showed in Table 3, for both the educational level and gender multi-group CFA; differences in the CFI across the three invariance models (configural, metric, and scalar) were negligible (ΔCFI < .05) and the χ² differences were not statistically significant, which indicates invariance between these two groups. Only the metric invariance between gender groups was considered as a slight worsening in fit, with a difference in CFI of .017, larger than the most stringent criterion by Cheung and Rensvold (2002).

Nomological Validity

Pearson correlations of Scholar Burnout, its dimensions, and the selected criterions are shown in Table 4. Scholar burnout correlated r = -.33 (p < .001) with academic achievement, r = -.45 (p < .001) with well-being, r = -.33 (p < .001) with self-efficacy, r = .33 (p < .001) with test-anxiety, and r = -.22 (p < .001) with school engagement. Exhaustion only shows a statistically significant correlation with wellbeing, r = -.40 (p < .001), self-efficacy r = -.10 (p < .05), and test-anxiety r = .26 (p < .001). On its part, cynicism correlated r = -.44 (p < .001) with academic achievement, r = -.32 (p < .001) with well-being, r = -.32 (p < .001) with self-efficacy, r = .24 (p < .001) with test-anxiety, and r = -.32 (p < .001) with school engagement. Inadequacy correlated r = -.31 (p < .001) with academic achievement, r = -.38 (p < .001) with well-being, r = -.33 (p < .001) with self-efficacy, r = .31 (p < .001) with anxiety and r = -.17 (p < .001) with school engagement.

Discussion

The main objective of this study was to analyse the factorial validity and invariance by gender and educational level of the School Burnout Inventory (SBI) in a sample of Spanish adolescents (12-18 years). Previous validation studies conducted in the Spanish context (Moyano & Riaño-Hernández, 2013) introduced substantial changes in both the content of the items and their distribution among the corresponding factors compared to the original version of the SBI (Salmela-Aro, Kiuru, et al., 2009). This issue warranted verifying the adequacy of the original version, which has also been validated in a wide range of countries, as well as its factorial invariance by gender and educational level, which has not been previously assessed in adolescent Spanish population. Furthermore, it was of particular interest to examine its relationship with a broad set of academic outcomes, especially relevant at this educational stage, such as self-regulated learning, academic engagement, well-being, and academic achievement.

To assess the factorial validity of the original structure of the SBI three alternative factorial models were analysed to determine which one best captured its underlying structure. Establishing structural validity is essential, as it ensures the alignment between the theoretical dimensions that underpin the development of the instrument and the empirical relationships observed among its items. This process provides crucial evidence of the questionnaire’s ability to accurately measure the intended constructs. Furthermore, the analyses of factorial invariance by gender and educational level are warranted by the need to confirm that the instrument operates equivalently across different population groups. This is a critical step for ensuring the generalizability of the results and promoting equity in psychological measurement across diverse groups, particularly given the lack of prior research on this topic in Spanish adolescent population.

In line with the findings from the original validation of the SBI (Salmela-Aro, Kiuru, et al., 2009) and its adaptation in different sociocultural contexts (e.g., Carmona-Halty et al., 2022; Fiorilli et al., 2017; Hoferichter et al., 2022; Meylan et al., 2015; Secer et al., 2013) the results show that both the three-factor correlated model (exhaustion, cynicism, and inadequacy) and the hierarchical model, which considers also a second-order factor of global school burnout, exhibit satisfactory fit indices, whereas the one-factor model of the SBI proves clearly inadequate. Furthermore, the invariance analysis indicates that the three-factor correlated model is invariant at the configural, metric, and scalar levels across educational levels. However, the metric invariance by gender shows slightly lower fit indices than the more stringent criteria highlighted in the scientific literature (Cheung & Rensvold, 2002). In any case, these results support the applicability of the SBI across different population subgroups without measurement bias.

In addition, reliability analysis, based on Cronbach’s alpha and McDonald’s omega indexes, indicated good internal consistency, showing values similar to those highlighted by previous validation studies conducted in other contexts. Higher values were observed in the Cynicism subscale and in the overall global scale, while lower values were found in the inadequacy subscale. These results are worth noting. In particular, as also evidenced in various previous studies, the reliability indexes for inadequacy are clearly lower than they should. The SBI use only two items to measure inadequacy may be behind them (Madigan & Curran, 2021), and it should be noted that this lack of reliability can lead to an attenuation of the effect sizes, a reduction in statistical power, higher standard errors, and inaccurate or even misleading estimates (Haferman et al., 2021; Nimon et al., 2012). On the other hand, Bollen (1989) has shown that estimation problems are hard to detect in multivariate analyses. All this should be in mind when this scale is used and these same reliability problems arise. Finally, the convergent validity of the SBI was confirmed through correlational analysis between the subscale scores and various relevant educational outcomes. All observed relationships were consistent with the results of previous investigation, reinforcing the relevance and usefulness of the instrument for assessing psychological constructs in educational context.

In summary, the results provide strong support for the factorial validity of the Spanish adaptation of the SBI conducted in this study, which scrupulously respects the item content and underlying structure of its original version (Salmela-Aro, Kiuru, et al., 2009). The results confirmed the adequacy of the structure obtained in the original validation study, consisting of three correlated underlying factors (exhaustion, cynicism, and inadequacy), and additionally providing an overall academic burnout score. Furthermore, its invariance across gender and educational level was also evidenced, as well as its internal consistency and convergent validity with a wide range of academic outcomes particularly relevant in secondary education.