Assessment of a Multidimensional School Collective Efficacy Scale to Prevent Student Bullying: Examining Dimensionality and Measurement Invariance

Introduction

Bullying is an aggressive, goal-directed behavior that occurs when someone intentionally inflicts physical, emotional, or social injuries upon another. This behavior is carried out repeatedly and over time in interpersonal relationships, with unequal power between victim and aggressor (Olweus, 1993; Volk et al., 2014). Bullying is associated with short-term and long-term consequences, such as lower academic outcomes, psychological disorders, and behavioral problems (du Plessis et al., 2019; Garaigordobil & Machimbarrena, 2019; Nielsen et al., 2015; Sigurdson et al., 2015; Sourander et al., 2016). It is, in fact, a public health issue faced worldwide (Modecki et al., 2014; United Nations Educational, Scientific and Cultural Organization [UNESCO, 2017]; Zych et al., 2017). Some researchers have argued bullying is a pervasive behavior that occurs in social contexts, regardless of whether it is promoted, supported, or rejected (Espegale et al., 2013; Martínez-Ferrer et al., 2011; Swearer & Hymel, 2015). However, other scholars (de Oliveira et al., 2015; Hymel et al., 2015; Roland & Idsøe, 2001; Valdés-Cuervo et al., 2018) have found that school-level variables are responsible for differences in bullying rates between schools. Specifically, research has highlighted school collective efficacy as a whole-level school factor is associated with lower rates of peer aggression (Barchia & Bussey, 2011; Olsson et al., 2017; Sapouna, 2010; Thornberg et al., 2019; Williams & Guerra, 2011).

School Collective Efficacy

The collective efficacy theory developed from the study of violence in neighborhoods (Sampson & Raudenbush, 1999; Sampson et al., 1997). It is rooted in social capital (Bourdieu, 1984; Coleman, 1990) and social cognitive theories (Bandura, 1986, 1989). It aimed to examine the mechanism through which the social context might affect individual behavior. The collective efficacy theory acknowledges that violence is associated with the social resources that the community holds, as well as the beliefs shared by its members (Häuberer et al., 2011; Sampson et al., 1997; Wilcox et al., 2018). Hence, social resources and shared beliefs are critical to collective agency (Bandura, 2000, 2001). Therefore, violence prevention in schools relies on the whole-school context, rather than only on the students themselves.

In this regard, some scholars (Coleman, 1990; Sampson, 2006) assert that friendships and fellowships are not enough to avoid violent episodes inside schools. They argue that students must feel a strong tie and belong to ‘the group’ in order to take co-responsibility for its success and to protect its members. According to the collective efficacy theory, shared feelings of cohesion and trust, along with a willingness of members to exert informal social control, lead to the values of the community, but also sets the goals in terms of violence prevention (Sampson et al., 1997; Wilcox et al., 2018). Cohesion and trust refer to the level of confidence and support among members of a group, who act accordingly to accomplish commons goals (Sampson et al., 1999). Informal social control refers to the willingness of community members to intervene for the common good (Sampson et al., 1997). It involves informal practices and arrangements that contribute to the institutional and community order (Black, 1984; Taylor, 2001). Informal social control is particularly effective because it reduces coercion (Hirschi, 2001; Janowitz, 1975; Wiatrowski et al., 1981), that is, results are more effective when leading individuals of a community embrace collective value goals, rather than forcing them to adjust themselves to a social reality.

In bullying research, school collective efficacy is conceptualized as school members’ beliefs about the cohesion and trust between students and adults at school, as well as the willingness of school members to support victims of bullying (Hymel et al., 2015; Williams & Guerra, 2011). School cohesion and informal social control have influence on both adults and peers to provide support when bullying occurs (Capone et al., 2018; Kurnianingshi et al., 2012; Payne & Gottfredson, 2004; Thornberg & Elvstrand, 2012; Trach et al., 2013; Wach et al., 2018; Williams & Guerra, 2011).

Measures of Students’ Perception of School Collective Efficacy

Although researchers have developed scales for measuring school collective efficacy as a means to predict bullying episodes, most are considered either unidimensional (e.g., Barchia & Bussey, 2011) or multidimensional (physical, relational, and verbal aggression) indicators of social informal control (e.g., Wänström et al., 2017). That is, none of these scales have included indicators of cohesion and trust. While current scales allow researchers to assess overall information on school collective efficacy, this type of measurement does not provide insights into the theoretical multidimensionality of the construct (Sampson & Raudenbush; 1999; Sampson et al., 1997). Thus, if school collective efficacy is a multidimensional construct that comprises cohesion, trust, and social informal control, then, the existing instrument is limited to the exploration of differences between school efficacy dimensions. More recently, scholars (e.g., Sapouna, 2010; Williams & Guerra, 2011) have developed other scales to address the lack of a theoretical multidimensional model; these scales, however, have presented limited evidence about their psychometric properties.

Specifically, Sapouna (2010) developed a two-dimensional instrument to measure social cohesion and trust, and informal social control with Greek students ranging in age from 11 to 14 years. However, the author did not report the psychometric properties of the scale. In the case of Williams and Guerra (2011), their three-dimensional instrument had several important issues as well. Scholars tested American pre-adolescents and adolescents by running an exploratory factor analysis with principal components extraction and oblique rotation, which supported a three-dimensional (cohesion and trust, students’ informal social control, and teachers’ informal social control) solution (67% explained variance). Nonetheless, their results should be taken with caution since the principal component analysis (PCA) method has an important weakness: PCA assumes that all indicator variance is common. The assumption is rigorous and does not allow for specific variance; that is, it assumes that scores are perfectly reliable (Kline, 2013; Park et al., 2002). Also, PCA only analyzes observed variables; therefore, it should not be used if scholars seek to obtain parameters reflecting latent constructs (Gaskin & Happell, 2014; Widaman, 1993). Finally, Williams and Guerra (2011) do not report the correlation between the factors, which makes it difficult to analyze the nature of the relationship between factors and the scoring of the scale.

After conducting a literature review, more important gaps became apparent. First, there is a dearth of studies comparing the adjustment of different measurement models of the construct (school collective efficacy). Second, no study known by the authors has analyzed the measurement invariance of the proposed model with relevant variables, such as gender and stage of development (e.g., childhood, pre-adolescence, adolescence, adulthood). Third, research about the external construct validity of measures of school collective efficacy, remains scarce. And fourth, to our knowledge, no previous studies have analyzed the psychometric properties of a scale to measure collective school efficacy in Latin American adolescents, specifically.

Considering all of the above, in an effort to address these gaps in the current body of literature, we sought to develop an instrument capable of measuring the three dimensions of school collective efficacy (cohesion and trust, students’ informal social control, and teachers’ informal social control). We drew on the work of Sampson and Raudenbush (1999), and Sampson et al. (1997), to develop a theoretically grounded and psychometrically robust scale, capable of measuring the different indicators of school collective efficacy in Mexican adolescents.

Measurement Invariance

Prior studies (Barchia & Bussey, 2011; Marées & Petermann, 2010) suggest student perception of school collective efficacy differs by gender and educational level. Overall, girls report higher perceptions of school collective efficacy in bullying prevention than boys throughout schooling (Barchia & Bussey, 2011; Cho, 2017; Lee & Jo, 2017). Nonetheless, research on school collective efficacy has produced limited evidence on age differences, with a single study found by the authors reporting that secondary students perceive a higher school collective efficacy than high school students (Barchia & Bussey, 2011). These studies, however, did not consider the effects of measurement invariance when making group comparisons, so it is still unknown to what extent school collective efficacy is interpreted in the same way across gender and educational level (Brown, 2015; Millsap & Olivera-Aguilar, 2012). Thus, if school collective efficacy is different across gender and educational level, and these variations are not considered, it is inadequate to compare school collective efficacy across these groups (Millsap & Olivera-Aguilar, 2012; Putnick & Bornstein, 2016). Therefore, it is essential to examine whether the measurement of school collective efficacy is similar by gender and stage of adolescence. Without confirming measurement invariance, group comparisons are prone to be misrepresented. Measurement invariance allows a comparison of school collective efficacy among these groups or common scores that can be used to predict school bullying or other variables (Byrne, 2016; Van de Schoot et al., 2015).

Concurrent Validity

Scholars suggest that the influence of school collective efficacy on bullying prevention is explained by its relationship with whole-school variables (Hymel et al., 2015; Williams & Guerra, 2011). Although research about benefits or consequences of school collective efficacy on school environment is scarce, related studies (e.g., Barchia & Bussey, 2011; Capone et al., 2018; Thornberg et al., 2019; Thornberg et al., 2020) show that collective efficacy is associated to indicators of quality of school environment. Thus, to assess the external validity of the scale, we analyzed relationships of school collective efficacy with whole-school level variables, such as school collective moral disengagement (Barchia & Bussey, 2011; Gini et al., 2015; Pozzoli et al., 2012; Thornberg et al., 2019), sense of school community (Capone et al., 2018; Hymel et al., 2015; Prati et al., 2017), and school fairness (Konishi et al., 2017; Lenzi et al., 2014; Vieno et al., 2013).

The literature defines collective moral disengagement as the perception of individuals about the inclination of school members to use cognitive mechanisms to justify immoral actions (Allison & Bussey, 2017; Bandura, 2002). In this regard, some studies (Barchia & Bussey, 2011; Gini et al., 2015; Pozzoli et al., 2012; Thornberg et al., 2019) report a negative association between school collective efficacy and collective moral disengagement in bullying studies. These results suggest that school collective efficacy to prevent bullying leads school members’ moral responsibility in immoral behaviors. In the case of sense of school community, some scholars (Albanesi et al., 2007; Prati et al., 2017) argue that school collective efficacy spurs sense of school community in bullying prevention. Albanesi et al. (2017), explain that the sense of community involves students’ perceptions of emotional connection with peers and school staff, and opportunities to participate in school life. In fact, it is argued that personal needs are satisfied through such school membership (Albanesi et al., 2007; Prati et al., 2017). Hence, school collective efficacy promotes school fairness, as it prompts students to perceive school rules and discipline practices as just and fair (Peguero et al., 2017; Peterson & Seligman, 2004).

The Present Study

Prior studies have reported a significant effect of school collective efficacy on reducing peer aggression and bullying (Barchia & Bussey, 2011; Olsson et al., 2017; Sapouna, 2010; Thornberg et al., 2019; Williams & Guerra, 2011). However, there are important gaps in the measurement of student perceptions of collective school efficacy. Under this context, this study sought to: (1) analyze the internal structure of validity by examining the fit of different measurement models of the construct: only one factor, two first-order factors (cohesion and trust and social control), and three first-order factors (cohesion and trust, students’ social control, and teachers’ social control) (see Figure 1; see Appendix); (2) test reliability by running average variance extracted (AVE) and internal consistency of scores; (3) explore measurement invariance by gender and educational level (secondary vs. high school); (4) compare latent variable mean differences across groups if measurement invariance was present; and (5) analyze (discriminant and concurrent) external validity, by analyzing its relationships with collective school moral disengagement, sense of school community, and school fairness.

To accomplish these goals, five hypotheses were explored. Hypothesis 1 (internal structure): the three first-order factor structure has a better adjustment to the data than the one or two first-order factor models. Hypothesis 2 (measurement invariance): the scale is invariant by gender and educational level. Hypothesis 3 (means comparison): girls hold a greater perception of school collective efficacy than boys. Given the scarcity of studies on education level group differences, we make no specific hypotheses regarding these differences. Hypothesis 4 (discriminant validity): each subscale discriminates from the other subscales of the school collective efficacy scale. And, hypothesis 5 (concurrent validity): school collective efficacy has positive relations with sense of school community and school fairness, and a negative one with school collective moral disengagement.

See complete item question in Appendix.

Method

Participants

Participants came from urban public secondary (n = 83) and high schools (n = 30) across different cities in the state of Sonora, Mexico (Hermosillo, Obregón, and Guaymas). The sample of secondary students included 584 participants (6 to 8 students per school), 300 males and 284 females, aged from 11 to 14 years old (M_age = 12.67 years old, SD = .77 years old); 204 (35%) of them were enrolled in seventh grade, 187 (32%) in eighth grade, and 193 (33%) were in ninth grade. The sample from high school (17 to 20 students per school) included 279 (48.3%) males and 305 (51.7%) females who ranged from 14 to 17 years old (M_age = 15.95 years old, SD = .78 years); 187 (32%) attended tenth grade, 210 eleventh grade, and 187 (32%) twelfth grade. Like most public urban schools in Mexico, these schools included students from low and middle socio-economic status (National Institute for the Evaluation of Education [INEE, 2018]).

Measures

School collective efficacy to bullying prevention. The work conducted by previous authors (Barchia & Bussey, 2011; Sapouna, 2010; Williams & Guerra, 2011) was used to develop a self-reported scale. After building 30 items aimed to measure school collective efficacy, we sought assistance from five knowledgeable researchers in the field of bullying and two school counselors as well. Specifically, they were asked to assess the relevance of each item. The experts rated the relevance of each item on a 4-point Likert-type scale (e.g., “Is this item relevant to measure school collective efficacy?”; 1 = not relevant, 2 = unable to assess relevance without item major revision, 3 = relevant, and 4 = very relevant). Through this process, 14 items were most appropriate and included in the School Collective Efficacy Scale (SCEE), as they had a context validity index (CVI) greater than .78 (Almanasreh et al., 2019; Lynn, 1986).

Included items are indicators of cohesion (e.g., “I believe my school has achieved agreement from students to end bullying”); informal students’ social control (e.g., “Students at my school stop bullying when a student is victim of teasing from a stronger student”); and informal teachers’ social control (e.g., “Teachers at my school stop bullying when a student is victim of rumors and lies”). Research participants responded using a four-point Likert scale from (from 0 = strongly disagree, to 4 = strongly agree). Items were coded so that higher scores indicate higher school collective efficacy to prevent bullying.

School collective moral disengagement. The Classroom Collective Moral Disengagement Scale (CMD; Kollerová et al., 2018) was used. The back-translation method was used for the appropriateness of the scale to Mexican adolescents. This scale is a uni-dimensional measurement model with 10 items (e.g., “In this school, how many students believe that it is all right to fight to protect friends,” α = .81, ω = .82, AVE = .62). Students were asked about “In your school, how many students think that” in a 5-point scale (0 = none 0%, to 4 = everyone 100%). The CFA indicated a good model fit to the data (SBχ² = 48.51, df = 31, p = .024; SRMR = .03; AGFI = .98; CFI = .99; TLI = .99; RMSEA = .02, CI 90 [.01, .04]).

Sense of school community. The Scale of the Sense of Community in the School (Prati et al., 2017) was used. The back-translation method was used for the translation of the scale from English to Spanish. The scale included 11 items grouped in three dimensions: (a) membership (3 items, e.g., “I’m proud to belong to this school,” α = .78, ω = .79, AVE = .51), (b) emotional connection (3 items, e.g., “I feel that I can share experiences and interest with other students at my school,” α = .79, ω = .81, AVE = .53), and (c) opportunities (5 items, e.g., “The students at my school are involved in organizing school events”, α = .82, ω = .82, AVE = .57). The CFA supported a good model fit (SBχ² = 57.24, df = 31, p = .003; SRMR = .04; AGFI = .97; CFI = .99; TLI = .98; RMSEA = .03, CI 90 [.01, .04]).

School fairness. The subscale of fairness adapted the International Personality Item Pool (IPIP; du Plessis & Bruin, 2015). The back-translation method was used for the translation of the scale from English to Spanish. Then, the scale was adjusted so that the student refers to the whole-school instead of personal fairness. The subscale comprises 9 items (e.g., “In this school, students admit when they are wrong,” α = .86, ω = .86, AVE = .63). Likert scale responses with options from 0 = totally disagree, to 4 = totally agree were used. The CFA supported that the model fit the data (SBχ² = 50.28, df = 24, p = .002; SRMR = .03; AGFI = .95; CFI = .97; TLI = .96; RMSEA = .05, CI 90 [.03, .06]).

Procedure

Researchers first gained permission from the Ethical Committee of the Technological Institute of Sonora (Number 2020_0014). An invitation to participate in the study was sent, through the education authorities, to the principals from different secondary and high schools from three cities of the state of Sonora, Mexico (Hermosillo, Obregon City, and Guaymas). Schools with directors that expressed their interest in participating were included in the study. Then, a consent letter was sent to potential research participants’ parents to request their children participate in the study. Only 6% of parents refused to allow their children to participate. We explained the purpose of the study to research participants; we also underlined their participation would always be voluntary. They were also informed about the anonymity of their participation, which was ensured throughout the procedures. Data collection was carried out grouping students from each institution with sessions of about 45 minutes.

A back-translation method was used to ensure accuracy for the translation of the scales from English to Spanish. A bilingual Mexican researcher translated scales from English to Spanish. Then, a professional translator was asked to complete the Spanish translation back to English. Two other translators then compared that new translation with the original text. Finally, any differences between the two versions were reconciled in the translator focus group and elaborated into the final Spanish version.

Data Analysis

The percentage of missed data was 3%. Missing data was treated by the means of multiple imputation, which is available in SPSS 25. We examined the hypothesized factor structure of the measurement of school collective efficacy and its invariance by gender and educational level (secondary vs. high school). The analyses were conducted using the robust maximum likelihood estimator (MLR) in Mplus 8. Mplus was chosen due to the sample having students nested within schools, and Mplus is able to account for the nested nature of the data by fitting a multilevel model with random-effects and produces adjusted standard errors in the model estimations.

Dimensionality analysis. To examine the factorial validity of the SCEE, several confirmatory factor analyses (CFA) were calculated. The models described above were analyzed and compared (uni-dimensional, two-dimensional, and three-dimensional factor model). In evaluating the goodness of fit of the hypothesized models with MLR estimator, we used the χ² statistic based on the Satorra-Bentler scaled statistic (SBχ²; Satorra & Bentler, 2010). The SBχ² statistic incorporates a scaling correction for the χ² statistic (Hu et al., 1992). In addition, we reported the standardized root mean square error of approximation (SRMR ≤ .05), comparative fit index (CFI ≥ .95), Tucker-Lewis index (TLI ≥ 90), and root mean square error of approximation (RMSEA ≤ .05) with their confidence interval (Byrne, 2016; Sharma et al., 2005). In order to compare models, we used differences in SBχ² (ΔSBχ²) and Bayesian information criterion (BIC). We assumed that when differences in the ΔSBχ² result was significant, a model with less SBχ² had better adjustment to the data (Byrne, 2016; Wang & Wang, 2012). Differences in BIC > 10 indicates differences in the model’s fit to the data, a model with less BIC has strong adjustment (Byrne, 2016; Muthén & Muthén, 2017; Raftery, 1995).

Reliability analysis. Reliability of each scale was calculated using the McDonald coefficient (ω), composite reliability (CR), and average variance extracted (AVE). Values of w and CR ≥ .70 and AVE ≥ .50 were considered indicators of an acceptable reliability of the scores (Bacon et al., 1995; Green, 2015; Hair et al., 2017).

Measurement invariance analysis. Nested factor models were fit in order to test measurement invariance, following the methods proposed in the literature (Millsap & Olivera-Aguilar, 2012; Putnick & Bornstein, 2016; Widaman & Resie, 1997). We tested configural invariance (constrained the number of factor and pattern of fixed and free factor loading to be the same across gender and educational level), metric invariance (constrained factor loadings to be equal across groups), and scalar invariance (constrained measurement intercept across groups). The nested models were compared by using the difference in the ΔSBχ², albeit with a correction to the value, as this difference in not distributed as a χ² statistic. Differences in ΔSBχ² greater than the critical value (ΔSBχ² with p > .001) suggests that constraints imposed are not equivalent across groups (Byrne, 2016; Muthén & Muthén, 2017). Nevertheless, the ΔSBχ² statistic is sensitive to larger samples sizes (Tomarken & Waller, 2005; Wang & Wang, 2012); thus, past researchers have recommended to use goodness-of-fit indexes, such as ΔCFI and ΔRMSEA (Byrne, 2016; Cheung & Rensvold, 2002; Sass & Smith, 2013). These guidelines were relied on for this study, with a difference in CFI (ΔCFI) less than .01, and differences in RMSEA (ΔRMSEA) less than .015. In cases where methods disagreed, we relied more on differences in CFI and RMSEA due to the large sample size in this study.

Based on evidence of invariant factor loading and intercepts, we tested mean differences by gender and educational level (secondary vs. high school). For this, the factor means for the reference group (boys and secondary students) were set to zero, while we estimated groups’ factor means freely. A z statistic was used to test differences between the latent means of the two groups (Aiken et al., 1994; Byrne, 2016).

Discriminant validity analysis. Discriminant validity requires evidence that the hypothesized differences between the constructs are empirically grounded (Farrell, 2010). Therefore, it requires that a latent variable can account for more variance in the observed variable associated with it than other constructs within a similar framework (Fornell & Larcker, 1981). For the purpose of this study, we used the criterion that average variance extracted (AVE) in each SCE subscale is greater than the square of this correlation with the other subscales (Fornell & Larcker, 1981; Hair et al, 2010).

Concurrent validity analysis. Concurrent validity evidence refers to which scale scores are correlated in an expected manner with other constructs that are measured at the same time (Furr & Bacharach, 2014). To examine concurrent validity of the SCEE, correlations between the constructs involved in the study were calculated. Then, the correlation between school collective efficacy was calculated with similar (school fairness and school sense of community) and different constructs (school collective moral disengagement). Also, we evaluated the effect-size of these correlations based on the guidelines proposed by Funder and Ozer (2019). They suggested that an effect-size r of .10 is small for single events but potentially consequential, an effect-size r of .20 reveals a medium effect that is of some practical use in both the short and long term, and an effect size r of .30 suggests a large effect that is influential in the explanation in both short and long term.

Results

Descriptive Analysis and Multivariate Normality

Table 1 shows mean, standard deviation, range, skew, and kurtosis statistics. In most of the items (10 items), means are centered in the neither agree nor disagree middle category), while in the remaining (4 items) it is centered in the disagree category. These findings imply that respondents did not ‘agree’ with any of the item statements. Regarding the values of skewness and kurtosis, these suggest normal univariate distribution in all items. Finally, the Mardia coefficient was 4.98, which suggests multivariate normality.

Assessing Measurement Models

Confirmatory factor analysis of student responses to the 14 items verified that only the three-dimensional model fits best to the data. We compared the three-dimensional model (Model C) with the uni-dimensional (Model A), and the fit of Model A was worse to a statistically significant degree than Model C (ΔSBχ² = 493.93, df = 3, p < .001; ΔBIC = 261.44). We further compared the three-dimensional model (Model C) and the two-dimensional model (Model B), the results show that Model B was also worse that Model C with statistical significance (ΔSBχ² = 118.79, df = 2, p < .001; ΔBIC = 155.8). Therefore, based on theory and goodness of fit of the model, Model C was the preferred model, with the remaining analytical report based on this model (Table 2).

The estimated standardized factor loadings for the three-dimensional model is shown in Figure 2. The standardized factor loadings ranged from .63 to .83, and were statistically significant at the .05 level. The three-factor model provided the most acceptable fit to the data (SBχ² = 52.77, df = 30, p = .006; SMRM = .04, TLI = .96; CFI = .99; TLI = .98; RMSEA = .03 IC 90 [.01, .04]). Furthermore, patterns in the bivariate correlations of the factors were found. Cohesion, students’ social control, and teachers’ social control were positively correlated. The correlations among the six factors were moderate to high (.28-.44). The reliability statistic for each latent variable is acceptable: cohesion (McDonald omega, ω = . 81, composite reliability CR = .89, and average variance extracted AVE = .58), students’ social control (ω = .72, CR = .79, AVE = .50), and teachers’ social control (ω = .81, CR = .82, AVE = .54). In sum, these results provide empirical evidence for measuring school collective efficacy with a three-dimensional model, rather than uni-dimensional or two-dimensional construct.

All correlations and standardized factor loadings are statistically significant (p < .001). See complete item questions in Appendix.

Assessing Measurement Invariance by Gender

As shown in Table 3, the goodness of fit statistic suggests a measurement invariance for this model by gender. The configural model, which imposed no equality constraints between groups, served as a baseline model. Results showed the goodness of fit statistic had an acceptable fit to this model (SBχ² = 85.98, df = 60, p = .016; SRMR = .05; CFI = .98; TLI = .97; RMSEA = .03, CI [.01, .04]). When all factor loadings were constrained to be equal across gender (metric invariance), the difference between this model and the configural model was not statistically significant, ΔSBχ² = 5.67, df = 7, p = .578, also the CFI and RMSEA values differed by no more than .01 and .015, respectively. In addition, intercepts of the observed variables were constrained to be equal by gender, with the differences of the metric model being not statistically significant, ΔSBχ² = 20.29, df = 6, p = .317, and differences in CFI and RMSEA are small (CFI < .01, RMSEA < .015).

Assessing Measurement Invariance by Educational Level (Secondary vs. High School)

To examine whether the three-dimensional factor model is invariant between secondary and high school students, a nested model similar to those explored for gender were tested (see Table 3). The baseline model (configural invariance) fit well to the data, which confirmed the validity of the model’s internal structure (SBχ² = 103.11, df = 60, p < .001; SRMR = .05; TLI = .96; CFI = .97; RMSEA = .03 CI 90 [.02, .04]. The model with factor loadings constrained (measure invariance) across groups fit well, based on the criteria of the SBχ² difference and the change of CFI and RMSEA (ΔSBχ² = 16.95, Δdf = 12, p = .151). These results suggest that the factor loadings are consistent for secondary and high school students. Finally, we added the intercept constraints to the model comparison (scalar invariance). In these models, there was no significant statistical differences in SBχ² (ΔSBχ² = 31.55, Δdf = 6, p = .025) and changes in CFI and RMSEA were smaller than the standard (ΔCFI = .004, ΔRMSEA = .001) suggested, so there are no substantial differences in the intercept of observed variables between early and middle adolescents.

Assessing Group Differences in Three-Dimensional Model Factor Means

To estimate the difference between the three-dimensional model means by gender, the boys were chosen as a reference group and their latent means were set to zero. Then, the latent means of the girls informed the difference in factor means between the two groups (see Table 4). The test revealed statistically significant mean differences by gender on one of the factors. Particularly, girls had higher scores on cohesion. The gender differences in students’ informal social control, and teachers’ informal social control were not statistically significant.

In terms of latent means differences by educational level, we chose the secondary students’ group as the reference group and estimated the latent mean of the middle adolescent group. Results showed one statistically significant means difference between secondary and high school students (see Table 4). High school students had a higher score on students’ informal social control than secondary students. The differences in cohesion and teachers’ informal social control were not statistically significant.

Discriminant Validity

Table 5 shows the correlations between the scales, the squares of those correlations (in parenthesis), and the average variance explained (AVE). The squared of the correlations (R²) between the subscales of SCEE is lower than AVE. This result indicates evidence of discriminant validity, according to the standard criteria (see Fornell & Larcker, 1981; Hair et al, 2010).

Concurrent Validity

With respect to concurrent validity, the subscales of SCEE correlated significantly with construct measures in the study (see Table 5). As expected, the subscales of SCEE had a positive correlation to school fairness and sense of school community and a negative correlation to school collective moral disengagement. Moreover, the effect-size of these correlations has potential consequences. The correlation between teachers’ social control and school collective moral disengagement had a small effect-size (r > .10), which indicates a potential consequence at the level of individual events. The effect-size of all other correlations ranged between medium (r > .20) and large (r > .30), which may indicate important short and long-term practical consequences (Funder & Ozer, 2019). In summary, these results provide evidence of concurrent validity of the SCE subscales.

Discussion

In the present study, uni-dimensional and multidimensional conceptualizations of school collective efficacy were tested. To do so, we examined psychometric properties of these models, as well as the invariance and external validity (discriminant and concurrent) of a three-dimensional factors model. Based on Williams and Guerra (2011), we hypothesized that the three-factor structure is better fit to measure school collective efficacy in the prevention of bullying. This study contributes to the literature regarding school collective efficacy in several ways. First, findings confirmed a better adjustment to the data of the three-factors model and found empirical evidence to support a multidimensional structure of the school collective efficacy construct. Second, empirical evidence supports the three-dimensional factor model for boys and girls, and secondary and high school students. This psychometric feature is critical for studies about antecedents and consequences of school collective efficacy. Finally, in terms of correlations between scales, we found expected correlations that provide evidence of discriminant and concurrent validity.

School Collective Efficacy as a Multidimensional Construct (Hypothesis 1 internal structure)

Findings support the theory that school collective efficacy is a multidimensional construct (Sampson et al., 1997; Sampson & Raudenbush, 1999). The hypothesized three-dimensional model showed that responses to the measurement of school collective efficacy can be grouped in three first-order factors (cohesion, students’ social control, and teachers’ social control). Compared to one-dimensional and two-dimensional factor models, the three-dimensional model had better adjustment to the data. Future research should examine whether the different dimensions of school collective efficacy have different antecedents and consequences.

Measurement Invariance across Gender and Educational Level (Hypothesis 2 measurement invariance)

The results provide empirical evidence supporting measurement invariance of school collective efficacy to prevent bullying by gender and educational level (secondary vs. high school). For group comparisons by gender and education level, school collective efficacy can be measured with the same metric. Thus, score differences in the factors reflect group differences in the perception of school collective efficacy rather than item bias. Furthermore, we are confident that comparing variations of the construct in these groups are meaningful. Establishing measurement invariance allows research to make appropriate conclusions about antecedent and consequence variables associated with school collective efficacy to bullying prevention in these groups.

Means Comparison (Hypothesis 3 means comparison)

Considering that both factor loading and intercepts were invariant, we examined latent means differences on the three-dimensional model. Regarding gender differences, results suggest that mean scores for girls were higher than for boys in perception of cohesion in school, but not in students’ social control or teachers’ social control. These results are consistent with other recent research (Barchia & Bussey, 2011; Cho, 2017; Lee & Jo, 2017), and thus provide validity to these scales. With regard to educational level differences, we found the mean scores for high school students to be higher than secondary students on perceptions of students’ social control but not on cohesion or teachers’ social control. We suspect that these differences may reflect psycho-social developmental differences. Future research should examine causes and consequences of differences by gender and educational level in students’ perception of school collective efficacy to prevent bullying.

Discriminant Validity (Hypothesis 4 discriminant validity)

In this study, based on Williams and Guerra (2011), we hypothesized that school collective efficacy to prevent bullying has a three-dimensional internal structure. Relationships between these factors should be further established to guide studies about the construct. Results about discriminant validity of the factors have shown that each subscale measures a subconstruct of a different nature (Barchia & Bussey, 2011; Capone et al., 2018; Hymel et al., 2015). These results suggest that research in bullying should analyze the causes and consequences of these dimensions in bullying prevention.

Concurrent Validity (Hypothesis 5 discriminant validity)

Moreover, the SCE showed concurrent validity. We found positive, statistically-significant correlations between school collective efficacy factors, school fairness, and sense of school community. Also, a negative correlation to school collective moral disengagement was found. The values of these correlations provide evidence of effect sizes with practical consequences in short- and long-term. In summary, findings suggest that these whole-school variables are associated and should be possible to consider them as indicators of a healthy school.

Practical Implications

The findings suggest that unidimensional interpretations for school collective efficacy to prevent bullying may not provide sufficient evidence of associated factors. In fact, this study provides empirical support to measure school collective efficacy as a three-dimensional construct. Our study provides a scale to practitioners for assessing students’ perception of school collective efficacy to prevent bullying as an element to design school interventions able to reduce bullying rates. Practitioners have a useful instrument to measure a set of factors of school collective efficacy that can be targeted in antibullying intervention. The analysis of means differences allows for identifying groups of students (boys and secondary students) who are perceived to have less healthy school functioning, and suggests that intervention should focus on them in particular.

Our analysis suggests school collective efficacy is associated to other indicators of whole-school health. Specifically, it shows that dimensions of school collective efficacy have positive associations to school fairness and sense of community and has a negative influence on school collective moral disengagement. As research indicates, school collective efficacy is associated with a healthy whole-school climate and bullying prevention (e.g., Goddard et al., 2015; Hymel et al., 2015; Olsson et al., 2017; Thornberg et al., 2019). Practitioners may consider aiming to intervene in the different dimensions of school collective efficacy studied here in order to enhance students’ perceptions of school safety and bullying prevention. Students with lower views of school collective efficacy could be identified for special interventions. These prevention efforts are more effective in bullying prevention when they involve strategies that improve school members’ cohesion, students’ informal social control, and teachers’ informal social control.

Limitations

Several limitations of this study should be taken into account. First, results rely on self-reported data aimed to assess school collective efficacy to bullying prevention. Therefore, we acknowledge further studies should include multiple sources of data (e.g., teachers and principals) and measurement methods (e.g., interview or observation) as well, as a means to offer a more robust instrument to measure school collective efficacy. Second, our findings are from a sample from a specific urban region of Mexico. It is essential to consider that adolescents from rural and indigenous communities might have different experiences regarding school efficacy to prevent bullying. It is desirable to use cross-national samples from different age groups and across diverse environments to provide empirical support for the school collective efficacy scale in future studies. Third, although school collective efficacy is a relevant variable to prevent bullying, more research is needed to establish an association between a new measure and bullying. Fourth, the same sample was used to establish the dimensionality of the scale and to study its measurement invariance, which in turn may be overestimating the fit of invariance models. Finally, the cross-sectional nature of the design prevents from assessing how the construct changes or remains constant across time. Future studies should be conducted to examine the changes in school collective efficacy across time.

Implications for Future Research

These findings have methodological implications for researchers studying issues relative to school collective efficacy to prevent bullying, as well as practitioners seeking to hinder bullying in schools that rely, in part, on the extent to which they can promote cohesion, students’ social control, and teachers’ social control. Given the values of the construct of school collective efficacy in bullying research, it is critical that studies of school collective efficacy use the best measurement instruments possible to ensure usable inferences. Future studies on the conceptualization and assessment of students’ perception of school collective efficacy to bullying prevention should be enhanced by the theoretical argument and empirical results of this study. Additionally, the measurement invariance of factor structure is relevant for practitioners who implement whole-school bullying prevention programs who might want to assess whether interventions have different effects on different groups within their school. Finally, the correlation of collective efficacy with positive whole-school variables implies that further studies should deepen the relationships between these variables.