A Cross-cultural Study of the Validity of a Battery of Questionnaires for Assessing School Climate Quality

Introduction

The main objective of any school is to ensure learning and development of students. In this complex process, the school climate created through the relationships between members of an educational community is essential. Empirical research has shown that norms, expectations, and belief systems that characterize school climate shape individual experience and student learning and, thus, influence their behavior and relationships (Thapa et al., 2013). Actually, school climate has been shown to be one of the main determinants of student academic achievement and school climate reform supports effective violence prevention (Gendron et al., 2011; Wang & Eccles, 2013). Because of this fact, and of the potential malleability of school climate, a great deal of school reform initiatives focus on improving such a climate (Durlak et al., 2011).

School climate represents an inherently multidimensional construct for which there is no universal definition (Anderson, 1982; Wang & Degol, 2015). The National School Climate Council (2007) defines school climate as “patterns of people’s experiences of school life that reflects norms, goals, values, interpersonal relationships, teaching and learning practices, and organizational structures” (p. 4). However, very different definitions have been used to describe it, which have led to a lack of theoretical coherence in many school climate measures and a lack of consensus regarding dimensions that should be regularly measured (Clifford et al., 2012; Thapa et al., 2013). In addition, Cohen et al. (2009) pointed out that schools often use homemade instruments to assess school climate. The importance of using psychometrically sound measures lies in the need to assess school climate in a valid and reliable way in order to make data-driven decisions, which allows optimizing schools resources such as effort, time, and money. Therefore, efforts to progress in the measurement of school climate and to develop reliable and useful instruments for schools are still needed.

Over the last years, different measures have been developed to assess school climate, most of them based on student perceptions. However, recent studies suggest that student perceptions of school climate may actually be based on what occurs in smaller ecologies, such as classroom, where students spend most of their time (Mitchell & Bradshaw, 2013). In addition, certain aspects that school climate encompasses (e.g., leadership or quality of relationships among teachers) would not be so evident or relevant from student perspectives because of their role within school (Koth et al., 2008). In this context, teacher perspectives become relevant since they perceive and are sensitive to school climate, but also play a critical role in facilitating a supportive classroom climate for students (Künsting et al., 2016; Reeve, 2006). Therefore, it is worth asking which are the aspects of school climate that may condition the work and efficacy of teachers.

Prior research has shown that a leadership in which responsibilities are shared, participation in school decisions is encouraged, and opportunities to improve teaching are provided is related to a high-quality pedagogy among teachers (Sun & Leithwood, 2015). Similarly, support received from other colleagues is essential to develop an identity as a resilient teacher and influences teacher commitment and retention (Johnson et al., 2015). Teachers’ perceptions of students have been shown to influence their effort and the teaching strategies they use (Wenglinsky, 2000) and, often, student behavior patterns influence teacher attrition and frustration (Robers et al., 2012). In addition, teachers’ perceptions of parental involvement are related to sense of self-efficacy and expectations teachers hold about students (Hauser-Cram et al., 2003).

Another aspect that has received less attention, but significantly related to a teacher’s efficacy, is school goal structure. According to the achievement goal theory, there are two types of goal orientations: a mastery or learning goal structure and a performance goal structure (Midgley et al., 1995; Skaalvik & Skaalvik, 2011). The former defines an environment in which success is based on learning and improvement, student effort is recognized, and mistakes are considered a part of the learning process. In contrast, a performance goal structure defines an environment in which success is based on social comparison, achievement of high marks is emphasized, and competition is often encouraged. Research has shown that when students perceive a learning-goal structure they tend to pursue these types of goals and tend to use more effective learning strategies, while students who perceive a performance-goal structure tend to use surface-level cognitive strategies (Ames & Archer, 1988; Midgley et al., 1995). More recent studies have shown that when teachers perceive that school emphasizes the importance of learning they also tend to adopt teaching goals oriented towards student learning (Cho & Shim, 2013). In addition, Skaalvik and Skaalvik (2011) found that teachers feel a greater sense of school belonging and greater job satisfaction when a learning-goal structure is emphasized.

In an effort to integrate the above aspects, the School Climate Questionnaire for Secondary and High School Teachers (Quijada et al., 2020) was developed. The initial study supported a seven-factor structure assessing the quality of leadership, quality of relationships among teachers, learning-goal structure, performance-goal structure, teacher-student relationships, student-student relationships, and parental involvement. Measures already exist that provide an inclusive perspective of school climate by evaluating leadership, quality of relationships among students, appearance and resources of school, discipline environment, faculty relationships, and parental involvement (e.g., School Climate Assessment Instrument by Alliance for the Study of School Climate, 2004; School Climate Teacher Survey by Liu et al., 2014). Other existing measures provide a school climate perspective useful to evaluate bullying prevention program effects by assessing rules and safety at school as well as the existence of respectful relationships between teacher and students (e.g., Delaware School Climate Survey-Teacher/Staff by Bear et al., 2014; Authoritative School Climate Survey by Huang and Cornell (2015). However, from a perspective focused on school climate aspects that can improve student learning, it is essential to take into account school goal structure while other aspects, such as safety and school resources, become less relevant (Wang & Degol, 2015). In this regard, SCQ-SHST constitutes a school climate measure especially useful for schools interested in articulating interventions aimed at improving learning environment. The initial validation of SCQ-SHST provided evidence of its validity and reliability to be used with Spanish teachers at secondary and high schools.

In a second study conducted as part of a project for the Ministry of Public Education of Costa Rica (Alonso-Tapia, 2017), the proper functioning of the SCQ-SHST under a different factor structure from the initially proposed was proved. In this case, dimensions of SCQ-SHST were considered as independent questionnaires, and the underlying factor structure of each of them was explored. Learning-goal structure and performance-goal structure were considered as a single questionnaire with two correlated factors. The dimensions referred to teacher-student and student-student relationships were also considered as a single questionnaire with three first-order factors and a second-order factor labeled quality of student attitudes. Quality of leadership was considered as a single questionnaire, as well as quality of relationships among teachers, and parental involvement. The first two questionnaires model included a second-order factor, while the parental involvement questionnaire was unidimensional.

Although the seven-factor structure initially proposed proved useful to assess teacher perceptions of school climate, factor structures tested in Costa Rica context offer two advantages. First, the joint use of these questionnaires provides schools with a comprehensive perspective of school climate through the scores obtained in each one of them. In addition, schools may decide to use only some of these questionnaires and/or supplement their use with other scales that they consider of interest according to school needs. Second, and more important, multilevel factor structure underlying those questionnaires, including first- and second-order factors allows to obtain scores not only on the general construct, but also on specific aspects of each construct assessed. Therefore, it provides a more detailed information on school strengths and on those aspects susceptible to improvement, which is especially useful to guide an intervention.

This analytic strategy proved useful in assessing perceptions of Costa Rican teachers in a valid and reliable way as well as in designing future interventions for schools involved. However, factor structures proposed were not tested in the Spanish context, so there may be significant differences across both cultures. Although we did not find studies comparing teachers’ perceptions of school climate cross-culturally, there is some evidence suggesting that school climate perceptions vary according to gender, race, ethnicity, and culture. For example, in the context of North American schools Black and Hispanic students were found to perceive school climate less favorably and reported lower levels of security and connectedness to school than their White peers (Voight & Nation 2015). Yang et al. (2013) conducted a cross-cultural study among Chinese and American students. Researchers found that the latter tended to perceive school climate less favorably while Chinese students attributed more importance to academic achievement, respect for teachers, and social harmony. Likewise, significant differences have been found in the importance attributed by Arab and Jewish schools to certain values (Daniel et al., 2013). These findings suggest the importance of considering possible differences in school climate perceptions and justify the need to continue exploring such discrepancies, especially among cultures. Therefore, the present study had two objectives: 1) to test the fit of Spanish data to the models validated in Costa Rica and 2) to test whether there are differences across the two countries and to deduce implications for measurement and intervention if such differences are present.

Method

Participants

A total of 348 secondary and high school teachers from 92 schools in Spain participated in this study. After removing 5 participants with missing data, the final sample was composed of 343 respondents (54% female and 46% male). Age ranged from 25 to 65 years (mean = 45,1, SD = 9,1), and teaching experience from 1 to 46 years (mean = 17, SD = 10). Per subject, 13% of teachers taught Spanish language and literature, 13% mathematics, 11.5% social sciences, 6.2% natural sciences, 4.3% physics or chemistry, 17.6% foreign languages, 3.4% art, 1.2% religion, and 29.8% other subjects.

The Costa Rican sample was composed of 185 secondary and high school teachers from 14 schools. After removing 7 participants with missing data, the final sample was composed of 178 participants. Females were 53% and males 47%. Age ranged from 21 to 64 years (mean = 37.7, SD = 0.3), and teaching experience from 1 to 36 (mean = 10.9, SD = 6.6). Per subject, 11.8% teachers taught Spanish language and literature, 10.1% mathematics, 11.2% social sciences, 6.2% natural sciences, 7.9% physics or chemistry, 19.7% foreign languages, 2.8% art, 1.7% religion, and 27% other subjects.

Procedure

Spanish teachers participated voluntarily after their schools received an informative letter explaining the purpose of the study. Since research has shown that paper-and-pencil and Internet data collection methods are generally equivalent (Weigold et al., 2013), questionnaires were provided in both formats to make them more accessible. Data were collected from October to February 2015. The Research Ethics Committee of the Universidad Autónoma of Madrid granted approval for the present study.

Costa Rica data were collected in July 2017, as part of a project carried out for the Ministry of Public Education of Costa Rica (Alonso-Tapia, 2017). The purpose of this project was to assess teachers’ perceptions of school climate and to provide advice to improve teaching quality. Teachers completed the questionnaires in paper-and-pencil format.

Instruments

Leadership Quality Questionnaire (LQ-Q). It includes 24 items, measuring 4 aspects related to Communication and Support to Teachers: 1) democratic leadership, 2) clear and constant communication, 3) support to demands of the teaching staff, 4) support to teachers facing difficulties with families/students; and 5 aspects related to Management and Values of the School: 1) orientation towards learning versus performance, 2) planned management, 3) interest in obtaining resources for the school, 4) cooperative work among teachers stimulation, and 5) positive coexistence climate fostering. This questionnaire showed satisfactory psychometric properties in the study conducted with Costa Rica’s sample: ω = .91 for the first subscale; ω = .79 for the second subscale; and ω = .99 for the global scale.

Teachers’ Quality of Support Questionnaire (TQS-Q). It includes 12 items, measuring 2 aspects related to Desire to Improve as a Teacher: 1) showing a continuous effort in the daily work, 2) proposing new initiatives; 2 aspects related to Work Support among coworkers: 1) helping other teachers, 2) working cooperatively; and 2 aspects related to Emotional Support: 1) comprehension versus criticism, and 2) help with worries. This questionnaire also showed adequate psychometric properties: ω = .83 for the first subscale; ω = .75 for the second subscale; ω = .66 for the third subscale; and ω = .96 for the global scale.

School Motivational Orientation Questionnaire (SMO-Q). It includes 8 items, 4 measuring School Motivational Orientation towards Learning (e.g., “at my school, the emphasis is placed on students really understanding, not just memorizing”). On the contrary, the remaining 4 items measured School Motivational Orientation towards Performance (e.g., “at my school students with higher grades are congratulated publicly to motivate the rest of the students”). This questionnaire showed good reliability: ω = .88 for the learning motivation orientation scale, and ω = .93 for the performance motivation orientation scale.

Attitudinal Quality of Students Questionnaire (AQS-Q). It includes 14 items, measuring 2 aspects related to the teachers’ perceptions of Students Confidence at the classroom: 1) confidence to communicate with the teacher, 2) assertiveness when teacher proposes tasks; 3 aspects related to Cooperation in the classroom: 1) cooperation with teacher, 2) cooperation among classmates, 3) classroom discipline; and 2 aspects related to Students Motivation: 1) interest in the subject, and 2) effort to achieve learning goals. The reliability coefficient was .81 for the first subscale, .79 for the second subscale, .78 for the third subscale, and .94 for the global scale.

Quality of Parental Support Questionnaire (QPS-Q). It includes 8 items, with 2 items measuring aspects related to Cooperation with the Teacher (e.g., most of the families respond positively to my indications and suggestions); 2 items measuring aspects related to Contribution from Parents to Students Trusting the Teacher (e.g., “most of the families of the majority contribute to their children trusting me”); 2 items measuring aspects related to Respect towards Teacher (e.g., “most of the families try that the students value positively what I tell them”); and the 2 remaining items assessing the Role of Parents in the Interest and Effort of Students to Learn (e.g., “if my students are interested and strive to learn it is thanks to the cooperation of their families”). The reliability coefficient was .82.

Teachers Satisfaction Scale (TSS). To obtain additional information on the validity of these questionnaires, a scale consisting of 14 items was used as a criterion measure. This scale measures 5 aspects related to Satisfaction with the School and Coworkers: 1) satisfaction with the work of the management team, 2) satisfaction with the teaching-learning system, 3) satisfaction with the relationships among teachers, 4) satisfaction with school rules, 5) satisfaction with the coexistence and security climate of the school; and 2 aspects related to the Satisfaction with the Students and their Families: 1) satisfaction with the learning and recognition shown by the students, and 2) satisfaction with the support received from parents. The reliability coefficient was .91 for the first subscale, .80 for the second subscale, and .73 for the global scale.

Statements were balanced to avoid acquiescence so that there was the same number of positively and negatively worded items in each questionnaire. Items were gauged using five-point Likert-type scales ranging from 1 (strongly disagree) to 5 (strongly agree).

Data Analysis

Firstly, several confirmatory factor analysis (CFA) were performed using Amos 24 (Arbuckle, 2016) to test the fit of the Spanish data to the models proposed in the study carried out in Costa Rica. Factor structures described for each questionnaire were used as baseline models. Therefore, in the first CFA, LQ-Q factor structure was tested, in the second CFA, TQS-Q factor structor was tested, and so on. Item parcels were used since this procedure has known advantages such as fewer parameters to estimate and fewer chances for residuals to be correlated or dual loadings to emerge (MacCallum et al., 1999). Therefore, this procedure allows us to specify a clear latent construct (Little et al., 2002), which provides a simpler and potentially more useful interpretation. Maximum likelihood was used to estimate the proposed models since this estimation procedure is reasonably robust within the context of structural equations even if multivariate normality is not fulfilled (West, Finch, & Curran, 1995). In addition, according to Rodríguez and Ruiz (2008), exploratory and confirmatory factor analysis results will not be distorted by this fact if Mardia’s coefficient is below 70, as is the case of the present study since Mardia’s coefficient reached values between 4.20 and 25.75. Model fit was assessed using comparative fit index (CFI), Tucker-Lewis index (TLI), root mean square error of approximation (RMSEA), and standardized root mean square residual (SRMR). CFI and TLI values ≥ .90 indicate acceptable fit, while values β .95 indicate a good fit (Hu & Bentler, 1999). RMSEA and SRMR values between .05 and .08 represent an acceptable fit, whereas values lower than .05 indicate good fit (McDonald & Ho, 2002). The criteria suggested by Hair et al. (2010) according to which χ²/df ≤ 5 indicates a good model fit was also considered. Additionally, the internal reliability of each questionnaire for the Spanish sample was determined through omega coefficient (McDonald, 1999).

Secondly, multi-group confirmatory factor analyses (MG-CFA) were performed to cross-validate the results. Models described for each questionnaire were used as baseline models for comparison across the two samples (i.e., Spanish and Costa Rica sample) without any parameter constraints. Against the unconstrained model tested for each questionnaire, several models, in which equality between groups was imposed on different sets of parameters, were tested and compared. The relative decline in goodness-of-fit was assessed by means of the difference in chi-square statistic between the model with restrictions imposed and the model without restrictions. In case of significant decline in goodness of fit, it was decided to analyse the reasons for such decline, testing which differences between regression weights were significant using the z-test by Clogg et al. (1995).

Thirdly, in order to determine whether aspects of school climate evaluated by the questionnaires were positively and significantly related to the point of being considered as indicators of the same construct (i.e., school climate), a second-order model with six specific factors (i.e., leadership, support among teachers, learning motivation orientation, performance motivation orientation, student attitudes, and parental support) was tested. Additionally, the discriminant validity was evaluated following the indications proposed by Hair et al. (2014) with the objective of exploring whether these six components of school climate are sufficiently different in terms of their empirical standards. According to these authors, when the average variance extracted by each dimension is greater than the squared correlation between these two dimensions, discriminant validity is ensured.

Fourthly, scales’ reliability was analyzed using McDonald’s (1999) ω index.

Fifthly, to obtain additional information on the concurrent validity of these questionnaires and to examine to what extent each of the aspects of the school climate assessed was related to teacher satisfaction, correlation and regression analysis were performed.

Results

Confirmatory Factor Analysis and Cross-validation Analyses

Figure 1 shows standardized factor loadings for confirmatory models tested with the sample of Spanish teachers. All factor loadings were significant (p < .001) and had a medium-high size. Table 1 shows fit statistics for each baseline model. For SMO-Q (CFA-5) and TSS (CFA-11), χ²/df ratios were > 5 probably due to sample size (Hair et al., 2010), but the remaining fit indexes were well inside the limits that allowed the models to be accepted. The rest of the models showed satisfactory goodness-of-fit indexes according to recommended cutoff values.

Results of the cross-validation analysis using the Spanish and Costa Rican teacher samples (n = 343, 178) are displayed in Table 1. As can be seen, except for the QPS-Q (CFA-10), chi-square values were significant, again probably due to sample size, but χ²/df ratios as well as the rest of fit statistics were satisfactory enough to accept these models.

Multi-group Confirmatory Factor Analyses

In order to test the validity of these models across both countries, a multi-group analysis was performed for each questionnaire. Comparison statistics are shown in Table 2. In the case of quality of teachers support (CFA-4), quality of student attitudes (CFA-8), and teachers’ satisfaction (CFA-12), model fit is not reduced even when equality restrictions across samples were imposed, except for restrictions referred to residuals. It is widely accepted that testing for residual equivalence represents an overly restrictive criteria and does not compromise conclusions drawn from this analysis (Byrne, 2010), so this finding was expected.

In the case of questionnaires related to quality of leadership (CFA-2), school motivational orientation (CFA-6), and parental support (CFA-10), fit was significantly reduced when restrictions across samples were imposed. This means that perceptions of Spanish and Costa Rican teachers differ in some aspects of the school climate evaluated by these questionnaires. To know which aspects were valued differently by teachers of each group, regression weights were compared using z-statistic of Clogg et al. (1995). Results of this analysis for the leadership quality questionnaire are shown in Table 3. As can be seen, there were significant differences (z > 1.96) in the weights of three aspects. On one side, first, the relationship between the degree to which “management team supports teachers’ demands” and “communication and support” – a dimension of the quality of leadership – is greater for Spanish teachers (difference: 0.13, z = 2.44). Besides, the relationship between “management and school values” – the second dimension of quality of leadership – and “quality of leadership” is greater also for Spanish Teachers (difference: 0.27, z = 4.25). On the other side, the relationship between “stimulation of cooperative work among teachers” and “management and school values” is greater for Costa Rican teachers (difference: -1.13, z = -2.77).

Table 4 shows results for the school motivational orientation questionnaire. In this case, Costa Rican teachers valued three of the aspects related to such motivational orientation to a greater degree than Spanish teachers. These aspects have to do with perception of mistakes as part of the learning process (difference: -0.28, z = -3.55), with the use of public congratulations to those students who obtain the highest grades (difference: -0.67, z = -3.20), and with the importance of supporting students with better academic attitudes (difference: -0.87, z = -3.62).

Finally, with regard to parental support, Costa Rican teachers value the interest of parents for their children’s learning to a greater extent (difference: -0.20, z = -4.34), while Spanish teachers value more respect for the work of the teacher (difference: 0.13, z = 2.70), and parents’ collaboration in the confidence of students towards teacher (difference: 0.20, z = 3.55). These results are shown in Table 5.

Evidence of the Second-Order Model Structural Validity

The model fit was satisfactory for the sample of Spanish teachers (χ²/df = 3.54 < 5; CFI = .93 >.90; TLI = .92 >.90; RMSEA = .06 < .08; SRMR = .07 < .08). In the cross-validation analysis, the χ² statistic was significant, but again χ²/df ratio and the remaining fit indices were within acceptable limits (χ²/df = 2.69 < 5; CFI = .92 >.90; TLI = .91 >.90; RMSEA = .05 < .08; SRMR = .07 < .08). As can be seen in Figure 2, there are differences in both factor loadings of school climate components and across samples. In both samples, quality of leadership, support among teachers, and learning motivational orientation present higher factor loadings with respect to the three remaining components. Likewise, there are differences across countries in the way of perceiving performance motivational orientation. Costa Rican teachers consider that performance motivational orientation contributes positively to school climate, while the opposite happens in the case of Spanish teachers.

Although model fit was good, reliability in both samples was low (ω < .30). Therefore, additional models were tested to examine whether reliability varies according to the number of school climate components included. When an alternative model with three lower-order factors (leadership, support among teachers, and motivational learning orientation) and the same higher-order factor (school climate) was tested, reliability was good (.80 for the Spanish sample, and .83 for the Costa Rican sample) and results of the cross-validation analysis were also satisfactory (CFI = .97 >.90; TLI = .95 >.90; RMSEA = .08 ≤ .08; SRMR = .03 < .08). However, when this same model was tested by adding a fourth factor of a lower-order (performance motivational orientation), reliability decreased again (.61 for the Spanish sample and .58 for the Costa Rican sample) even though results of the cross-validation analysis were satisfactory (CFI = .95 >.90; TLI = .93 >.90; RMSEA = .08 ≤ .08; SRMR = .05 < .08). The same occurs when this lower-order fourth factor is replaced by attitudes of students (.57 for the Spanish sample, and .60 for the Costa Rican sample), or by parental support (.57 for the Spanish sample, and .58 for the Costa Rican sample).

Table 6 shows the average variance extracted for each of the six components of school climate and squared correlations between them. As can be seen, the average variance extracted was greater than squared correlations, which means that components are sufficiently different from each other and discriminant validity is established.

Reliability

In Figure 1, McDonald’s ω coefficients for the scales of each questionnaire have been placed above latent factors. They are between .83 and .99, what implies a good internal consistency.

Correlation and Regression Analysis

Correlations between the six school climate dimensions and the two teacher satisfaction dimensions are shown in Table 7. All the dimensions were related in the expected way and the correlations obtained for both samples were similar, except for performance motivational orientation, which was negatively related to teacher satisfaction in the Spanish sample but was positively related to satisfaction in the Costa Rican sample.

Figure 3 shows the relative weight of each school climate dimension in predicting teachers’ satisfaction. Model fit was good for both the sample of Spanish teachers (χ²/df = 2.93 < 5; CFI = .95 >.90; TLI = .94 >.90; RMSEA = .06 <.08; SRMR = .05 <.08) and Costa Rican teachers (χ²/df = 1.65 < 5; CFI = .92 >.90; TLI = .90 = .90; RMSEA = .06 <.08; SRMR = .06 <.08). In both cases, leadership was the aspect of school climate that contributes the most to explain teacher satisfaction with school (β = .65, β = .45), while student attitudes were the aspect that contributes most to explain teacher satisfaction with students and families (β = .50, β = .61). Overall, dimensions of school climate explained 79% of teacher satisfaction variance related to school for the Spanish sample and 70% for the Costa Rican sample, and 54% of satisfaction related to students and families for the Spanish sample and 44% for the Costa Rican sample.

Discussion

The aim of this study was to explore whether there are differences in teacher perceptions of school climate across different cultural contexts as well as theoretical and practical implications. A secondary purpose was to provide evidence on a different factor structure than that initially proposed for the SCQ-SHST in support of its use among schools. Results of this study are discussed in light of these two objectives.

First of all, the results based on the different CFAs indicate that questionnaires have a well-estimated factor structure and are valid in the context of Costa Rica and Spain. This means that, to the extent that schools promote and emphasize the set of aspects measured and evaluated by these questionnaires, they will be able to create a positive school climate for most teachers. Likewise, results obtained supported the use of SCQ-SHST as modified in this study. The individual use of each questionnaire provides schools with detailed information on different components of the school climate assessed, which makes it possible to detect those aspects to be modified. In addition, to the extent that these questionnaires are used together, they provide an overview of the school climate perceived through the facets evaluated. This information will be useful to detect the strengths of a school and potential areas of intervention as well as the impact of intervention programs, especially of those aimed at improving learning environment.

Second, results of this study support and extend findings of previous studies (e.g., Yang et al., 2013) showing that differences in school climate perceptions are present when comparing teachers from different cultural contexts. A priori, there is not a strong cultural contrast between the two countries included in this study since, for example, both share the same language. However, our results point out that there are significant differences in the way teachers perceive school climate as well as in the value they attribute to some aspects of it. For example, when it comes to leadership quality, Costa Rican teachers consider more important aspects related to school management (i.e., encouraging cooperative work among teachers). In contrast, Spanish teachers value the fact that the school’s management team involves the teaching staff in decision-making and considers their opinions and needs. In Spain, teachers who are part of a school can decide who will play the principal’s role, while in Costa Rica teachers do not have the opportunity to decide on this school’s aspect. Therefore, often, in Spanish schools the principal is a teacher who has been part of the teaching staff and perhaps, for this reason, teachers consider that it is more important that their demands and opinions are heard by him/her. However, this is only a possible explanation for these findings and further research is needed.

In the case of school, significant differences across countries were found in motivational orientation and parental involvement. Considering mistakes as a natural part of the learning process, praising students who achieve higher grades, and encouraging smarter students is valued more positively in Costa Rica than in Spain. Likewise, the interest and effort of parents in their children’s learning is valued more positively in Costa Rica, while Spanish teachers value to a greater extent that parents encourage respect for the teacher and student discipline. Consequently, it is convenient to take into account these differences when conducting cross-cultural studies or when promoting actions aimed at creating a positive school climate for teachers. However, it is interesting to reflect on the possible consequences of emphasizing the importance of achieving high marks or praising the smartest students to be a role model for the rest of the students. Cho and Shim (2013) found that some teachers with a high sense of efficacy consider that a performance motivational orientation is also important and, thus, constitute a desirable goal for teaching. However, the existing literature has consistently reported the benefits of a learning-oriented climate versus a performance-oriented climate. Research has pointed out that when teachers create a learning-oriented classroom climate, students tend to pursue these kinds of goal and use more effective learning strategies such as discriminating relevant information from the one that is not, trying to integrate the new information with the one they know, being more persistent and creative, and taking on more challenging tasks. Conversely, students who perceive a performance-oriented climate tend to endorse this goal and use surface-level strategies such as rereading the text, memorizing, guessing and show cheating behaviors (Ames & Archer, 1988). Likewise, Skaalvik, and Skaalvik (2017) found that teachers’ perception of a performance-goal structure is related to the experience of greater emotional exhaustion and motivation to leave the teaching profession, and a lower sense of self-efficacy and job satisfaction. Therefore, it is convenient to reflect on the type of goals that are considered desirable and that are emphasized at school, especially when trying to improve the conditions that favor student learning.

Third, the results of this study suggest that not all components of school climate are part of it in the same degree or in the same way. The results of each questionnaire taken individually indicate that they reliably evaluate each of the six components of school climate. However, the reliability of all of them taken together as part of the same questionnaire is not adequate, probably due to the degree to which they contribute to school climate. Leadership quality, support among teachers, and learning motivational orientation represent aspects directly related to the teaching staff. Therefore, it makes sense that these three components considered together show adequate reliability.

When it comes to performance motivational orientation, there are important differences across countries, since Spanish teachers consider that this aspect negatively defines school climate, while Costa Rican teachers consider that it is an aspect that contributes positively to this climate. In addition, this component showed the lowest factor loading on the higher-order factor referred to school climate. Thus, it makes sense that the reliability of the questionnaire decreased when this aspect was included.

Student attitudes were referred to the behavior of students both with teachers and with their classmates. Therefore, although these are aspects on which school and teachers exercise an active role, they are also aspects that do not depend totally on them, which could explain why reliability of the model decreased when this component was included.

Finally, parental involvement can be very variable due to multiple factors that do not depend on school itself such as sociocultural level or working conditions (Eccles & Harold, 1996). Therefore, although it is a relevant aspect of school climate, it often constitutes an aspect external to school.

These results reflect school climate’s complex reality, since there are multiple aspects that compose it and there are differences in terms of level of involvement in school (e.g., parental involvement) and the responsibility that school has over them (e.g., student attitudes). This fact may have important implications in the effectiveness of interventions carried out by school. Those aspects on which school has a more direct control (e.g., quality of leadership or support among teachers) may be more easily malleable and susceptible of improvement compared to those aspects that do not depend directly on school or teachers (e.g., student attitudes or parental involvement). Therefore, these findings should be considered by schools when assessing school climate and deciding on what aspects are worth investing more time and effort.

Fourth, the results of correlation analyses showed that, in both samples, leadership was the aspect most strongly related to satisfaction with school. Again, differences were found in the way performance motivational orientation was related to satisfaction as a function of the country. In line with results discussed above, it was found that Spanish teachers consider that such motivational orientation is negatively related to their satisfaction with school, while for Costa Rican teachers performance motivational orientation contributes positively to their satisfaction both with school and with students and families. The results of the regression analysis also supported the conclusion that leadership is the aspect that most contributes to explaining satisfaction of both Spanish and Costa Rican teachers with school. This finding is consistent with previous studies that show the relationship of leadership not only with satisfaction of teachers, but also with the capacity for innovation, commitment of teachers to implement interventions adopted by the school, and motivation of teachers (Pietsch & Tulowitzki, 2017; Sun & Leithwood, 2015). In addition, because principals control important school-wide conditions, it seems appropriate to consider quality of leadership when evaluating and planning an intervention focused on school climate improvement, especially when it comes to foster motivation and commitment of teachers and, ultimately, to improve student learning.

Finally, the results and conclusions presented should be interpreted considering the limitations of the current study. In the first place, the sample of teachers from Costa Rica is limited by its size, which makes it difficult to generalize the results obtained. In addition, the difference in sample size between the two countries can decrease the power to detect measurement equivalence across samples (Chen, 2007). Second, the language of the questionnaires was adapted to the Spanish spoken in Latin America before being administered. However, the questionnaires were initially developed in the Spanish context, so it is possible that they do not capture aspects of school climate more specific to Latino culture.