psedPsicología EducativaPsicología Educativa1135-755X2174-0526Colegio Oficial de la Psicología de Madrid10.5093/psed2020a200003research-articleA Cross-cultural Study of the Validity of a Battery of Questionnaires for Assessing School Climate QualityEl perfil de clima escolar: estudio transcultural de la validez de una batería de cuestionarios para evaluar el clima escolarAlonso-TapiaJesúsaQuijadaAnaaRuizMiguelaHuertasJuan A.aUlateMaría A.bBiehlMaría L.cUniversidad Autónoma de MadridSpainUniversidad Autónoma de Madrid, SpainMinisterio de Educación PúblicaCosta RicaMinisterio de Educación Pública, Costa RicaInter-American Development BankUnited StatesInter-American Development Bank - Education Division
Conflict of Interest
The authors of this article declare no conflict of interest.
Correspondence: jesus.alonso@uam.es (J. Alonso-Tapia).145202052020262109119111120191312201921022020This is an Open Access article distributed under the terms of the Creative Commons Attribution-Noncommercial No Derivative License, which permits unrestricted non-commercial use, distribution, and reproduction in any medium provided the original work is properly cited and the work is not changed in any way.ABSTRACT
This paper studies the validity of the School Climate Battery of Questionnaires for Secondary and High School Teachers (SCBQ-SHST). The battery includes five questionnaires: Quality of Leadership, Quality of Teachers’ Support, School Motivational Orientation, Quality of Students’ Attitude, and Quality of Parental Support. A total of 178 teachers from Costa Rica were compared with 343 Spanish teachers. Confirmatory factor analyses showed that the questionnaires allow assessing teachers’ perceptions of the different dimensions of school climate in a valid and reliable way in both Spain and Costa Rica. Differences in the perception of Spanish and Costa Rican teachers about school climate shown by multigroup confirmatory factor analyses are discussed as well as theoretical and practical implications.
RESUMEN
Este artículo estudia la validez de la Batería de cuestionarios sobre clima escolar para profesores de Secundaria y Bachillerato (BQCE-SB). La batería incluye cinco cuestionarios: calidad de liderazgo, calidad de apoyo docente, orientación motivacional del centro, calidad de las actitudes de los estudiantes y calidad del apoyo de los padres. Un total de 178 docentes de Costa Rica se compararon con 343 docentes españoles. Los análisis factoriales confirmatorios mostraron que los cuestionarios permiten evaluar de manera válida y confiable las percepciones de los docentes sobre las diferentes dimensiones del clima escolar, tanto en España como en Costa Rica. Se discuten las diferencias en la percepción de los docentes españoles y costarricenses sobre el clima escolar que muestran los análisis factoriales confirmatorios multigrupo, así como sus implicaciones teóricas y prácticas.
KeywordsSchool climateTeachers’ perceptionsCross-cultural validitySchool climate assessmentSchool leadershipPalabras claveClima escolarPercepción de los profesoresValidez transculturalEvaluación del clima escolarLiderazgo escolarSpanish Ministry of Economy and CompetitivenessEDU2017-89036-PThis study was carried out with funding from the Spanish Ministry of Economy and Competitiveness (Project EDU2017-89036-P), from the Banco Interamericano de Desarrollo, and from the Ministerio de Educación Pública of Costa Rica.Introduction
The main objective of any school is to ensure learning and development of students. In this complex process, the school climate created through the relationships between members of an educational community is essential. Empirical research has shown that norms, expectations, and belief systems that characterize school climate shape individual experience and student learning and, thus, influence their behavior and relationships (Thapa et al., 2013). Actually, school climate has been shown to be one of the main determinants of student academic achievement and school climate reform supports effective violence prevention (Gendron et al., 2011; Wang & Eccles, 2013). Because of this fact, and of the potential malleability of school climate, a great deal of school reform initiatives focus on improving such a climate (Durlak et al., 2011).
School climate represents an inherently multidimensional construct for which there is no universal definition (Anderson, 1982; Wang & Degol, 2015). The National School Climate Council (2007) defines school climate as “patterns of people’s experiences of school life that reflects norms, goals, values, interpersonal relationships, teaching and learning practices, and organizational structures” (p. 4). However, very different definitions have been used to describe it, which have led to a lack of theoretical coherence in many school climate measures and a lack of consensus regarding dimensions that should be regularly measured (Clifford et al., 2012; Thapa et al., 2013). In addition, Cohen et al. (2009) pointed out that schools often use homemade instruments to assess school climate. The importance of using psychometrically sound measures lies in the need to assess school climate in a valid and reliable way in order to make data-driven decisions, which allows optimizing schools resources such as effort, time, and money. Therefore, efforts to progress in the measurement of school climate and to develop reliable and useful instruments for schools are still needed.
Over the last years, different measures have been developed to assess school climate, most of them based on student perceptions. However, recent studies suggest that student perceptions of school climate may actually be based on what occurs in smaller ecologies, such as classroom, where students spend most of their time (Mitchell & Bradshaw, 2013). In addition, certain aspects that school climate encompasses (e.g., leadership or quality of relationships among teachers) would not be so evident or relevant from student perspectives because of their role within school (Koth et al., 2008). In this context, teacher perspectives become relevant since they perceive and are sensitive to school climate, but also play a critical role in facilitating a supportive classroom climate for students (Künsting et al., 2016; Reeve, 2006). Therefore, it is worth asking which are the aspects of school climate that may condition the work and efficacy of teachers.
Prior research has shown that a leadership in which responsibilities are shared, participation in school decisions is encouraged, and opportunities to improve teaching are provided is related to a high-quality pedagogy among teachers (Sun & Leithwood, 2015). Similarly, support received from other colleagues is essential to develop an identity as a resilient teacher and influences teacher commitment and retention (Johnson et al., 2015). Teachers’ perceptions of students have been shown to influence their effort and the teaching strategies they use (Wenglinsky, 2000) and, often, student behavior patterns influence teacher attrition and frustration (Robers et al., 2012). In addition, teachers’ perceptions of parental involvement are related to sense of self-efficacy and expectations teachers hold about students (Hauser-Cram et al., 2003).
Another aspect that has received less attention, but significantly related to a teacher’s efficacy, is school goal structure. According to the achievement goal theory, there are two types of goal orientations: a mastery or learning goal structure and a performance goal structure (Midgley et al., 1995; Skaalvik & Skaalvik, 2011). The former defines an environment in which success is based on learning and improvement, student effort is recognized, and mistakes are considered a part of the learning process. In contrast, a performance goal structure defines an environment in which success is based on social comparison, achievement of high marks is emphasized, and competition is often encouraged. Research has shown that when students perceive a learning-goal structure they tend to pursue these types of goals and tend to use more effective learning strategies, while students who perceive a performance-goal structure tend to use surface-level cognitive strategies (Ames & Archer, 1988; Midgley et al., 1995). More recent studies have shown that when teachers perceive that school emphasizes the importance of learning they also tend to adopt teaching goals oriented towards student learning (Cho & Shim, 2013). In addition, Skaalvik and Skaalvik (2011) found that teachers feel a greater sense of school belonging and greater job satisfaction when a learning-goal structure is emphasized.
In an effort to integrate the above aspects, the School Climate Questionnaire for Secondary and High School Teachers (Quijada et al., 2020) was developed. The initial study supported a seven-factor structure assessing the quality of leadership, quality of relationships among teachers, learning-goal structure, performance-goal structure, teacher-student relationships, student-student relationships, and parental involvement. Measures already exist that provide an inclusive perspective of school climate by evaluating leadership, quality of relationships among students, appearance and resources of school, discipline environment, faculty relationships, and parental involvement (e.g., School Climate Assessment Instrument by Alliance for the Study of School Climate, 2004; School Climate Teacher Survey by Liu et al., 2014). Other existing measures provide a school climate perspective useful to evaluate bullying prevention program effects by assessing rules and safety at school as well as the existence of respectful relationships between teacher and students (e.g., Delaware School Climate Survey-Teacher/Staff by Bear et al., 2014; Authoritative School Climate Survey by Huang and Cornell (2015). However, from a perspective focused on school climate aspects that can improve student learning, it is essential to take into account school goal structure while other aspects, such as safety and school resources, become less relevant (Wang & Degol, 2015). In this regard, SCQ-SHST constitutes a school climate measure especially useful for schools interested in articulating interventions aimed at improving learning environment. The initial validation of SCQ-SHST provided evidence of its validity and reliability to be used with Spanish teachers at secondary and high schools.
In a second study conducted as part of a project for the Ministry of Public Education of Costa Rica (Alonso-Tapia, 2017), the proper functioning of the SCQ-SHST under a different factor structure from the initially proposed was proved. In this case, dimensions of SCQ-SHST were considered as independent questionnaires, and the underlying factor structure of each of them was explored. Learning-goal structure and performance-goal structure were considered as a single questionnaire with two correlated factors. The dimensions referred to teacher-student and student-student relationships were also considered as a single questionnaire with three first-order factors and a second-order factor labeled quality of student attitudes. Quality of leadership was considered as a single questionnaire, as well as quality of relationships among teachers, and parental involvement. The first two questionnaires model included a second-order factor, while the parental involvement questionnaire was unidimensional.
Although the seven-factor structure initially proposed proved useful to assess teacher perceptions of school climate, factor structures tested in Costa Rica context offer two advantages. First, the joint use of these questionnaires provides schools with a comprehensive perspective of school climate through the scores obtained in each one of them. In addition, schools may decide to use only some of these questionnaires and/or supplement their use with other scales that they consider of interest according to school needs. Second, and more important, multilevel factor structure underlying those questionnaires, including first- and second-order factors allows to obtain scores not only on the general construct, but also on specific aspects of each construct assessed. Therefore, it provides a more detailed information on school strengths and on those aspects susceptible to improvement, which is especially useful to guide an intervention.
This analytic strategy proved useful in assessing perceptions of Costa Rican teachers in a valid and reliable way as well as in designing future interventions for schools involved. However, factor structures proposed were not tested in the Spanish context, so there may be significant differences across both cultures. Although we did not find studies comparing teachers’ perceptions of school climate cross-culturally, there is some evidence suggesting that school climate perceptions vary according to gender, race, ethnicity, and culture. For example, in the context of North American schools Black and Hispanic students were found to perceive school climate less favorably and reported lower levels of security and connectedness to school than their White peers (Voight & Nation 2015). Yang et al. (2013) conducted a cross-cultural study among Chinese and American students. Researchers found that the latter tended to perceive school climate less favorably while Chinese students attributed more importance to academic achievement, respect for teachers, and social harmony. Likewise, significant differences have been found in the importance attributed by Arab and Jewish schools to certain values (Daniel et al., 2013). These findings suggest the importance of considering possible differences in school climate perceptions and justify the need to continue exploring such discrepancies, especially among cultures. Therefore, the present study had two objectives: 1) to test the fit of Spanish data to the models validated in Costa Rica and 2) to test whether there are differences across the two countries and to deduce implications for measurement and intervention if such differences are present.
MethodParticipants
A total of 348 secondary and high school teachers from 92 schools in Spain participated in this study. After removing 5 participants with missing data, the final sample was composed of 343 respondents (54% female and 46% male). Age ranged from 25 to 65 years (mean = 45,1, SD = 9,1), and teaching experience from 1 to 46 years (mean = 17, SD = 10). Per subject, 13% of teachers taught Spanish language and literature, 13% mathematics, 11.5% social sciences, 6.2% natural sciences, 4.3% physics or chemistry, 17.6% foreign languages, 3.4% art, 1.2% religion, and 29.8% other subjects.
The Costa Rican sample was composed of 185 secondary and high school teachers from 14 schools. After removing 7 participants with missing data, the final sample was composed of 178 participants. Females were 53% and males 47%. Age ranged from 21 to 64 years (mean = 37.7, SD = 0.3), and teaching experience from 1 to 36 (mean = 10.9, SD = 6.6). Per subject, 11.8% teachers taught Spanish language and literature, 10.1% mathematics, 11.2% social sciences, 6.2% natural sciences, 7.9% physics or chemistry, 19.7% foreign languages, 2.8% art, 1.7% religion, and 27% other subjects.
Procedure
Spanish teachers participated voluntarily after their schools received an informative letter explaining the purpose of the study. Since research has shown that paper-and-pencil and Internet data collection methods are generally equivalent (Weigold et al., 2013), questionnaires were provided in both formats to make them more accessible. Data were collected from October to February 2015. The Research Ethics Committee of the Universidad Autónoma of Madrid granted approval for the present study.
Costa Rica data were collected in July 2017, as part of a project carried out for the Ministry of Public Education of Costa Rica (Alonso-Tapia, 2017). The purpose of this project was to assess teachers’ perceptions of school climate and to provide advice to improve teaching quality. Teachers completed the questionnaires in paper-and-pencil format.
Instruments
Leadership Quality Questionnaire (LQ-Q). It includes 24 items, measuring 4 aspects related to Communication and Support to Teachers: 1) democratic leadership, 2) clear and constant communication, 3) support to demands of the teaching staff, 4) support to teachers facing difficulties with families/students; and 5 aspects related to Management and Values of the School: 1) orientation towards learning versus performance, 2) planned management, 3) interest in obtaining resources for the school, 4) cooperative work among teachers stimulation, and 5) positive coexistence climate fostering. This questionnaire showed satisfactory psychometric properties in the study conducted with Costa Rica’s sample: ω = .91 for the first subscale; ω = .79 for the second subscale; and ω = .99 for the global scale.
Teachers’ Quality of Support Questionnaire (TQS-Q). It includes 12 items, measuring 2 aspects related to Desire to Improve as a Teacher: 1) showing a continuous effort in the daily work, 2) proposing new initiatives; 2 aspects related to Work Support among coworkers: 1) helping other teachers, 2) working cooperatively; and 2 aspects related to Emotional Support: 1) comprehension versus criticism, and 2) help with worries. This questionnaire also showed adequate psychometric properties: ω = .83 for the first subscale; ω = .75 for the second subscale; ω = .66 for the third subscale; and ω = .96 for the global scale.
School Motivational Orientation Questionnaire (SMO-Q). It includes 8 items, 4 measuring School Motivational Orientation towards Learning (e.g., “at my school, the emphasis is placed on students really understanding, not just memorizing”). On the contrary, the remaining 4 items measured School Motivational Orientation towards Performance (e.g., “at my school students with higher grades are congratulated publicly to motivate the rest of the students”). This questionnaire showed good reliability: ω = .88 for the learning motivation orientation scale, and ω = .93 for the performance motivation orientation scale.
Attitudinal Quality of Students Questionnaire (AQS-Q). It includes 14 items, measuring 2 aspects related to the teachers’ perceptions of Students Confidence at the classroom: 1) confidence to communicate with the teacher, 2) assertiveness when teacher proposes tasks; 3 aspects related to Cooperation in the classroom: 1) cooperation with teacher, 2) cooperation among classmates, 3) classroom discipline; and 2 aspects related to Students Motivation: 1) interest in the subject, and 2) effort to achieve learning goals. The reliability coefficient was .81 for the first subscale, .79 for the second subscale, .78 for the third subscale, and .94 for the global scale.
Quality of Parental Support Questionnaire (QPS-Q). It includes 8 items, with 2 items measuring aspects related to Cooperation with the Teacher (e.g., most of the families respond positively to my indications and suggestions); 2 items measuring aspects related to Contribution from Parents to Students Trusting the Teacher (e.g., “most of the families of the majority contribute to their children trusting me”); 2 items measuring aspects related to Respect towards Teacher (e.g., “most of the families try that the students value positively what I tell them”); and the 2 remaining items assessing the Role of Parents in the Interest and Effort of Students to Learn (e.g., “if my students are interested and strive to learn it is thanks to the cooperation of their families”). The reliability coefficient was .82.
Teachers Satisfaction Scale (TSS). To obtain additional information on the validity of these questionnaires, a scale consisting of 14 items was used as a criterion measure. This scale measures 5 aspects related to Satisfaction with the School and Coworkers: 1) satisfaction with the work of the management team, 2) satisfaction with the teaching-learning system, 3) satisfaction with the relationships among teachers, 4) satisfaction with school rules, 5) satisfaction with the coexistence and security climate of the school; and 2 aspects related to the Satisfaction with the Students and their Families: 1) satisfaction with the learning and recognition shown by the students, and 2) satisfaction with the support received from parents. The reliability coefficient was .91 for the first subscale, .80 for the second subscale, and .73 for the global scale.
Statements were balanced to avoid acquiescence so that there was the same number of positively and negatively worded items in each questionnaire. Items were gauged using five-point Likert-type scales ranging from 1 (strongly disagree) to 5 (strongly agree).
Data Analysis
Firstly, several confirmatory factor analysis (CFA) were performed using Amos 24 (Arbuckle, 2016) to test the fit of the Spanish data to the models proposed in the study carried out in Costa Rica. Factor structures described for each questionnaire were used as baseline models. Therefore, in the first CFA, LQ-Q factor structure was tested, in the second CFA, TQS-Q factor structor was tested, and so on. Item parcels were used since this procedure has known advantages such as fewer parameters to estimate and fewer chances for residuals to be correlated or dual loadings to emerge (MacCallum et al., 1999). Therefore, this procedure allows us to specify a clear latent construct (Little et al., 2002), which provides a simpler and potentially more useful interpretation. Maximum likelihood was used to estimate the proposed models since this estimation procedure is reasonably robust within the context of structural equations even if multivariate normality is not fulfilled (West, Finch, & Curran, 1995). In addition, according to Rodríguez and Ruiz (2008), exploratory and confirmatory factor analysis results will not be distorted by this fact if Mardia’s coefficient is below 70, as is the case of the present study since Mardia’s coefficient reached values between 4.20 and 25.75. Model fit was assessed using comparative fit index (CFI), Tucker-Lewis index (TLI), root mean square error of approximation (RMSEA), and standardized root mean square residual (SRMR). CFI and TLI values ≥ .90 indicate acceptable fit, while values β .95 indicate a good fit (Hu & Bentler, 1999). RMSEA and SRMR values between .05 and .08 represent an acceptable fit, whereas values lower than .05 indicate good fit (McDonald & Ho, 2002). The criteria suggested by Hair et al. (2010) according to which χ2/df ≤ 5 indicates a good model fit was also considered. Additionally, the internal reliability of each questionnaire for the Spanish sample was determined through omega coefficient (McDonald, 1999).
Confirmatory Standardized Solution of the Baseline Models Tested for Each Questionnaire, and its Reliability Coefficients in the Spanish Sample.
Values in parentheses represent standardized factor loadings for the Costa Rican sample and values without parentheses loadings for the Spanish sample.
Godness of Fit Statistics for each Baseline Model Tested in the Spanish Sample and for Multi-Group Cross-validation Analysis
Note. BM = baseline model; CV = cross-validation.
Secondly, multi-group confirmatory factor analyses (MG-CFA) were performed to cross-validate the results. Models described for each questionnaire were used as baseline models for comparison across the two samples (i.e., Spanish and Costa Rica sample) without any parameter constraints. Against the unconstrained model tested for each questionnaire, several models, in which equality between groups was imposed on different sets of parameters, were tested and compared. The relative decline in goodness-of-fit was assessed by means of the difference in chi-square statistic between the model with restrictions imposed and the model without restrictions. In case of significant decline in goodness of fit, it was decided to analyse the reasons for such decline, testing which differences between regression weights were significant using the z-test by Clogg et al. (1995).
Thirdly, in order to determine whether aspects of school climate evaluated by the questionnaires were positively and significantly related to the point of being considered as indicators of the same construct (i.e., school climate), a second-order model with six specific factors (i.e., leadership, support among teachers, learning motivation orientation, performance motivation orientation, student attitudes, and parental support) was tested. Additionally, the discriminant validity was evaluated following the indications proposed by Hair et al. (2014) with the objective of exploring whether these six components of school climate are sufficiently different in terms of their empirical standards. According to these authors, when the average variance extracted by each dimension is greater than the squared correlation between these two dimensions, discriminant validity is ensured.
Fourthly, scales’ reliability was analyzed using McDonald’s (1999) ω index.
Fifthly, to obtain additional information on the concurrent validity of these questionnaires and to examine to what extent each of the aspects of the school climate assessed was related to teacher satisfaction, correlation and regression analysis were performed.
ResultsConfirmatory Factor Analysis and Cross-validation Analyses
Figure 1 shows standardized factor loadings for confirmatory models tested with the sample of Spanish teachers. All factor loadings were significant (p < .001) and had a medium-high size. Table 1 shows fit statistics for each baseline model. For SMO-Q (CFA-5) and TSS (CFA-11), χ2/df ratios were > 5 probably due to sample size (Hair et al., 2010), but the remaining fit indexes were well inside the limits that allowed the models to be accepted. The rest of the models showed satisfactory goodness-of-fit indexes according to recommended cutoff values.
Chi-Square Differences for Model Comparison against the Unconstrained Multi-Sample Model
Results of the cross-validation analysis using the Spanish and Costa Rican teacher samples (n = 343, 178) are displayed in Table 1. As can be seen, except for the QPS-Q (CFA-10), chi-square values were significant, again probably due to sample size, but χ2/df ratios as well as the rest of fit statistics were satisfactory enough to accept these models.
Multi-group Confirmatory Factor Analyses
In order to test the validity of these models across both countries, a multi-group analysis was performed for each questionnaire. Comparison statistics are shown in Table 2. In the case of quality of teachers support (CFA-4), quality of student attitudes (CFA-8), and teachers’ satisfaction (CFA-12), model fit is not reduced even when equality restrictions across samples were imposed, except for restrictions referred to residuals. It is widely accepted that testing for residual equivalence represents an overly restrictive criteria and does not compromise conclusions drawn from this analysis (Byrne, 2010), so this finding was expected.
In the case of questionnaires related to quality of leadership (CFA-2), school motivational orientation (CFA-6), and parental support (CFA-10), fit was significantly reduced when restrictions across samples were imposed. This means that perceptions of Spanish and Costa Rican teachers differ in some aspects of the school climate evaluated by these questionnaires. To know which aspects were valued differently by teachers of each group, regression weights were compared using z-statistic of Clogg et al. (1995). Results of this analysis for the leadership quality questionnaire are shown in Table 3. As can be seen, there were significant differences (z > 1.96) in the weights of three aspects. On one side, first, the relationship between the degree to which “management team supports teachers’ demands” and “communication and support” – a dimension of the quality of leadership – is greater for Spanish teachers (difference: 0.13, z = 2.44). Besides, the relationship between “management and school values” – the second dimension of quality of leadership – and “quality of leadership” is greater also for Spanish Teachers (difference: 0.27, z = 4.25). On the other side, the relationship between “stimulation of cooperative work among teachers” and “management and school values” is greater for Costa Rican teachers (difference: -1.13, z = -2.77).
Differences between Measurement and Structural Raw Weights in the Quality of School Management Perceived by Spanish and Costa Rican Teachers
Note. SE = standard error; SP = Spain; CR = Costa Rica.
Differences between Raw Measurement Weights in the School Motivational Orientation Perceived by Spanish and Costa Rican Teachers
Note. SE = standard error; SP = Spain; CR = Costa Rica.
Standardized Factor Loadings of the Confirmatory Second-order Model Tested to Prove the Structural Validity of the Questionnaires.
Values in parentheses represent the standardized factor loadings for the model tested with the Costa Rican sample.
Table 4 shows results for the school motivational orientation questionnaire. In this case, Costa Rican teachers valued three of the aspects related to such motivational orientation to a greater degree than Spanish teachers. These aspects have to do with perception of mistakes as part of the learning process (difference: -0.28, z = -3.55), with the use of public congratulations to those students who obtain the highest grades (difference: -0.67, z = -3.20), and with the importance of supporting students with better academic attitudes (difference: -0.87, z = -3.62).
Finally, with regard to parental support, Costa Rican teachers value the interest of parents for their children’s learning to a greater extent (difference: -0.20, z = -4.34), while Spanish teachers value more respect for the work of the teacher (difference: 0.13, z = 2.70), and parents’ collaboration in the confidence of students towards teacher (difference: 0.20, z = 3.55). These results are shown in Table 5.
Differences between Measurement Weights in the Parental Support Perceived by Spanish and Costa Rican Teachers
Note. SE = standard error; SP = Spain; CR = Costa Rica.
Structural Equation Model Tested to Examine the Relation between Each Component of School Climate and Teacher Satisfaction.
Values in parentheses represent the results obtained with the Costa Rican sample. R2 is the proportion of variance explained by the model.Evidence of the Second-Order Model Structural Validity
The model fit was satisfactory for the sample of Spanish teachers (χ2/df = 3.54 < 5; CFI = .93 >.90; TLI = .92 >.90; RMSEA = .06 < .08; SRMR = .07 < .08). In the cross-validation analysis, the χ2 statistic was significant, but again χ2/df ratio and the remaining fit indices were within acceptable limits (χ2/df = 2.69 < 5; CFI = .92 >.90; TLI = .91 >.90; RMSEA = .05 < .08; SRMR = .07 < .08). As can be seen in Figure 2, there are differences in both factor loadings of school climate components and across samples. In both samples, quality of leadership, support among teachers, and learning motivational orientation present higher factor loadings with respect to the three remaining components. Likewise, there are differences across countries in the way of perceiving performance motivational orientation. Costa Rican teachers consider that performance motivational orientation contributes positively to school climate, while the opposite happens in the case of Spanish teachers.
Although model fit was good, reliability in both samples was low (ω < .30). Therefore, additional models were tested to examine whether reliability varies according to the number of school climate components included. When an alternative model with three lower-order factors (leadership, support among teachers, and motivational learning orientation) and the same higher-order factor (school climate) was tested, reliability was good (.80 for the Spanish sample, and .83 for the Costa Rican sample) and results of the cross-validation analysis were also satisfactory (CFI = .97 >.90; TLI = .95 >.90; RMSEA = .08 ≤ .08; SRMR = .03 < .08). However, when this same model was tested by adding a fourth factor of a lower-order (performance motivational orientation), reliability decreased again (.61 for the Spanish sample and .58 for the Costa Rican sample) even though results of the cross-validation analysis were satisfactory (CFI = .95 >.90; TLI = .93 >.90; RMSEA = .08 ≤ .08; SRMR = .05 < .08). The same occurs when this lower-order fourth factor is replaced by attitudes of students (.57 for the Spanish sample, and .60 for the Costa Rican sample), or by parental support (.57 for the Spanish sample, and .58 for the Costa Rican sample).
Table 6 shows the average variance extracted for each of the six components of school climate and squared correlations between them. As can be seen, the average variance extracted was greater than squared correlations, which means that components are sufficiently different from each other and discriminant validity is established.
Squared Correlations between the Six Components of the School Climate and the Average Variance Extracted for Each One of Them
Note. Values in parentheses represent the average variance extracted for each school climate component.
Reliability
In Figure 1, McDonald’s ω coefficients for the scales of each questionnaire have been placed above latent factors. They are between .83 and .99, what implies a good internal consistency.
Correlation and Regression Analysis
Correlations between the six school climate dimensions and the two teacher satisfaction dimensions are shown in Table 7. All the dimensions were related in the expected way and the correlations obtained for both samples were similar, except for performance motivational orientation, which was negatively related to teacher satisfaction in the Spanish sample but was positively related to satisfaction in the Costa Rican sample.
Correlations between the Six Components of School Climate and the Two Components of Teacher Satisfaction
**p < .01.
Figure 3 shows the relative weight of each school climate dimension in predicting teachers’ satisfaction. Model fit was good for both the sample of Spanish teachers (χ2/df = 2.93 < 5; CFI = .95 >.90; TLI = .94 >.90; RMSEA = .06 <.08; SRMR = .05 <.08) and Costa Rican teachers (χ2/df = 1.65 < 5; CFI = .92 >.90; TLI = .90 = .90; RMSEA = .06 <.08; SRMR = .06 <.08). In both cases, leadership was the aspect of school climate that contributes the most to explain teacher satisfaction with school (β = .65, β = .45), while student attitudes were the aspect that contributes most to explain teacher satisfaction with students and families (β = .50, β = .61). Overall, dimensions of school climate explained 79% of teacher satisfaction variance related to school for the Spanish sample and 70% for the Costa Rican sample, and 54% of satisfaction related to students and families for the Spanish sample and 44% for the Costa Rican sample.
Discussion
The aim of this study was to explore whether there are differences in teacher perceptions of school climate across different cultural contexts as well as theoretical and practical implications. A secondary purpose was to provide evidence on a different factor structure than that initially proposed for the SCQ-SHST in support of its use among schools. Results of this study are discussed in light of these two objectives.
First of all, the results based on the different CFAs indicate that questionnaires have a well-estimated factor structure and are valid in the context of Costa Rica and Spain. This means that, to the extent that schools promote and emphasize the set of aspects measured and evaluated by these questionnaires, they will be able to create a positive school climate for most teachers. Likewise, results obtained supported the use of SCQ-SHST as modified in this study. The individual use of each questionnaire provides schools with detailed information on different components of the school climate assessed, which makes it possible to detect those aspects to be modified. In addition, to the extent that these questionnaires are used together, they provide an overview of the school climate perceived through the facets evaluated. This information will be useful to detect the strengths of a school and potential areas of intervention as well as the impact of intervention programs, especially of those aimed at improving learning environment.
Second, results of this study support and extend findings of previous studies (e.g., Yang et al., 2013) showing that differences in school climate perceptions are present when comparing teachers from different cultural contexts. A priori, there is not a strong cultural contrast between the two countries included in this study since, for example, both share the same language. However, our results point out that there are significant differences in the way teachers perceive school climate as well as in the value they attribute to some aspects of it. For example, when it comes to leadership quality, Costa Rican teachers consider more important aspects related to school management (i.e., encouraging cooperative work among teachers). In contrast, Spanish teachers value the fact that the school’s management team involves the teaching staff in decision-making and considers their opinions and needs. In Spain, teachers who are part of a school can decide who will play the principal’s role, while in Costa Rica teachers do not have the opportunity to decide on this school’s aspect. Therefore, often, in Spanish schools the principal is a teacher who has been part of the teaching staff and perhaps, for this reason, teachers consider that it is more important that their demands and opinions are heard by him/her. However, this is only a possible explanation for these findings and further research is needed.
In the case of school, significant differences across countries were found in motivational orientation and parental involvement. Considering mistakes as a natural part of the learning process, praising students who achieve higher grades, and encouraging smarter students is valued more positively in Costa Rica than in Spain. Likewise, the interest and effort of parents in their children’s learning is valued more positively in Costa Rica, while Spanish teachers value to a greater extent that parents encourage respect for the teacher and student discipline. Consequently, it is convenient to take into account these differences when conducting cross-cultural studies or when promoting actions aimed at creating a positive school climate for teachers. However, it is interesting to reflect on the possible consequences of emphasizing the importance of achieving high marks or praising the smartest students to be a role model for the rest of the students. Cho and Shim (2013) found that some teachers with a high sense of efficacy consider that a performance motivational orientation is also important and, thus, constitute a desirable goal for teaching. However, the existing literature has consistently reported the benefits of a learning-oriented climate versus a performance-oriented climate. Research has pointed out that when teachers create a learning-oriented classroom climate, students tend to pursue these kinds of goal and use more effective learning strategies such as discriminating relevant information from the one that is not, trying to integrate the new information with the one they know, being more persistent and creative, and taking on more challenging tasks. Conversely, students who perceive a performance-oriented climate tend to endorse this goal and use surface-level strategies such as rereading the text, memorizing, guessing and show cheating behaviors (Ames & Archer, 1988). Likewise, Skaalvik, and Skaalvik (2017) found that teachers’ perception of a performance-goal structure is related to the experience of greater emotional exhaustion and motivation to leave the teaching profession, and a lower sense of self-efficacy and job satisfaction. Therefore, it is convenient to reflect on the type of goals that are considered desirable and that are emphasized at school, especially when trying to improve the conditions that favor student learning.
Third, the results of this study suggest that not all components of school climate are part of it in the same degree or in the same way. The results of each questionnaire taken individually indicate that they reliably evaluate each of the six components of school climate. However, the reliability of all of them taken together as part of the same questionnaire is not adequate, probably due to the degree to which they contribute to school climate. Leadership quality, support among teachers, and learning motivational orientation represent aspects directly related to the teaching staff. Therefore, it makes sense that these three components considered together show adequate reliability.
When it comes to performance motivational orientation, there are important differences across countries, since Spanish teachers consider that this aspect negatively defines school climate, while Costa Rican teachers consider that it is an aspect that contributes positively to this climate. In addition, this component showed the lowest factor loading on the higher-order factor referred to school climate. Thus, it makes sense that the reliability of the questionnaire decreased when this aspect was included.
Student attitudes were referred to the behavior of students both with teachers and with their classmates. Therefore, although these are aspects on which school and teachers exercise an active role, they are also aspects that do not depend totally on them, which could explain why reliability of the model decreased when this component was included.
Finally, parental involvement can be very variable due to multiple factors that do not depend on school itself such as sociocultural level or working conditions (Eccles & Harold, 1996). Therefore, although it is a relevant aspect of school climate, it often constitutes an aspect external to school.
These results reflect school climate’s complex reality, since there are multiple aspects that compose it and there are differences in terms of level of involvement in school (e.g., parental involvement) and the responsibility that school has over them (e.g., student attitudes). This fact may have important implications in the effectiveness of interventions carried out by school. Those aspects on which school has a more direct control (e.g., quality of leadership or support among teachers) may be more easily malleable and susceptible of improvement compared to those aspects that do not depend directly on school or teachers (e.g., student attitudes or parental involvement). Therefore, these findings should be considered by schools when assessing school climate and deciding on what aspects are worth investing more time and effort.
Fourth, the results of correlation analyses showed that, in both samples, leadership was the aspect most strongly related to satisfaction with school. Again, differences were found in the way performance motivational orientation was related to satisfaction as a function of the country. In line with results discussed above, it was found that Spanish teachers consider that such motivational orientation is negatively related to their satisfaction with school, while for Costa Rican teachers performance motivational orientation contributes positively to their satisfaction both with school and with students and families. The results of the regression analysis also supported the conclusion that leadership is the aspect that most contributes to explaining satisfaction of both Spanish and Costa Rican teachers with school. This finding is consistent with previous studies that show the relationship of leadership not only with satisfaction of teachers, but also with the capacity for innovation, commitment of teachers to implement interventions adopted by the school, and motivation of teachers (Pietsch & Tulowitzki, 2017; Sun & Leithwood, 2015). In addition, because principals control important school-wide conditions, it seems appropriate to consider quality of leadership when evaluating and planning an intervention focused on school climate improvement, especially when it comes to foster motivation and commitment of teachers and, ultimately, to improve student learning.
Finally, the results and conclusions presented should be interpreted considering the limitations of the current study. In the first place, the sample of teachers from Costa Rica is limited by its size, which makes it difficult to generalize the results obtained. In addition, the difference in sample size between the two countries can decrease the power to detect measurement equivalence across samples (Chen, 2007). Second, the language of the questionnaires was adapted to the Spanish spoken in Latin America before being administered. However, the questionnaires were initially developed in the Spanish context, so it is possible that they do not capture aspects of school climate more specific to Latino culture.
Cite this article as: Alonso-Tapia, J., Quijada, A., Ruiz, M., Huertas, J. A., Ulate, M. A., & Biehl, M. L. (2020). A cross-cultural study of the validity of a battery of questionnaires for assessing schoolclimate quality. Psicología Educativa, 26(2), 109-119. https://doi.org/10.5093/psed2020a2
Funding: This study was carried out with funding from the Spanish Ministry of Economy and Competitiveness (Project EDU2017-89036-P), from the Banco Interamericano de Desarrollo, and from the Ministerio de Educación Pública of Costa Rica.
ReferencesAlliance for the Study of School Climate. (2004). The School Climate Assessment Instrument. http://web.calstatela.edu/centers/schoolclimate/services/products.htmlAlliance for the Study of School Climate2004http://web.calstatela.edu/centers/schoolclimate/services/products.htmlAlonso-Tapia, J. (2017). Sistema de evaluación del clima escolar y del clima de clase (SEVACES-CR) (versión para Costa Rica, informe técnico 2). Ministerio de Educación Pública/Banco Interamericano de Desarrollo. Costa Rica.Alonso-TapiaJ.2017(SEVACES-CR) (versión para Costa Rica, informe técnico 2)Ministerio de Educación Pública/Banco Interamericano de DesarrolloCosta RicaAmes, C., & Archer, J. (1988). Achievement goals in the classroom: Students’ learning strategies and motivation processes. Journal of Educational Psychology, 80, 260-267. https://doi.org/10.1037/0022-0663.80.3.260AmesCArcherJ.1988Achievement goals in the classroom: Students’ learning strategies and motivation processes8026026710.1037/0022-0663.80.3.260Anderson, C. (1982). The search for school climate: A review of the research. Review of Educational Research, 52, 368-420. https://doi.org/10.3102/00346543052003368AndersonC.1982The search for school climate: A review of the research5236842010.3102/00346543052003368Arbuckle, J. L. (2003). Amos 5.0 update to the Amos user’s guide. Small Waters.ArbuckleJ. L.2003Small WatersBear, G. G., Yang, C., Pell, M., & Gaskins, C. (2014). Validation of a brief measure of teachers’ perceptions of school climate: Relations to student achievement and suspensions. Learning Environments Research, 17, 339-354. https://doi.org/10.1007/s10984-014-9162-1BearG. GYangCPellMGaskinsC.2014Validation of a brief measure of teachers’ perceptions of school climate: Relations to student achievement and suspensions1733935410.1007/s10984-014-9162-1Byrne, B. M. (2010). Structural equation modeling with AMOS: Basic concepts, applications and programming (2nd ed). Taylor & Francis Group.ByrneB. M.20102nd edTaylor & Francis GroupChen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14, 464-504. https://doi.org/10.1080/10705510701301834ChenF. F.2007Sensitivity of goodness of fit indexes to lack of measurement invariance1446450410.1080/10705510701301834Cho, Y.-J., & Shim, S. S. (2013). Predicting teachers’ achievement goals for teaching: The role of perceived school goal structure and teachers’ sense of efficacy. Teaching and Teacher Education, 32, 12-21. https://doi.org/10.1016/j.tate.2012.12.003ChoY.-JShimS. S.2013Predicting teachers’ achievement goals for teaching: The role of perceived school goal structure and teachers’ sense of efficacy32122110.1016/j.tate.2012.12.003Clifford, M., Menon, R., Gangi, T., Condon, C., & Hornung, K. (2012). Measuring school climate for gauging principal performance: A review of the validity and reliability of publicly accessible measures. American Institutes for Research.CliffordMMenonRGangiTCondonCHornungK.2012American Institutes for ResearchClogg, C., Petkova, E., & Haritou, A. (1995). Statistical methods for comparing regression coefficients between models. The American Journal of Sociology, 100, 1261-1293. https://doi.org/10.1086/230638CloggCPetkovaEHaritouA.1995Statistical methods for comparing regression coefficients between models1001261129310.1086/230638Cohen, J., McCabe, E. M., Michelli, N. M., & Pickeral, T. (2009). School climate: Research, policy, practice, and teacher education. Teachers College Record, 111(1), 180-213.CohenJMcCabeE. MMichelliN. MPickeralT.2009School climate: Research, policy, practice, and teacher education1111180213Daniel, E., Hoffman-Towfigh, N., & Knafo, A. (2013). School values: A new typology of school value dimensions across three cultures. SAGE Open. https://doi.org/10.1177/2158244013482469DanielEHoffman-TowfighNKnafoA.2013SAGE Open10.1177/2158244013482469Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82, 405-432. https://doi.org/10.1111/j.1467-8624.2010.01564.xDurlakJ. AWeissbergR. PDymnickiA. BTaylorR. DSchellingerK. B.2011The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions8240543210.1111/j.1467-8624.2010.01564.xEccles, J., & Harold, R. (1996). Family involvement in children’s and adolescent’s schooling. In A. Booth & J. Dunn (Eds.), Family-school links: How do they affect educational outcomes (pp. 3-34). Lawrence Erlbaum Associates, Inc.EcclesJHaroldR.1996Family involvement in children’s and adolescent’s schoolingBoothA.DunnJ.334Lawrence Erlbaum Associates, IncGendron, B., Williams, K., & Guerra, N. (2011). An analysis of bullying among students within schools: Estimating the effects of individual normative beliefs, self-esteem, and school climate. Journal of School Violence, 10, 150-164. https://doi.org/10.1080/15388220.2010.539166GendronBWilliamsKGuerraN.2011An analysis of bullying among students within schools: Estimating the effects of individual normative beliefs, self-esteem, and school climate1015016410.1080/15388220.2010.539166Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate data analysis. Pearson-Prentice Hall.HairJ. FBlackW. CBabinB. JAndersonR. E.2010Pearson-Prentice HallHair, J. F., Hult, G. T. M., Ringle, C. M., & Sarstedt, M. (2014). A primer on partial least squares structural equation modeling (PLS-SEM). Sage.HairJ. FHultG. T. MRingleC. MSarstedtM.2014SageHauser-Cram, P., Sirin, S. R., & Stipek, D. (2003). When teachers’ and parents’ values differ: Teachers’ ratings of academic competence in children from low-income families. Journal of Educational Psychology, 95, 813-820. https://doi.org/10.1037/0022-0663.95.4.813Hauser-CramPSirinS. RStipekD.2003When teachers’ and parents’ values differ: Teachers’ ratings of academic competence in children from low-income families9581382010.1037/0022-0663.95.4.813Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55. https://doi.org/10.1080/10705519909540118HuL. TBentlerP. M.1999Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives615510.1080/10705519909540118Huang, F., & Cornell, D. (2015). Using multilevel factor analysis with clustered data: Investigating the factor structure of the Positive Values Scale. Journal of Psychoeducational Assessment, 34, 3-14. https://doi.org/10.1177/0734282915570278HuangFCornellD.2015Using multilevel factor analysis with clustered data: Investigating the factor structure of the Positive Values Scale3431410.1177/0734282915570278Johnson, B., Down, B., Le Cornu, R., Peters, J., Sullivan, A. M., Pearce, J., & Hunter, J. (2015). Promoting early career teacher resilience: A socio-cultural and critical guide to action. Routledge.JohnsonBDownBLe CornuRPetersJSullivanA. MPearceJHunterJ.2015RoutledgeKoth, C. W., Bradshaw, C. P., & Leaf, P. J. (2008). A multilevel study of predictors of student perceptions of school climate: The effect of classroom-level factors. Journal of Educational Psychology, 100, 96-104. https://doi.org/10.1037/0022-0663.100.1.96KothC. WBradshawC. PLeafP. J.2008A multilevel study of predictors of student perceptions of school climate: The effect of classroom-level factors1009610410.1037/0022-0663.100.1.96Künsting, J., Neuber, V., & Lipowsky, F. (2016). Teacher self-efficacy as a long-term predictor of instructional quality in the classroom. European Journal of Psychology of Education, 31, 299-322. https://doi.org/10.1007/s10212-015-0272-7KünstingJNeuberVLipowskyF.2016Teacher self-efficacy as a long-term predictor of instructional quality in the classroom3129932210.1007/s10212-015-0272-7Little, T. D., Cunningham, W. A., Shahar, G., & Widaman, K. F. (2002). To parcel or not to parcel: Exploring the question, weighing the merits. Structural Equation Modeling, 9, 151-173. https://doi.org/10.1207/S15328007SEM0902_1LittleT. DCunninghamW. AShaharGWidamanK. F.2002To parcel or not to parcel: Exploring the question, weighing the merits915117310.1207/S15328007SEM0902_1Liu, Y., Ding, C., Berkowitz, M. W., & Bier, M. C. (2014). A psychometric evaluation of a revised School Climate Teacher Survey. Canadian Journal of School Psychology, 29, 54-67. https://doi.org/10.1177/0829573514521777LiuYDingCBerkowitzM. WBierM. C.2014A psychometric evaluation of a revised School Climate Teacher Survey29546710.1177/0829573514521777MacCallum, R. C., Widaman, K. F., Zhang, S., & Hong, S. (1999). Sample size in factor analysis. Psychological Methods, 4, 84-99. https://doi.org/10.1037/1082-989X.4.1.84MacCallumR. CWidamanK. FZhangSHongS.1999Sample size in factor analysis4849910.1037/1082-989X.4.1.84McDonald, R. P. (1999). Test theory. A unified treatment. Erlbaum.McDonaldR. P.1999Test theoryErlbaumMcDonald, R. P., & Ho, M. H. R. (2002). Principles and practice in reporting structural equation analyses. Psychological Methods, 7, 64-82. https://doi.org/10.1037/1082-989X.7.1.64McDonaldR. PHoM. H. R.2002Principles and practice in reporting structural equation analyses7648210.1037/1082-989X.7.1.64Midgley, C., Anderman, E., & Hicks, L. (1995). Differences between elementary and middle school teachers and students: A goal theory approach. Journal of Early Adolescence, 15, 90-113. https://doi.org/10.1146/annurev.psych.56.091103.070258MidgleyCAndermanEHicksL.1995Differences between elementary and middle school teachers and students: A goal theory approach159011310.1146/annurev.psych.56.091103.070258Mitchell, M., & Bradshaw, C., (2013). Examining classroom influences on student perceptions of school climate: The role of classroom management and exclusionary discipline strategies. Journal of School Psychology, 51, 599-610. https://doi.org/10.1016/j.jsp.2013.05.005MitchellMBradshawC.2013Examining classroom influences on student perceptions of school climate: The role of classroom management and exclusionary discipline strategies5159961010.1016/j.jsp.2013.05.005National School Climate Council. (2007). The School Climate Challenge: Narrowing the gap between school climate research and school climate policy, practice guidelines and teacher education policy. http://www.schoolclimate.org/climate/advocacy.phpNational School Climate Council2007http://www.schoolclimate.org/climate/advocacy.phpPietsch, M., & Tulowitzki, P. (2017). Disentangling school leadership and its ties to instructional practices – an empirical comparison of various leadership styles. School Effectiveness and School Improvement, 28, 629-649. https://doi.org/10.1080/09243453.2017.1363787PietschMTulowitzkiP2017Disentangling school leadership and its ties to instructional practices – an empirical comparison of various leadership styles2862964910.1080/09243453.2017.1363787Quijada, A., Ruiz, M., Huertas, J. A., & Alonso-Tapia, J. (2020). Development and validation of the School Climate Questionnaire for Secondary and High School Teachers (SCQ-SHST). Anales de Psicología, 36, 155-165. https://doi.org/10.6018/analesps.341001QuijadaARuizMHuertasJ. AAlonso-TapiaJ.2020Development and validation of the School Climate Questionnaire for Secondary and High School Teachers (SCQ-SHST)3615516510.6018/analesps.341001Reeve, I. (2006). Teachers as facilitators: What autonomy-supportive teachers do and why their students benefit. The Elementary School Journal, 106, 225-236. https://doi.org/10.1086/501484ReeveI.2006Teachers as facilitators: What autonomy-supportive teachers do and why their students benefit10622523610.1086/501484Robers, S., Zhang, J., Truman, J., & Snyder, T. D. (2012). Indicators of school crime and safety: 2011. National Center for Education Statistics & the Bureau of Justice Statistics. http://nces.ed.gov/pubs2012/2012002.pdfRobersSZhangJTrumanJSnyderT. D.2012National Center for Education Statistics & the Bureau of Justice Statisticshttp://nces.ed.gov/pubs2012/2012002.pdfRodríguez, M. N., & Ruiz, M. A. (2008). Atenuación de la asimetría y de la curtosis de las puntuaciones observadas mediante transformaciones de variables: incidencia sobre la estructura factorial. Psicológica, 29, 205-227.RodríguezM. NRuizM. A2008Atenuación de la asimetría y de la curtosis de las puntuaciones observadas mediante transformaciones de variables: incidencia sobre la estructura factorial29205227Skaalvik, E. M., & Skaalvik, S. (2011). Teachers’ feeling of belonging, exhaustion, and job satisfaction: The role of school goal structure and value consonance. Anxiety, Stress, & Coping: An International Journal, 24, 369-385. https://doi.org/10.1080/10615806.2010.544300SkaalvikE. MSkaalvikS.2011Teachers’ feeling of belonging, exhaustion, and job satisfaction: The role of school goal structure and value consonance2436938510.1080/10615806.2010.544300Skaalvik, E. M., & Skaalvik, S. (2017). Motivated for teaching? Associations with school goal structure, teacher self-efficacy, job satisfaction and emotional exhaustion. Teaching and Teacher Education, 67, 152-160. https://doi.org/10.1016/j.tate.2017.06.006SkaalvikE. MSkaalvikS.2017Motivated for teaching? Associations with school goal structure, teacher self-efficacy, job satisfaction and emotional exhaustion6715216010.1016/j.tate.2017.06.006Sun. J., & Leithwood, K. (2015). Direction-setting school leadership practices: A meta-analytical review of evidence about their influence. School Effectiveness and School Improvement, 26, 499-523. https://doi.org/10.1080/09243453.2015.1005106SunJ.LeithwoodK.2015Direction-setting school leadership practices: A meta-analytical review of evidence about their influence2649952310.1080/09243453.2015.1005106Thapa, A., Cohen, J., Guffey, S., & Higgins-D’Alessandro, A. (2013). A review of school climate research. Review of Educational Research, 83, 357-385. https://doi.org/10.3102/0034654313483907ThapaACohenJGuffeySHiggins-D’AlessandroA.2013A review of school climate research8335738510.3102/0034654313483907Voight, A., & Nation, M. (2016). Practices for improving secondary school climate: A systematic review of the research literature. American Journal of Community Psychology, 58(1-2), 174-191. https://doi.org/10.1002/ajcp.12074VoightANationM.2016Practices for improving secondary school climate: A systematic review of the research literature581-217419110.1002/ajcp.12074Wang, M. T., & Degol, J. L. (2015). School climate: A review of the construct, measurement, and impact on student outcomes. Educational Psychology Review, 28, 315-352. https://doi.org/10.1007/s10648-015-9319-1WangM. TDegolJ. L.2015School climate: A review of the construct, measurement, and impact on student outcomes2831535210.1007/s10648-015-9319-1Wang, M. T., & Eccles, J. S. (2013). School context, achievement motivation, and academic engagement: A longitudinal study of school engagement using a multidimensional perspective. Learning and Instruction, 28, 12-23. https://doi.org/10.1016/j.learninstruc.2013.04.002WangM. TEcclesJ. S.2013School context, achievement motivation, and academic engagement: A longitudinal study of school engagement using a multidimensional perspective28122310.1016/j.learninstruc.2013.04.002Weigold, A., Weigold, I. K., & Russell, E. J. (2013). Examination of the equivalence of self-report survey-based paper-and-pencil and Internet data collection methods. Psychological Methods, 18, 53-70. https://doi.org/10.1037/a0031607WeigoldAWeigoldI. KRussellE. J.2013Examination of the equivalence of self-report survey-based paper-and-pencil and Internet data collection methods18537010.1037/a0031607Wenglinsky, H. (2000). How teaching matters: Bringing the classroom back into discussions of teacher quality. Educational Testing Service.WenglinskyH.2000Educational Testing ServiceWest, S. G., Finch, J. F., & Curran, P. J. (1995). Structural equation models with nonnormal variables: Problems and remedies. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 56–75). Sage Publications, Inc.WestS. GFinchJ. FCurranP. J.1995Structural equation models with nonnormal variables: Problems and remediesHoyleR. H.5675Sage Publications, IncYang, C., Bear, G. G., Chen, F. F., Zhang, W., Blank, J. C., & Huang, X. (2013). Students’ perceptions of school climate in the U.S. and China. School Psychology Quarterly, 28, 7-24. https://doi.org/10.1037/spq0000002YangCBearG. GChenF. FZhangWBlankJ. CHuangX.2013tudents’ perceptions of school climate in the U.S. and China2872410.1037/spq0000002