Assessing Organizational Capacity for School Mental Health: Validation of the School Mental Health Capacity Instrument (SMHCI) in Chile

Introduction

Children-and-adolescents’ mental health has emerged as a priority concern on global educational and healthcare agendas, due to its direct connection with learning, school engagement, and socio-emotional development. It is estimated that approximately 50% of mental disorders begin before the age of 14, and that one in five minors will experience some form of psychiatric condition before reaching adulthood (Kieling et al., 2011; World Health Organization [WHO, 2021]). Within this context, the school is no longer viewed solely as an academic space; rather, it has assumed a strategic role in promoting psychological well-being and preventing mental disorders—particularly in settings characterized by high levels of inequality and fragmented services (Hoover & Bostic, 2021; Weist et al., 2023).

In Latin America, various studies have documented a high prevalence of emotional and behavioral symptoms among school-aged populations. An epidemiological study conducted in Chile estimated a general prevalence of psychiatric disorders at 22.5% among children and adolescents (Vicente et al., 2012). More recently, a national study involving over one thousand students documented high post-pandemic levels of self-reported symptoms of depression, anxiety, and stress, as assessed through screening measures. Prevalence rates reached 60.2% for depressive symptoms, 63.6% for anxiety symptoms, and 50.2% for stress symptoms. Higher levels were observed among female students, adolescents with separated parents, and those reporting sleep difficulties or low self-esteem (Martínez-Líbano & Yeomans-Cabrera, 2024). Similar findings have been reported in regional studies from the northern part of the country, highlighting a significant burden of depressive and anxious symptoms among adolescents living in vulnerable areas (Caqueo-Urízar et al., 2023).

This body of evidence has prompted a conceptual shift regarding the role of schools, recognizing them as privileged settings for fostering institutional practices of care, emotional support, and socio-affective development (Fonseca-Pedrero et al., 2023). Within this new paradigm, school mental health is no longer confined to the clinical referral of individual cases; rather, it is understood as a collective responsibility that demands deep and sustained organizational transformations.

Institutional responses to the growing demand for school mental health support have, in many countries, led to the implementation of organizational models that integrate promotion, prevention, and intervention within a coherent and continuous system. One of the most well-established approaches is the Multi-Tiered Systems of Support (MTSS), which proposes a tiered service architecture based on the level of need. This model distinguishes three levels: promotion/universal (for all students), early intervention/targeted (for those exhibiting early signs of distress), and treatment/intensive (for students requiring specialized care or clinical referral) (Hoover & Bostic, 2021).

This model has been adopted in various public policies, regulatory frameworks, and school-based programs in countries such as the United States, Canada, Australia, and more recently in pilot initiatives across Latin America (Moore et al., 2024; Rojas-Andrade et al., 2023). The preventive and tiered logic of MTSS enables schools to move beyond a solely reactive stance—responding only in times of crisis—toward becoming environments for early identification, continuous support, and the promotion of psychological well-being throughout the entire school community (Stephan et al., 2015; Weist et al., 2014).

The transformative potential of the MTSS approach lies in its ability to reorganize institutional work, distribute responsibilities, and foster systematic practices of monitoring, adaptation, and inclusion. Unlike fragmented models or those focused exclusively on individual care, MTSS promotes an integrated response that brings together pedagogical, clinical, and community-based knowledge (Raines & Dibble, 2011). However, its successful implementation depends on each institution’s organizational readiness, the availability of appropriate technical support, and the existence of an intersectoral ecosystem capable of sustaining interventions beyond initial enthusiasm or the individual capacities of professionals (Moore et al., 2024; Rojas-Andrade et al., 2023).

The effective implementation of school mental health models, such as the MTSS, depends not only on their conceptual quality or regulatory endorsement, but also on their capacity to be translated into sustained institutional practices. Numerous studies have noted that schools—particularly those in highly vulnerable contexts—face significant challenges in adopting and embedding complex models without structured and sustained support (Fixsen et al., 2009). In this context, technical assistance (TA) has emerged as a key strategy to support these processes, facilitating organizational change and strengthening local implementation capacities.

From the perspective of implementation science, TA is understood as a systematic, adaptive, and bidirectional process in which external actors collaborate with schools to install evidence-based practices, address contextual barriers, and promote continuous improvement processes (Dunst et al., 2019; Wandersman et al., 2012). Its objective is not to replace local decision-making, but to enhance organizational autonomy by strengthening competencies, fostering collective reflection, and developing structures that support long-term sustainability.

The effectiveness of TA, however, is not guaranteed by its mere presence. Various studies have identified key conditions that determine its impact, including the duration and frequency of support, the relevance of its content, the capacity to adapt to local contexts, and the quality of the relational dynamic between consultants and school teams (Domitrovich et al., 2008; Meyers et al., 2012). More recently, scholars have argued that these conditions must be assessed in relation to the organizational readiness of schools, understood as the set of structural, cultural, and technical capacities that enable the adoption and implementation of complex models (Moore et al., 2024; Rojas-Andrade et al., 2023). From this perspective, TA not only supports task execution, but also functions as a strategic facilitator that activates latent internal conditions, enabling schools to move from intention to sustainable action.

In Chile, this perspective has begun to be applied to nationally scaled programs. A representative example is the A Convivir se Aprende program, promoted by the Chilean Ministry of Education (MINEDUC), and technically supported by public universities. In this case, technical assistance extended beyond team training and was structured as a prolonged advisory process, which included participatory diagnostics, strategic planning, reflective evaluation, and context-sensitive accompaniment in each local territory. Implementation study results indicated that both adherence to core components and the receptiveness of school teams were decisive factors in perceptions of the model’s feasibility (Rojas-Andrade et al., 2023).

This evidence reinforces the notion that TA should not be viewed as an optional add-on, but rather as a structural component of school mental health policies. However, for this strategy to fulfill its guiding purpose, it is essential to have tools that clearly and accurately identify the technical support needs of schools during their implementation processes (Dunst et al., 2019). This presents the challenge of developing specific instruments for assessing school mental health advisory needs, which constitutes the focus of the following section.

One of the most significant conceptual contributions to the development of comprehensive school mental health models has been made by Mark Weist and his team, who since the late 1990s have promoted the “Expanded school mental health” approach as a collaborative framework that integrates universal promotion, targeted prevention, and clinical intervention within educational settings. Within this framework, Weist et al. (2005) proposed a set of ten principles to guide the implementation of school-based programs. These principles range from universal accessibility and curricular integration to technical quality, school community participation, and coordination with external services. Widely validated and adapted across various countries, these principles laid the foundation for identifying critical implementation components within MTSS, becoming both normative and operational benchmarks for assessing schools’ institutional capacity.

Since then, various tools have been developed with the aim of operationalizing these principles and translating them into useful resources for planning and technical support. One example is the Survey of School Promotion of Emotional and Social Health (SSPESH), developed in Australia by Dix et al. (2019), which allows for the classification of schools based on their perceived level of implementation across four domains: universal promotion, social-emotional skills, family engagement, and individual support.

In the Chilean context, Rojas-Andrade and Leiva (2019) developed the School Mental Health Evaluation Questionnaire (CESME), adapting Weist’s principles based on the experience of the Habilidades para la Vida program. This instrument assesses adherence to 40 strategic components distributed across areas such as direct intervention with students, pedagogical planning, institutional coordination, and family engagement. It has demonstrated sensitivity in identifying implementation patterns associated with better psychosocial outcomes.

Nevertheless, both nationally and regionally, a gap remains in the availability of specific instruments capable of validly and contextually identifying schools’ technical assistance needs during the implementation of multi-tiered support systems. In this regard, the School Mental Health Capacity Instrument (SMHCI), developed by Feigenberg et al. (2010) represents a significant advancement by offering a structured measure of institutional capacity to address mental health through a preventive, systematic and sustainable lens. This instrument was specifically designed as a diagnostic tool to guide differentiated technical assistance processes, evaluating organizational readiness across three domains: promotion, early identification, and intervention. Its structure and purpose make it a valuable reference for the development of instruments adapted to the school realities of Latin America.

Building on this background, the present study aims to advance the understanding and assessment of institutional readiness in Chilean public schools to implement a comprehensive school mental health approach, using a self-report questionnaire administered to professionals within the educational system. Specifically, the study seeks to examine whether the domains proposed by the SMHCI are valid and relevant within the local context, and whether they enable the identification of differentiated patterns of technical assistance needs that can inform future support processes.

The Present Study

This study aims to validate the theoretical structure of the SMHCI within the Chilean educational context to provide a reliable tool for assessing schools’ organizational capacity to implement comprehensive school mental health strategies. Through this validation process, the study seeks to examine the internal consistency and factorial structure of the adapted instrument, exploring its usefulness in diagnosing technical assistance needs in school contexts transitioning toward multi-tiered intervention models.

Given the growing demand for tools that guide planning and technical support in school mental health – particularly in education systems characterized by high organizational heterogeneity – the results of this study aim to provide an empirical and contextually grounded foundation to inform strategic decisions in the design and implementation of institutional strengthening policies and programs.

Method

Study design

An instrumental study was conducted using a non-experimental, cross-sectional design aimed at evaluating the psychometric properties of an adapted measure to assess schools’ institutional capacity in school mental health (Ato et al. 2013).

Participants

The sample consisted of 237 professionals from school psychosocial teams who voluntarily participated in a nationwide training course on school mental health. A purposive sampling strategy was employed, targeting the most common professional profiles within psychosocial support teams: psychology (68.2%), social work (29.1%), and other disciplines (2.7%) such as school climate coordinators, school counselors, and health personnel.

The sample was predominantly female (78.5%) with a mean age of 36 years (SD = 8.7). In terms of employment setting, 96.4% of participants worked within school-based teams, while 3.6% were affiliated with external community-based programs. Among those working in schools, 44.8% were part of school climate or mediation teams, 34.1% belonged to psychosocial duos, 12.1% worked in the Programa de Integración Escolar (PIE), and 5.4% were members of school leadership teams. All regions of the country were represented, with the highest participation from the Región Metropolitana (31.8%), Región de Valparaíso (11.7%), and Región del Biobío (10.3%).

Instrument

An adapted version of the SMHCI (Feigenberg et al., 2010) was used to assess schools’ institutional capacity in mental health. The adaptation process for the Chilean context involved forward–backward translation to ensure semantic and conceptual equivalence (Beaton et al., 2000; Sousa & Rojjanasrirat, 2011). Additionally, the response format was modified, reducing the original five-point scale to three options to enhance clarity and usability. Response options were explicitly labeled as follows: 1 = “We do not do this or do not have this” [No lo hacemos o no lo tenemos], 2 = “We do this to some extent or have some of this” [Lo hacemos un poco o tenemos un poco de esto], and 3 = “We do this to a great extent or have this” [Lo hacemos o tenemos de esto en gran medida].

The final version consists of 29 items distributed across three theoretical dimensions: (a) Promotion [Promoción y prevención], (b) Early identification [Derivación y acompañamiento], and (c) Intervention [Intervención en crisis]. Representative examples of each dimension include: “There are activities or programs that focus on developing students’ strengths and resilience”, “There are structures in place that enable staff to identify students with difficulties at an early stage,” and “There are follow-up protocols for students who experience mental health crises or emergencies”. The instrument has demonstrated high internal consistency and structural validity in previous studies (α = .95, r = .77) (Feigenberg et al., 2010).

Procedure

Data collection was conducted online through an electronic form integrated into the course platform provided by the Chilean Ministry of Health (MINSAL). Prior to completing the questionnaire, participants reviewed and accepted an informed consent form outlining the study’s objectives, ensuring the confidentiality of the information, and emphasizing the voluntary nature of participation.

The research protocol was approved under ethics report number 656/2022 by the ethics committee of the Universidad de Santiago, Chile.

Data Analysis

The analysis was structured into three complementary stages. Firstly, an item analysis was conducted to evaluate the individual behavior of each item. Mean, standard deviation, skewness, kurtosis, and corrected item-total correlations were calculated. Items with correlations above .30 were considered acceptable. Given the ordinal nature of the response scale, these statistics were used for descriptive purposes only.

Secondly, descriptive statistics were computed for each of the instrument’s three dimensions, including mean, standard deviation, observed minimum and maximum values, and internal consistency coefficients using Cronbach’s alpha. In addition, internal consistency was further examined using model-based reliability estimates, specifically the omega coefficient (ω), which are more appropriate for latent variable models (Flora, 2020; McNeish, 2018). Overall reliability of the scale was also estimated.

Thirdly, construct validity was assessed using confirmatory factor analysis (CFA). Two models were compared: a unidimensional model and a three-factor model consistent with the proposed theoretical structure. Given that the items were measured using a three-point ordinal response scale, model parameters were estimated using diagonally weighted least squares (DWLS) using Jamovi software. As an additional robustness check, the three-factor model was replicated using weighted least squares with mean and variance adjustment (WLSMV) in an alternative analytical environment.

To assess model fit, the following indices were used: the χ²/df ratio (values < 3), the comparative fit index (CFI) and the Tucker-Lewis index (TLI), with values ≥ .90 considered adequate and ≥ .95 excellent; the standardized root mean square residual (SRMR), with values below .08; and the root mean square error of approximation (RMSEA), with values < .08 considered acceptable and < .06 optimal (Hu & Bentler, 1999; Schermelleh-Engel et al., 2003). Convergent validity was examined using the average variance extracted (AVE), and discriminant validity was assessed using both the Fornell-Larcker criterion and the heterotrait-monotrait ratio (HTMT). Results were interpreted based on their theoretical coherence and usefulness in evaluating institutional capacities in school contexts.

Results

Item Analysis

Item-level descriptive statistics were examined to characterize response distributions. Given the ordinal nature of the three-point response scale, skewness and kurtosis were interpreted descriptively rather than as indicators of normality. Corrected item-total correlations exceeded .30 in most cases.

Confirmatory Factor Analysis (CFA)

To evaluate the structural validity of the SMHCI, a CFA was conducted comparing two models: a unidimensional model and a three-factor model consistent with the instrument’s theoretical structure.

For the three-factor model, the chi-square statistic was χ²(374) = 373,71, p < .01. Given the known sensitivity of the chi-square test to sample size, additional fit indices were examined. The comparative fit index (CFI) and the Tucker-Lewis index (TLI) both reached 100. The root mean square error of approximation (RMSEA) was .00, with a 90% confidence interval ranging from .000 to .02, and the standardized root mean square residual (SRMR) was .07.

In contrast, the unidimensional model showed lower fit indices, with a CFI of .84 and a TLI of .83. The RMSEA was .07 (90% CI [.06, .07]) and the SRMR was .06. Comparative fit indices for both models are presented in Table 2.

Factor Loadings

Standardized factor loadings for the items of the three-factor model are presented in Table 3. Most items showed standardized loadings above .50. Two items (B4 and C7) displayed loadings below .40.

Descriptive Statistics of the Validated Dimensions

Descriptive statistics and internal consistency coefficients were calculated for each of the three SMHCI dimensions. For each dimension, the number of items, mean scores, standard errors, observed ranges, and Cronbach’s alpha coefficients are presented in Table 4.

Discussion

The findings of this study support the empirical and theoretical validity of the SMHCI within the Chilean context. The three-dimensional structure—distinguishing the domains of promotion and prevention, early identification and referral, and crisis intervention—enables a differentiated assessment of schools’ organizational capacity in relation to school mental health. This differentiation is particularly useful in educational settings characterized by high institutional heterogeneity, where needs and levels of organizational maturity vary considerably.

The comparison between the three-factor and the unidimensional models indicated that the multidimensional structure provided a more adequate representation of the data. Specifically, the three-factor model showed substantially higher comparative fit indices (CFI = 1.00, TLI = 1.00) and lower residual-based indices (RMSEA = .000; 90% CI [.00, .02]) than the unidimensional model (CFI = .84, TLI = .83, RMSEA = .07). These differences suggest that schools’ institutional capacity in mental health is better conceptualized as a set of distinct yet interrelated organizational domains rather than as a single, undifferentiated construct. Interpreting school mental health capacity as a multidimensional phenomenon enables a more precise identification of institutional strengths and gaps, avoiding overly global assessments that may obscure meaningful variation across domains.

The validation of the SMHCI in Chile holds particular relevance within an educational system that has implemented school mental health programs in a fragmented manner, with limited systematization in their evaluation (Rojas-Andrade & Leiva, 2018). Although models such as MTSS have demonstrated international effectiveness, their adoption in Chile has been constrained by significant organizational barriers, including the lack of specialized training, limited psychosocial resources, and ambiguity in institutional roles (Rojas-Andrade et al., 2023). In this scenario, having an instrument that accurately diagnoses institutional capacities represents a strategic step toward more sustainable implementation processes.

Indeed, the results of the analysis show that the “promotion and prevention” dimension tend to obtain lower scores, suggesting that schools tend to focus their efforts on individual and reactive responses rather than on universal and proactive well-being strategies. This trend has been documented in recent studies that highlight an overrepresentation of damage-control approaches, to the detriment of a school culture that actively promotes psychological well-being (Durlak et al., 2011; Fonseca-Pedrero et al., 2023). In this context, the school emerges not only as a space for detection or referral, but also as an institutional arena where definitions of mental health are contested, and decisions are made about who manages it and how it is sustained.

From the perspective of implementation science, the SMHCI can be understood as a guiding tool that facilitates decision-making in institutional improvement processes. Its use enables the establishment of baselines, the identification of organizational readiness gaps, and the alignment of technical assistance with each school’s level of development (Fixsen et al., 2009; Moore et al., 2024). This type of diagnostic tool is especially relevant in settings where public policies promote multi-tiered support models without necessarily providing mechanisms to monitor their degree of implementation or effectiveness.

Regarding the psychometric properties, most items showed satisfactory factor loadings. However, two items (B4 and C7) displayed loadings below 0.40, suggesting the need to review their wording and contextual relevance. It is recommended that future studies incorporate qualitative techniques, such as interviews or focus groups, to explore how these items are interpreted by school staff. The continuous improvement of the instrument should aim not only to refine its statistical performance but also to ensure its applicability across diverse contexts, including rural, intercultural schools or those with high staff turnover.

This study has some limitations. The sample was obtained through purposive sampling within the framework of a training course, which may introduce a self-selection bias. Additionally, the response scale was reduced from five to three points, which could limit the instrument’s sensitivity in distinguishing intermediate levels of organizational capacity. Other forms of validity—such as criterion-related validity or measurement invariance across groups—were not assessed and should be addressed in future research.

Based on these findings, three lines of future development are identified. Firstly, exploring the relationship between SMHCI scores and relevant school outcomes, such as student well-being, school climate, or teachers’ perceptions of their preparedness in mental health. Secondly, validating the instrument into larger and more representative samples, including technical-vocational schools, rural schools, and those in under-resourced areas. Thirdly, expanding the scope of the instrument to also assess the well-being of teaching and support staff, whose emotional burden directly affects the sustainability of school mental health programs.

Taken together, the results of this study contribute to the development of a more robust model for school mental health in Chile—one in which the assessment of institutional capacities is integrated with processes of continuous improvement, technical assistance, and intersectoral coordination. The implementation of the SMHCI offers an empirical and context-sensitive foundation to strengthen public policies in this area and to reorganize institutional work around a logic of care, prevention, and educational justice.

Conclusion

The present study provides empirical evidence supporting the validity of the School Mental Health Capacity Instrument (SMHCI) in the Chilean context, highlighting its usefulness in assessing schools’ organizational capacity to promote school mental health and educational coexistence. The findings suggest that the instrument’s three-dimensional structure is appropriate, distinguishing promotion and prevention, referral and support, and crisis intervention as fundamental components of effective management in this field. Although some opportunities for improvement were identified in the precision of specific items, the evaluated model offers a solid foundation for the design and implementation of evidence-based strategies within the Chilean education system.

From an applied perspective, the validation of this instrument represents progress in identifying gaps and needs related to schools’ organizational readiness. Its implementation can contribute to strengthening institutional capacities to effectively respond to school mental health challenges by providing key information for decision-making in educational policy. However, the effectiveness of such tools depends on their integration into broader institutional improvement processes, which implies the need for technical support, teacher training, and intersectoral coordination with the health system (Durlak et al., 2011; Fixsen et al., 2019).

This study also underscores the importance of addressing school mental health through a systematic and sustained approach, avoiding fragmented or reactive strategies. The consolidation of an effective care model in this area requires not only diagnostic tools such as SMHCI, but also institutional commitment to the development of preventive strategies and the creation of support networks both within and beyond the school setting. In this regard, the results obtained can serve as a resource for the implementation of school mental health programs and the strengthening of public policies aimed at building schools’ organizational capacity in mental health and coexistence.

In conclusion, while this study represents a significant step in validating a specific instrument for assessing schools’ organizational capacity in mental health, further research is needed to refine its application and adaptability across different educational contexts. Future lines of research could focus on the relationship between institutional capacity indicators and outcomes in student mental health assessments, the impact of intervention programs aimed at strengthening organizational readiness, as well as the organizational capacity profiles of Chilean schools in addressing school mental health challenges. Additionally, it may be relevant to incorporate a broader perspective within the questionnaire—one that evaluates not only student mental health, but also the well-being of all members of the school community, including teachers, educational support staff, school leadership teams, and families. These efforts can contribute to advancing a more robust and sustainable model of school mental health in Chile.