While modern society is nowadays considered as an information society, we are still faced with achieving what is called a knowledge society that encourages collaboration between communities and organizations. Such a society’s purpose should be to improve living standards by managing and co-creating knowledge from different sources, based on information and communication technologies. Therefore, it becomes necessary to provide the means for a socioformative learning approach to nurture citizens, teams, and communities engaged with solving social issues and improving living standards. This should be based on an ethical life plan, entrepreneurship, collaborative work, the co-creation of knowledge, metacognition, and complex thinking (Tobón et al., 2018).

The purpose of the socioformative approach is to transform traditional—and ineffective—methods into suitable pedagogical practices designed for human and social development which promote both inclusion (Castro-Rubilar et al., 2017) and creativity (Karwowski & Lebuda, 2017). The core ideas of the socioformative approach are 1) project-based work intended to solve problems in a real context and contribute to improving living conditions; 2) progressive strengthening of an ethical life plan, as in a community contribution towards sustainable development by implementing such universal values as responsibility, respect, equity, and honorability; 3) development of complex thinking skills, such as systemic thinking, flexibility, and metanoia; 4) collaborative knowledge building through searching, systematizing, critically analyzing, creating, adapting, and implementing different types of knowledge; and 5) collaborative work focusing on achieving common goals related to social development and the articulation of different people’s actions (Fuentes-Arismendi & Malagón-Micán, 2019; Prado, 2018).

After years of research on teachers’ performance in terms of competencies, there has been a recent shift toward the concept of practices (Bolívar, 2019). A practice represents a more concrete realization of what a competency expresses. Practices are comprised of knowledge, abilities, and values (Gómez-Hurtado, 2013) but include other elements, such as goals aspiration, decision making, teachers’ professional identity, and emotional management (Bolívar & Ritacco, 2016; Burbules, 2020). Specific practices are not only more visible than competencies, but might also patently show how performance is innovative, sustainable, replicable, and suitable for specific needs depending on the context (Noman et al., 2018; Reyes, 2017). Furthermore, practices are without negative connotations (as is the case with competencies), especially within certain social sectors which link this term to neoliberalism (Angulo & Redon, 2011; Pérez et al., 2019).

Therefore, pedagogical practices are defined as actions that help students fulfill their learning process through different teaching methods and learning strategies (Álvarez-Álvarez, 2015). Due to their impact on the learning process, they must be regularly tested (Chrispeels et al., 2007). From a socioformative approach, pedagogical practices are actions through which the students learn problem-solving indirect contexts to improve living conditions, and strive toward sustainability and inclusion by collaborating with everybody both in school and around it: teachers, authorities, parents, communities, and other students (Tobón et al., 2018). This implies the collaborative creation and management of knowledge-based on valid sources. This represents a significative change to the concept of pedagogical practices, traditionally taken as simple tasks to be done by teachers. They are now understood as actions that impact citizens’ formation toward a knowledge society (Mansell, 2015).

For a better understanding of pedagogical practices and their relationship with social challenges, it is possible to distinguish four levels.

Level I refers to general pedagogical practices that must be executed by every teacher and are more related to what every teacher ordinarily does, with no regard for any specific educational model or an ideal society to achieve.

These practices are, for example, lesson planning, teaching, tutoring, learning resources managing, question answering, helping students to solve problems, and learning assessment. At this level, what is important is that teachers accomplish their duties no matter how much students learn.

Level II practices are those based on autonomous, meaningful, and reflexive learning, as the ones promoted by Ausubel’s (1963) meaningful learning and constructivism (Piaget, 1953). This level represents a step forward from Level I, as long as these practices are intended to lead learning toward situations that enclose real significance to students.

Level III refers to pedagogical practices based on creativity, pair work, collaboration, the linkage between technology and learning at different environments, as proposed by approaches such as socioconstructivism (Vygotsky, 1978), connectivism (Siemens, 2006), and invisible learning (Cobo & Moravec, 2011). At this level, the focus goes beyond the student himself and widens so to contemplate how context and social environment determine student’s learning. Level IV refers to pedagogical practices directed towards sustainable social development (Lee & Ávila García, 2019; Servín-Ramírez & Vázquez Sánchez, 2019) and are decisively oriented to train students to solve real-life problems and impact positively in every effort society makes in favor of environmental care, biodiversity conservation, as well as fostering a solidarity culture. The latter is something Level III practices do not consider at all. Level IV practices have been proposed by socioformation, for instance, real-life problem solving sustainably, inclusion, learning through service, human talent development through formative projects, transversality, and ethical project of life. Each of the four levels mentioned before includes those basic practices in previous levels. Therefore, in Level IV there are pedagogical practices from levels I, II, and III, while problems society faces nowadays, such as climate change, fake news, violence, and COVID-19 pandemic, are faced only through new pedagogical practices like those in Level IV.

The assessment of pedagogical practices has received much academic attention (Bell et al., 2012; Gettinger et al., 2011; Hamre & Pianta, 2010; Seidman et al., 2014; UNESCO, 2017); however, the majority of these studies have focused on Levels I and II. One of the most common instruments is the Classroom Assessment Scoring System (CLASS™) (Hamre et al., 2013; Pianta, 2011), a tool to observe teachers’ actions inside the classroom according to ten activity dimensions each teacher should perform within three latent factors: emotional support, instructive support, and classroom management (Hafen et al., 2015). Another assessment instrument, the Stallings Observation System (World Bank, 2015), focuses on observing classroom activities for short periods, especially time management and student learning activities. The Teacher Instructional Practices and Processes System (TIPPS) (Seidman et al., 2014) enables an 18-dimension assessment on how teachers work inside the classroom, evaluating certain actions regarding meaningful learning, such as “teacher employs students’ ideas and interests” or “teacher connects syllabus contents with students’ everyday life experiences”.

In recent years, new instruments have been developed and validated in Mexico to assess the pedagogical practices of basic education teachers (Arregui et al., 2018; Gutiérrez-Anguiano & Chaparro, 2020). These instruments have been adapted from Danielson’s Framework of Teaching (Danielson, 2013)—a very extensive four-dimension rubric type (planning and preparation, classroom environment, instruction, and professional responsibilities) taken as a basis for several studies in the USA (Finster & Milanowsky, 2018; Liu et al., 2019).

Generally speaking, it could be said that CLASS, the Stallings Observation System, and Danielson’s Framework of Teaching simply evaluate what every teacher does (Level I) without taking suitability and actual impact on pedagogic work into account. TIPPS, however, approaches meaningful learning actions (within Level II), but without considering specifically how teachers manage issues like problem-solving, inclusion, and project-based work (i.e., Level IV practices). In an Ibero-American context, certain assessment instruments have been proposed—most notably in universities—the majority of which focus on Levels II and III. Pérez et al. (2016) designed and validated the Teaching Practices Questionnaire (TPQ), a Likert type scale, to assess how frequently certain teaching activities are performed. These activities are organized regarding the following factors: 1) student-centered teaching; 2) teaching qualification; 3) process evaluation; 4) dialogic interaction; 5) instruction-centered teaching; and 6) technological resources employment. This instrument belongs to Level II due to its inclusion of student-centered teaching and meaningful learning, although it considers many Level I issues. Lizasoain-Hernández et al. (2017) designed an 18-item instrument to evaluate higher education teachers with a 3-factor structure: planning (items 1-5), process and development of teaching activities (items 6-16), and outcomes (items 17-18). It includes issues related to Level II, such as formative assessment, teamwork, autonomous learning, and the fostering of critical and reflective thinking.

Another instrument, classified as Level II, is the Teaching Strategies for Meaningful Learning Scale (EEDAS, to use its Spanish acronym) (Méndez & González, 2011). It evaluates 12 dimensions: previous information generating activity; introductory focal activity; positive-negative-interesting; guided discussion; goals and intentions; key diagram; conceptual mapping; prior answer-question-later answer; what I know–what I want to know–what I learned; summary table; analogies; and summary. Although these instruments are closely related to meaningful learning, they do not consider how teachers include context problem-solving activities—or any other similar actions—to promote student participation in enhancing living conditions or environmental sustainability. At Level III, an instrument designed by Barahona (2004) can be used to assess teachers’ creativity and teaching style. It has a 6-level scale and contains 48 items—none of which relate to real problem-solving, environmental care, or the building of better living conditions. There are few instruments at Level IV, but the Inclusive Pedagogical Practices Scale by Carrillo et al., (2018) for elementary and high school is worth examining. It has satisfactory values of reliability, content validity, and construct validity. It is structured by 45 items which range from totally agree (TA) to totally disagree (TD). Other instruments exist which focus on certain practices related to specific educational aspects, such as promoting good human interaction and socioemotional abilities (Moafian et al., 2019; Laudadío & Mazzitelli, 2018), ensuring inclusive education for talented and gifted children (Dağlıoğlu et al., 2019), and fostering humanistic, equitable, ecological, and socially just environments (Nava, et al., 2018). While the majority of these instruments were developed and validated through rigorous and accurate validation processes, core instructional practices are excluded.

As such, there are no instruments at Level IV which focus on pedagogical practices to resolve real problems or foster an ethical life plan with transversality and innovation. Existing instruments to assess pedagogical practices are, in general, either scales or registers of observation. While these can be useful as they allow a quick and easy evaluation of what occurs within the classroom, interpretations for each level may differ between evaluators (Marin-García & Santandreu-Mascarell, 2015). Moreover, scales do not allow for a precise grading of the exact level at which a specific practice is performed by teachers regarding knowledge society and sustainable social development purposes (USAID, 2010). It is, therefore, necessary to design other types of instruments (rubrics, for instance) as they facilitate teachers’ self-assessment of the level at which they are performing and the identification of improvement opportunities and needs in terms of pedagogical practices.

To fulfill the existing lack of instruments with which to assess Level IV pedagogical practices, a Pedagogical Practices Rubric SOCME-10 has recently been designed (Tobón et al, 2018) using the socioformative approach (see Appendix). This instrument is intended to enable self-assessment of teachers regarding 10 core pedagogical practices that contribute to socially sustainable development in the knowledge society. It has criterion validity for high school teachers in Mexico from an exploratory factor analysis that gave a single factor solution and a high-reliability level.

However, as yet there is no valid evidence of SOCME-10 for entry-level teachers of basic education. Such groups—having little teaching experience—need tools to self-assess their pedagogical practices to improve their teaching and consolidate their professional profile. According to several studies, a lack of support for novice teachers produces a significant rate of dropouts (Feroli, 2015; López et al., 2018). Socioformation is designed to help young and new teachers strengthen the pedagogical practices required to participate in the worldwide social sustainable development effort (Cummings et al., 2018; De Freitas-Fernández, 2019; Juárez-Hernández et al., 2019). This paper hopes that SOCME-10 could benefit this purpose.

Therefore the objectives of this study were: 1) to review items and descriptors or the rubric SOCME 10 to adapt them for new entered teachers to the public education professional service in Mexico; 2) to determine the instrument content validity through evaluation by judges; 3) to identify understandability quality, suitability, and level of satisfaction on the instrument from users opinion; 4) to establish instrument construct validity; and 5) to determine rubric reliability.

The study evidence that the Rubric of Pedagogical Practices from a socioformative approach (SOCME-10) has adequate levels of content validity—as shown by the results of Aiken’s V coefficient (the values of suitability, understandability, and satisfaction of the rubric as rated by experts were above .80). The examiners were teachers highly experienced in didactics, learning assessment processes, design of evaluation instruments, and the socioformative approach. The analysis of construct validity through the EFA indicates the correspondence with the theoretical model (stipulated as one-dimensional). As such, the only identified factor served to explain over 45% of the variance and integrated all of the items—highlighting that they all presented factorial loads above the established minimum (FL > .350) (Hair et al., 2014). The above denotes the importance of the expert judgment phase since, as indicated, content validity is an essential component of construct validity (Haynes et al., 1995).

The evaluation of the model through the CFA provided elements with which to verify its adjustment to the data since optimal values were presented for the different indicators used (χ²/df, absolute adjustment, and incremental indexes). An important aspect contributed by this analysis is the values of standardized factor loadings (SFL > .50) and Cronbach’s alpha and composite reliability. This allows for the verification of the proposed model’s empirical sustainability. While the average variance extracted (AVE) is relatively low, the data obtained is generally suggestive of the solid psychometric properties of the questionnaire. There are two important considerations here: first, item factor weights and high levels of reliability allow us to trust instrument’s working despite the low AVE (this criterion has been used in previous studies; see, for example, Igbaria et al., 1995); second, the characteristics of the instrument—based on a rubric of pedagogical practices—imply that the items are inevitably broader than a typical item on a psychometric scale, which may serve to explain the low value of AVE.

Delving into this aspect, Fornell and Larcker (1981), as well as Hair et al. (2014), consider that to demonstrate the existence of convergent validity it must be met that average variance extracted is greater than .50, standardized factorial load of each item is greater than .50, and that composite reliability is greater than .70. Following the results presented in this study, it could be argued that there is no convergent validity since the stipulated value of average variance extracted is not met. However, Moral de la Rubia (2019) states that the greater the number of items that make up a factor, the more the convergent validity measured through average variance extracted deteriorates, for which he recommends considering a mean extracted variance value adjusted to the number of items, in conjunction with the stipulated values for standardized factor loads and composite reliability. According to this proposal, the average variance extracted adjusted to the number of items in the present study is acceptable if it is greater than .300 and composite reliability greater than .80 to demonstrate an acceptable level of convergent validity. For both aspects, as well as related standardized factor loads, these guidelines are met, therefore it can be indicated that there is convergent validity, meaning that the construct is adequately measured by the proposed indicators (Cheung & Wang, 2017). In any case, the results were considered satisfactory for the first version of the questionnaire, although future studies could address reported weaknesses.

The single factor SOCME-10 evaluates (identified through EFA and CFA) has been labeled “mediation of problem-based training, collaboration, and inclusion.” Considering teachers’ work, each of the 10 practices contributes to this factor. Socioformation maintains that mediation goes above and beyond teaching or instruction (Brito-Lara et al., 2019), and consists of accompanying and supporting students so they learn problem-solving skills while working within an inclusive, collaborative, and ethical framework (Bautista et al., 2019). This framework implies participating in every social effort to improve living conditions and better care for the environment and natural biodiversity.

Users rated the rubric with a mean over expected (3.0) in the variables of suitability, understandability, and satisfaction. These aspects are rarely evaluated in similar instruments (see, for instance, Abós-Catalán et al., 2018; Gutiérrez-Anguiano & Chaparro, 2020), which means a relevant contribution of this study to the area. In this regard, it has been possible to establish measuring satisfaction about a scales is important for an assessment tool designing process (Seidman et al., 2018), because, although instruments show high validity and reliability values, frequently they are not suitable for users. Consequently, it can be concluded that SOCME-10 is an instrument that users validate as acceptable to be employed to help everyday practice, self-assessment, and continuous training of new teachers who join teaching service.

Several tools designed for the assessment of pedagogical practices have been mentioned in the relevant literature, which can be classified according to the level of articulation with challenges related to a knowledge society (Mansell, 2015). For instance, at Level I (regarding what every teacher must do) a significative number of instruments can be found (Hamre et al., 2013; Pianta, 2006; World Bank, 2015); at Level II (focusing meaningful and autonomous learning), only Seidman et al.’s (2014); at Level III (including creativity as well as social relationship), Barahona’s (2004); and at Level IV (highlighting the determining actions required to achieve sustainable social development, such as creativity and inclusion), Carrillo et al.’s (2018). Very few instruments belong to Level IV. Consequently, SOCME-10 rubric can contribute to the increase of instruments in this area that guide teachers in the transformation of education according to the challenges of sustainable social development (Luna-Nemecio, 2019).

SOCME-10 rubric contains some innovative elements regarding the construct it evaluates. For example, it takes practices as units of analysis, which is a more visible, concrete, and integrative professional trait to be assessed (Burbules, 2020; Pagés & Parcerisa, 2018). It also focuses on those actions essential to preparing citizens for a knowledge society, such as problem-solving (Hernández et al., 2018; Safiee et al., 2018), collaboration (Vázquez et al., 2017); values, an ethical life plan (Bautista et al., 2019; Prado, 2018), an interdisciplinary approach (Botero, 2008; Colombo & Alves, 2017; Díaz, 2010), creativity (Csikszentmihalyi, 2014; Karwowski & Lebuda, 2017), and metacognitive evaluation (Tobón et al., 2018).

Furthermore, this rubric has several innovative aspects: 1) the application of a socioformative taxonomy in the development of descriptors (Sánchez-Contreras, 2019), which provides teachers with a more precise and detailed tool to evaluate—and improve—their pedagogical practices (Seidman, 2012); 2) pedagogical practices are evaluated within a range of five levels, from a traditional content-focused practice to a more innovative educational practice, articulated with current and future social challenges (Mansell, 2015); and 3) the rubric seeks to contribute to training people committed to sustainable social development (Anastacio, 2020; Cummings et al., 2018; De Freitas-Fernández, 2019; Juárez-Hernández et al., 2018) and who have a key role in communities around the world—an objective that should guide current education and which is rarely evaluated in current scales of pedagogical practices.

As the present study was exploratory, it must be confirmed by applying the same tool in similar populations. Moreover, other aspects of its validity should be determined, such as consistency over time and concurrent and divergent validity of components with other tools or items already proven to be valid regarding pedagogical practices. Although the present study originally contemplated a factorial invariance measure, the characteristics of the sample used, with a high prevalence of women, did not allow this analysis to be carried out. Future studies with gender-balanced samples should also address this issue.

The rubric shows validity and reliability only for people who freshly got this occupation of teaching in Mexican schools; this means that its scope is rather limited and cannot be generalized to every basic education teacher. For this reason, further study is needed to determine its validity and reliability for other teachers despite the number of years in professional service. Another limitation of this study is that SOCME-10 psychometric properties as a rubric were not contrasted to those of other types of instruments, such as scales, which is an issue that might be deeply studied (Marin-Garcia et al., 2015).