Assessment of the Effectiveness of a Skills Program in Emotion Perception and Understanding for School-age Children: A Pilot Study

Introduction

An individual’s set of emotional and social adaptation skills is considered a comprehensive part of their emotional intelligence (EI) and is of vital importance for different aspects of the whole-person development (Segundo-Marcos et al., 2022). In recent years there have been studies that show the important role played by emotional skills in psychological well-being; they influence people’s adaptation to different life contexts and how they interact with others (Durlak et al., 2011; Extremera & Fernández-Berrocal, 2013; Joseph et al., 2015). In a broad sense, EI is considered to be trainable, and its development improves emotion awareness, identification and regulation; this in turn supports more effective resolution of specific situations, improves decision making, and facilitates more adaptive interaction with people and the environment (Mercedes et al., 2020; Torres et al., 2021).

The pedagogical movement of emotional education has emphasized the importance of working on these skills at school, thus prompting the appearance of numerous specific programs (Acosta, 2008; Bisquerra, 2008, 2011). Program application has demonstrated improvements in students’ social skills (Torres et al., 2021), academic achievement (Hernández, 2020), self-regulation and self-esteem (Kumar et al., 2019), positive attitudes (Usán et al., 2020), life satisfaction (Lopez et al., 2019), motivation (Vaquero-Solís et al., 2022), adaptability and participation in school (Guerrero & Reyes, 2020), reduced bullying behaviors (Ubago-Jiménez et al., 2021), and increased empathy (Gómez & Narvaez, 2020). Other studies also highlight improvements in teacher performance (Anwar et al., 2021). In addition, this type of emotional skills training is closely related to other social and psychological variables like improved communication, managing anxiety, reduction of violent behaviors and, consequently, living together better in different contexts (Gómez & Narvaez, 2020; Guerrero & Reyes, 2020; Hernández, 2020; Kumar et al, 2019; Llorent García et al., 2020; Lopez et al., 2019; Torres et al., 2021; Ubago-Jiménez et al., 2021; Usán et al., 2020; Vaquero-Solís et al., 2022).

Programs that present rigorous methodologies and are based on consolidated theoretical frameworks are considered to be programs with greater empirical support (Durlak et al., 2011; Fernández-Berrocal et al., 2015; Pérez-González, 2008). In this regard, two of the most well-grounded models are the skill model and the mixed model (Fernández-Berrocal et al., 2017; Mayer & Salovey, 1997). The present study took the Mayer and Salovey (1997) skill model as a reference, where EI is defined as a type of intelligence made up of four basic processes: perception and expression of emotions, emotion as facilitating thought, understanding and knowledge of emotions, and emotion regulation.

In the international sphere, programs based on the Social and Emotional Learning (SEL) approach or RULER (Recognizing, Understanding, Labeling, Expressing, and Regulating emotion) have gathered substantial empirical support in the social-emotional development of mainly primary and secondary students (Baumsteiger et al., 2022; CASEL, 2005; Hayashi et al., 2022; Muratori et al., 2022; Reyes et al., 2012; White et al., 2022). In the Spanish context, there are programs applied in primary education that work on components such as emotion identification, emotional understanding, communicating, and managing emotions (Berrios & Martos-Luque, 2016; Celdrán & Ferrándiz, 2012; Merchán et al., 2014). The INTEMO and INTEMO+ programs are designed for adolescents and are based on the skill model; they have reported empirical evidence in improved self-esteem, better prosocial attitude in the classroom, reduced anxiety and reduced stress after the intervention (Cabello et al., 2016; Cabello-Porras & Fernández-Berrocal, 2022; Ruiz-Aranda et al., 2013; Ruiz-Aranda, Salguero, et al., 2012).

All these programs have been applied in formal educational contexts, but if we look within Non-Formal Education (NFE), there are few studies to date that address programs carried out in real life or non-formal contexts (De Benedictis et al, 2019; Górriz & Ordoñez, 2014).

NFE is defined as any systematic, organized activity of an educational nature, not constrained to any fixed progression, carried out in a context other than the official educational framework, and promoting teaching and learning at all stages of life, complementing formal education (Caride, 2020; De Benedictis et al, 2019; UNESCO, 2012). Educational programs for adults, community programs on different topics (health, family planning, etc.), training programs, programs in emotional development, associations with educational purposes, summer schools, camps, and many other activities (Pastor, 2001, UNESCO, 2012) are included in the NFE framework. In this regard, NFE favors inclusion and attention to diversity, and is based on a flexible, innovative educational model (Ferrer-Fons et al., 2022). Márquez-García et al. (2021), however, stress the need to encourage a more direct connection between formal and non-formal education in interventions targeting emotional, interpersonal, and values-related aspects.

Despite the rise in program elaboration and application, based on these models, there is general recognition that not all programs have solid evidence to back their effectiveness.

Considering the previous literature, there are few validated programs targeting school-age children that work on emotional variables from the skills approach, and even fewer studies that measure effects in NFE contexts (Bond & Manser, 2009; De Benedictis et al, 2019; Pena & Extremera, 2016).

Educational environments influence multiple aspects of student development, such as psychological well-being, learning emotional competencies and social interactions; therefore, most research focused on this influence was conducted in formal educational contexts. However, formal and non-formal educational settings have different characteristics and hence may have different effects on development (Madjar & Cohen-Malayev, 2013).

Madjar and Cohen-Malayev (2013) suggest several characteristics of non-formal education that differ from formal education contexts. First, a characteristic of NFE is the willingness to join or leave the activity. This characteristic favors participants’ autonomy and provides opportunities for self-expression. Moreover, in this context organized activities are mainly aimed at enjoyment, which allows for a more flexible and autonomous educational environment compared to formal educational contexts. Secondly, in non-formal contexts it is possible to stablish a variety of activities that require different skills, including social and emotional skills, leading to the exploration and practice of these skills. Moreover, formal educational contexts focus mainly on academic skills and performance, promoting activities that allow for fewer social interactions between peers. Thirdly, in NFE, interactions are usually symmetrical among participants and therefore reciprocal and equal relationships are established, i.e., members are equal and they all agree on a set of rules and codes of behavior that make up an organized but egalitarian social structure. Formal educational contexts often have a hierarchical structure, and students are subordinate to the instructions and decisions of teachers. Finally, in NFE the sense of experimenting and interacting through trial and error is legitimate and even encouraged, whereas in formal contexts this mode of interaction is often more limited.

In conclusion, the main characteristics of NFE favor a flexible space for interaction where personal, social, and emotional skills, which in a formal context are usually relegated to the secondary place, can be put into practice. Therefore, it is reasonable to expect that non-formal education will be perceived differently and will have a different effect on these developmental processes (Madjar & Cohen-Malayev, 2013).

According to Manolescu et al. (2018), non-formal education represents any intentional and systematic educational activity usually conducted outside traditional school (formal education), whose content is adapted to individual needs and circumstances, with the purpose of maximizing learning and knowledge and minimizing the problems faced in the formal system.

In this line, the aim of the present pilot study was to evaluate the effectiveness of a skill development program in emotion perception and emotional understanding for school-aged boys and girls, designed from the skill-based model, and applied in an NFE context.

Method

We used an experimental design with subjects randomly assigned, pre-post repeated measures, and a group control (Montero & León, 2002).

Participants

The sample was composed of 22 subjects from 6 to 12 years of age (6 boys and 16 girls) (M = 8.55, SD = 1.76), who attended a summer school in Jaén, Spain. The subjects were divided into two age groups following the age assignment criteria, one group with 12 subjects (9 female, 3 male), ages 6-8 years (M = 7.2, SD = .75), and another group with 10 subjects (7 female, 3 male) ages 9 to 12 (M = 10.10, SD = 1.28). Subsequently, participants in each age group were randomly assigned to the experimental and control groups. Table 1 summarizes characteristics of the sample.

Instruments

We applied an adaptation of the skill tests from the Assessment of Children’s Emotion Skills-ACES (Schultz & Izard, 1998), which evaluate the skills of emotion perception and understanding emotions in 6-to-9-year-old children. This tool is administered individually and includes three subscales: a) Facial Expression, b) Social Situations, and c) Social Behaviors. Subscale A, Facial Expression, assesses the ability to perceive emotions conveyed by the facial expressions of other boys and girls. Subscale A contains 26 photographs that represent the basic emotions of happiness, sadness, anger, and fear, as well as ambiguous expressions that do not convey specific emotions. Subscales B (Social Situations) and C (Social Behaviors) contain 15 sketches each (total 30), with two or three phrases that describe situations and behaviors related to the same basic emotions (6 are ambiguous or neutral).

The original version, in English, has not been translated or adapted to Spanish, and we could not identify any other standardized skill test in Spanish that assessed these skills in school-age children. Therefore, one of the limitations for scientific development in this line of research lies in the lack of standardized instruments in Spanish aimed at schoolchildren (Fernández-Berrocal & Extremera, 2016; Montes Espinosa et al., 2017; Moreno et al., 2024).

Similarly, there is a great difficulty to find scientifically validated measurement instruments that fit the range of 6 to 12 years of age and that are useful for measuring emotional perception and understanding from Mayer and Salovey’s (1997) model. This is logical, since the natural evolution of emotional skills is largely related to the evolutionary development of children and changes over time, so that the ability to recognize, discriminate, and categorize basic facial and emotional expressions related to situations and behaviors appears throughout childhood and undergoes a gradual progression (Loeches et al., 2004; Navas et al., 2011). Thus, younger children tend to have more basic emotional skills and older children usually have more complex emotional skills, such as assimilation, understanding, and regulation of emotions (Mayer & Salovey, 1997). Based on these arguments, studies with school-age populations usually design specific ad-hoc ability tests or use tests of emotional recognition and understanding, taking other instruments as a reference (Ambrona et al., 2012; Celdrán & Ferrándiz, 2012; Márquez & Delgado, 2012), in which examples appropriate to the characteristics and level of understanding of the participants (school age, level of evolutionary development, etc.) are applied (Ruiz-Aranda et al., 2013).

Therefore, we carried out an adaptation of the key elements of the ACES. The following list are the adaptations made for this study:

• In the three subscales, ambiguous images and phrases were removed, since the purpose of these items (studying students’ tendency to attribute the emotion of anger) was beyond the aims of the present study. Consequently, the subscales of social situations and behaviors, targeting children from 6 to 9 years, contained a total of 24 items.
• In subscale A, Facial Expression, we added more complex emotions of surprise and disgust to be applied to school children from age 9 to 12, following the line of other research (Celdrán & Ferrándiz, 2012; Ekman & Friesen, 2003).
• The written statements in subscales B (Social Situations) and C (Social Behaviors) were translated using a process of inverse translation followed by peer review, where bilingual experts participated (two school teachers and a psychologist). Cases involving doubt were resolved by consensus.
• A new, ad-hoc subscale was elaborated, targeting children from age 9 to 12, named Understanding Emotions: 9-12 years. Nine statements were added, three for each emotion representing more complex social situations like embarrassment, pride or shame, so that after removing the ambiguous expressions the subscale contained 21 items. These were designed by the same experts, evaluated and selected through a peer review process and agreements established by consensus. The following is an example of the statements added: “José got a 10 on his test. José had studied hard and thinks he deserves it. How does José feel: afraid, angry, embarrassed, sad, proud, guilty or happy?”.

This version of the adapted test is administered individually and may require about 5 to 10 minutes for each subscale. The score is calculated as the number of correct answers over the total options presented.

The adapted version of this instrument was elaborated for this pilot study, and its consistency and reliability have not yet been tested; the reliability data of the original ACES test were therefore taken as a reference (α = .68, α = .71, and α = .75), where internal consistency was acceptable (Schultz et al., 2004; Trentacosta & Izard, 2007; Trentacosta et al., 2006). Nonetheless, studies that have applied the ACES test as a measure of understanding emotions, adapting it to their culture and mother tongue, have obtained high reliability indices (Myles-Pallister et al., 2014; Santos & Langil, 2019; Ünal Bozkan & Kömleksiz, 2014).

In addition, a questionnaire was designed to collect sociodemographic data, recording variables such as gender, age, grade in school, parents’ work, number of siblings, socioeconomic level, presence of psychological or learning problems, and whether or not they received support at school.

Procedure

First, the required permissions for carrying out the research study were obtained from the institutions involved. The whole process was carried out according to the ethical norms of the institutional research committee. A meeting was held with parents, where they were informed about the objectives and characteristics of the study, and signed consent was obtained for their children’s participation.

The following inclusion criteria were established to recruit subjects:

• Who were regularly enrolled in primary education boys and girls aged 6 to 12.
• Who were attending the summer school for at least two months.
• Who were regularly enrolled in primary education.

Exclusion criteria were also defined:

• Boys and girls under 6 and older than 12.
• Boys and girls not enrolled in school.
• Subjects who attended the summer school ocassionally or for less than 2 months.
• Subjects with a diagnosis of severe behavior disorders, learning disabilities, or neurodevelopmental disorders.

Based on these criteria, 22 subjects were selected. The following procedure was carried out to assign the 22 subjects to the different experimental conditions: participants were first divided according to age, making two groups, age 6 to 8 (n = 12) and 9 to12 (n = 10). Next, members of each group were randomly assigned to the conditions of experimental group (EG) or control group (CG), respectively. (See Table 1).

The experimental groups were designated for administration of the skill development program in emotion perception and emotional understanding, which was carried out as a complement to the summer school activities. The latter activities involved recreation, sports, and leisure. The control groups participated only in the conventional summer school activities.

Intervention

A psychoeducational and skills-training program was designed, based on Mayer and Salovey’s (1997) skill model, adapting key components from a consolidated program, the INTEMO program (Cabello-Porras & Fernández-Berrocal, 2022). From the INTEMO Program, which targets adolescents, we adapted mainly the modules of emotion perception and understanding emotions (Ruiz-Aranda et al., 2013). Two experts in developmental psychology and in education revised and redefined the characteristics of the modules to adjust them to the objectives and context of this study. In addition, the content and sequence of the activities were reorganized to adjust them to the participants’ level of development. The results of this process were collected in an application manual (Montes-Espinosa et al., 2017) that was used as a guide to administer the sessions, and would facilitate replication of this study.

The intervention timeframe consisted of a total of 10 sessions, the first two and the last two were devoted to subject assessment, given that the skill tests required sufficient time for their application. The other six sessions contained the intervention per se.

The program was divided into two phases, with three sessions of one hour each. Phase 1, targeting skill development in “emotion perception and expression”, consisted of identifying and recognizing one’s own emotions and others’, paying attention, and decoding emotional signs in the face, body, or tone of voice. Phase 2 addressed “understanding emotions”, which trained skills to capture emotional information, understand the causes and consequences of mood and how they are combined.

To assess program effectiveness, the instrument was applied at two times, pre- and post-intervention phases. The six-to-eight-year old were administered the adaptation of the ACES test subscales on facial expression, social situations, and social behaviors. The nine-to-twelve-year old were administered the adaptation of the facial expressions subscale of the ACES test, and the ad-hoc subscale of understanding emotions: 9 to12 years old. Figure 1 illustrates the procedure.

Statistical Analyses

A comparative analysis of the collected data was carried out using SPSS version 25.0 (IBM, 2017). To test for data normality in each group, the Shapiro-Wilks test was applied for samples with fewer than 50 subjects. The Levene’s test was conducted to determine homogeneity of variances. Likewise, a within-group analysis was conducted using the Student’s t-test for related samples, between pre-test and post-test phases in both age groups (in both the control group and the experimental group). Subsequently, a between-groups analysis was conducted, where a one-way ANOVA was applied after the intervention (post-test) to test for differences of means in the variables of emotion perception and understanding emotions, between the experimental and control groups (both age groups). In the same way, an ANCOVA test was conducted to test whether the intervention had had the intended effect, and which possible changes were produced in each of the variables measured after the intervention (post-test), comparing the experimental and control groups and taking the scores obtained on these same variables prior to the intervention (pre-test) as covariable. Finally, to determine the effect size, Cohen’s d was calculated, incorporating the values of means, standard deviations, and sample size of both groups (experimental and control) in both pre-test and post-test phases.

Results

First, normality of the data behavior was tested by applying the Shapiro-Wilks test to the general scores on the adapted ACES subscales, which measured total emotion perception and understanding emotions in all groups. All results yielded values greater than p > .05. To determine homogeneity of the variances, Levene’s test was applied, yielding results greater than p > .05, assuming similarity of variances in both groups. After verifying the criteria of normality and robustness in the data behavior, and homoscedasticity of variances, analyses were carried out for comparisons of means and effect size.

Emotion Perception and Understanding Emotions in Experimental Groups

To test for any possible change in the experimental groups in the variables of emotion perception and understanding emotions, a Student’s t-test for related samples was used following program application. In this way, the mean scores obtained in the pre- and post-intervention phases were compared, and the significance of these differences was established within each of the two groups (6-8 years, 9-12 years).

Participants in the two age groups obtained significantly lower overall scores in emotion perception and understanding emotions before the intervention than afterward, 6-8-year group: t(5) = -6.984, p < .05; 9-12-year-group: t(4) = -2.871, p < .05. A significant increase in both variables was found in most of the adapted ACES subscales after the intervention. Participants in the six-to-eight-year-old group presented significant changes on the facial expressions subscale (p < .05) and social behaviors subscale (p < .01), while the nine-to-twelve-year olds showed changes on the scale of understanding emotions: 9 to12 years old (p < .001). (See Table 2).

Emotion Perception and Understanding Emotions in Control Groups

To test for any possible change in the control groups in the variables of emotion perception and understanding emotions, mean scores obtained in pre- and post-intervention phases were compared. The results indicated that there were no significant changes in participants’ total score or subscale scores (Table 3).

Emotion Perception and Understanding Emotions, between-Group Comparison

One can see in Table 4 for the age group 6 to 8 that the mean scores of pre-intervention tests showed no significant between-group differences in the study variables. However, in post-intervention tests, the experimental group obtained higher scores (higher means) than the control group both in emotion perception and in understanding emotions. These means differences were analyzed using a one-way ANOVA, where the results yielded statistically significant differences in both variables. Finally, an ANCOVA test was conducted. Results revealed that for total scores in the age group of 6-8 years, participants who received the intervention got significantly higher mean scores in all the study variables after the intervention, compared to those who did not participate in the program, with a large effect size in favor of the treatment group. The intervention, therefore, proved effective.

In the same way, as seen in Table 5, the same analyses were carried out for the control and experimental group with ages 9 to 12, replicating the above results: no statistically significant differences were found in pre-intervention tests, but were found in post-intervention tests, with the experimental group obtaining higher means after the intervention, this difference being statistically significant. The magnitude of the effect was large, in favor of the treatment group, for total scores in the 9 to12 age group.

Discussion

The present study was able to check that the application of the skill-model based program improved the capacity to perceive, express, and understand emotions in school-age boys and girls.

The pre-post analysis of comparison of means in the control group substantiated that skills in both emotion perception and understanding emotions remained stable in boys and girls in this group. These data agree with other studies, such as the INTEMO Program in adolescents (Ruiz-Aranda, Castillo, et al., 2012), and Low et al.’s (2015), Merchán et al.’s (2014) and Wong et al.’s (2014) with primary school children. Considering that the scores of subjects who did not receive a specific intervention did not vary, instruction may be considered to play an important role for learning emotional skills and competencies (Ruiz-Aranda et al., 2013, Ruiz-Aranda, Castillo, et al., 2012). However, the results obtained in the study by Celdrán and Ferrándiz (2012) are worth analyzing, where there was a significant increase in emotional perception in the post-test scores of the control group, in children aged 6 to 7 years. In this case, exposure to the tasks and material from the initial assessment might have favored learning or recall in the control group, improving their later performance. By contrast, in the present study, though similar skill tests were used, we observe even a slight decline in the post-intervention scores of the control group, ages 6 to 8. In Celdrán and Ferrándiz’s (2012) study two items were used to assess each emotion, while in the present study three items were used for each. The greater number and diversity of situations and sketches presented here may have reduced the test-retest bias in the control group, making recall less likely. Therefore, the characteristics of the assessment instruments might have an influence on the results obtained when evaluating children from ages 6 to 8.

The mean comparison analysis showed a significant increase in the post-test scores of the experimental group in the variables of emotion perception and understanding emotions, both in the computed total score of the adapted ACES test and in each of the subscales. These results would indicate that this six-session intervention was effective for learning certain specific skills such as naming, expressing, and recognizing different emotions in themselves and in others, as well as improving understanding of the causes and consequences of multiple mood manifestations in various contexts. Moreover, in the 9 to 12 age group, the intervention was effective for increasing more complex skills, such as understanding complex emotions (e.g. envy, embarrassment, pride), understanding the combination of these, and the relationship between thoughts and specific emotional states. These findings are in line with several previous studies (Aguilar et al., 2021; Celdrán & Ferrándiz, 2012; Echeverría et al., 2020; Merchán et al., 2014).

A comparative analysis of means between the experimental and control groups and calculation of the magnitude of the differences substantiated that the children who participated in the program achieved significantly higher scores in emotional skill tests than children who did not participate in the program, with a large effect size in favor of the intervention.

According to different authors, focusing on specific objectives and skills, along with having a sufficient number of sessions, can increase the effectiveness of the intervention (Aguilar et al., 2021; Durlak et al., 2011; Evans et al., 2015). In our case, the intervention focused only on specific skills. Durlak et al. (2011) identifies different elements that may mediate the effect of interventions, some of these modulating characteristics being found in the proposed program. On one hand, it is based on a solid theoretical model and has been manualized to encourage its systematic use and implementation. On the other hand, the tasks are designed sequentially, correspond to the children’s level, and focus on specific aims, explicitly targeting improved emotional skills. In addition, active methodologies such as role-playing or rehearsing behaviors have been applied. These characteristics may have modulated the effect of the intervention, contributing to its effectiveness.

By way of conclusion, it is important to consider that there are few studies based on the skill model that demonstrate effectiveness; particularly with a Spanish sample, there are few studies in a primary school population (Acosta, 2008; Bisquerra, 2008; Bond & Manser 2009; Pena & Extremera, 2016; Ruiz-Aranda et al., 2013). Along these lines, studies carried out in a context of non-formal education are even less frequent (De Benedictis et al, 2019).

Therefore, the present study can help demonstrate that brief training programs focused on specific skills may be effective for improving the ability to understand, express, and regulate emotions in school-aged children in NFE contexts (Ferrer-Fons et al., 2022). In addition, it can serve as a reference for the design of other similar interventions that help connect the educational processes of formal and non-formal education (Márquez-García et al, 2021).

Limitations

This pilot study used an adapted version of the ACES test; while the adaptation aims to be rigorous and systematic, it has not yet been validated, thus constituting a limitation. Generally speaking, the lack of EI skill tests for school children in Spanish has led to the use of questionnaires or application of ad-hoc instruments. Accordingly, validation of the adapted ACES test with a Spanish-speaking population between the ages of 6 and 12 years, can be a line of development for future research studies. Another limitation of the present study is the small sample size used and the limited number of variables of interest that could be analyzed statistically, so it is suggested for future studies to increase the number of participants in order to be able to carry out more complex statistical analyses that include variables such as sex or age.

Other future lines of research could focus on increasing evidence on the benefits and positive consequences of emotional skills training in non-formal educational contexts and its influence on other psychological, social, and educational variables (assessing interpersonal relationships, academic performance and participation, motivation, self-esteem, reduction of violent behavior, among others). Furthermore, it would be relevant to obtain data on the long-term persistence of these benefits due to the intervention applied.