Effectiveness of the Mantente REAL Program for Preventing Alcohol Use in Spanish Adolescents

Introduction

Alcohol Use in Spanish Youth

Substance use in young people is one of the problems that most concerns today’s society (Moreno et al., 2016). Of all substances, alcohol is the most socially accepted in daily life and the most prevalently used drug in Spain across all ages (Delegación del Gobierno para el Plan Nacional sobre Drogas [DGPND, 2017]). According to official reports in 2017, in Spain 36.5% of population older than 15 use alcohol habitually and 29% occasionally (Instituto Nacional de Estadística [INE, 2017]). Among youth aged 15-24, 36% report having been drunk in the last year (DGPND, 2017), 11.9% report an episode of heavy drinking in the last month (INE, 2017) and 7.2% who answered the AUDIT scale met criteria for high risk alcohol consumption (DGPND, 2017). More specifically to adolescents, results from the Drug Use Survey in Secondary Education in Spain (Observatorio Español de las Drogas y las Adicciones [OEDA, 2019]) indicate that most of those aged 14-18 have consumed alcohol at least once in their lifetime (77.9%) and also in the last month (58.5%). In that study, 32.3% of students reported having had an episode of heavy drinking and 24.3% reported having been drunk, both in the last month.

Alarmingly, more than half of 14 year old students (56.1%) reported having consumed alcohol in the last year and 36% had consumed alcohol in the last month (OEDA, 2019). These reports are consistent with research that indicates that alcohol is the most widely used substance in Spain, including the most prevalent substance used in mid-adolescence. Some studies estimate the general mean age for initiation of alcohol consumption at 16.6 years old (DGPND, 2017). Other studies specify the mean age of first alcohol consumption at 14.0 years old and the mean age of a weekly alcohol use at 15.2 years old (OEDA, 2019). All these large, national studies show a relatively stable tendency in the prevalence of consumption as well as ages for initiating alcohol use. The early onset and widespread prevalence of alcohol use among youth in Spain is a critical situation that research and public policies should address.

Therefore, prevention becomes essential to delay the onset of consumption and avoid habitual alcohol consumption in adolescence. In Spain, from the Government Delegation for the National Plan on Drugs (PND), the National Strategy on Addictions 2017-2024 establishes the guiding principles of evidence, social participation, intersectorality, integrality, equity, and gender (PND, 2017). Above all, the need for evidence-based prevention programs is emphasized. Several programs have been developed at the intervention-prevention level to avoid or reduce substance use or other problematic behaviors among young people, and their effectiveness has been evaluated (e.g., Gómez-Fraguela et al., 2003; Moral et al., 2009; Orte et al., 2013). However, in order to determine the effectiveness of a prevention or intervention program, it is necessary to carry out a systematic and thorough evaluation processes in each specific context. Within this framework, the Mantente REAL program (originally Keepin’ it REAL; Marsiglia & Hecht, 2005) has been adapted to several cultural contexts and has been tested to assess its effectiveness.

Mantente REAL as an Effective Prevention Program

Mantente REAL is a school-based universal prevention program targeting drug use and other problematic behaviors specifically designed to be implemented at the beginning of adolescence and thus prevent early substance use. The program has been developed, implemented, and evaluated in the United States by an interdisciplinary team from the Global Center for Applied Health Research at Arizona State University. In its original version (2005), Keepin’ it REAL proved effective in preventing substance use by middle school students in Arizona (Hecht et al., 2003) and was evaluated as one of the three most cost-effective school-based substance use prevention programs (Miller & Hendrie, 2008). Linguistically and culturally adapted versions of the program have also been implemented and evaluated in several Spanish-speaking countries, such as Guatemala, Mexico, Spain, and Uruguay (Kulis et al., 2019; Marsiglia et al., 2018; Marsiglia et al., 2019).

Mantente REAL is a program with an innovative and efficient design that brings together the fundamental elements of evidence-based prevention programs for adolescents (Tobler et al., 2000), elements that are separated or absent in other implemented programs in Spain. The 12 lessons are implemented during the school year as part of students’ regular academic curriculum, which facilitates the integration of content and learning. Moreover, the program aims at preventing substance use generally and at alcohol, tobacco, and cannabis use specifically, but also includes content related to other pressure situations typical in adolescence; therefore, it indirectly addresses antisocial behavior that could be perpetrated as a result of peer pressure. Furthermore, the program is not simply focused on strengthening socio-emotional skills, but also addresses the decision-making and risk assessment processes as well as specific behavioral strategies for dealing with risky situations and how to find social support if needed; that is, it teaches life skills. In addition, Mantente REAL is a prevention program being implemented in other countries which allows cross-cultural comparisons that would help advance knowledge on substance use prevention.

The program focuses on a generic skill training through which students acquire concrete social pressure resistance skills to not engage in alcohol and other drug use or other antisocial behaviors (Alberts et al., 1991; Gosin et al, 2003; Moon et al., 1999). The main purpose of the program is to teach students how to resist substance use offers and opportunities, such as offers of alcohol, tobacco, and other drugs, using the REAL strategies (refuse, explain, avoid, and leave). Refuse refers to the ability of saying “No” in a clear, direct, and respectful manner, either verbally or non-verbally. Explain is to give one or more reasons to decline the offer. Avoid consists of not going to places, situations, or with people where alcohol or drugs might be offered or available. Leave refers to exiting situations where alcohol or drugs are offered or present. These are strategies used most commonly by youth who successfully resist using substances (Kulis et al., 2011; Marsiglia et al., 2009).

The program combines the acquisition of the REAL strategies with other personal and social skills that play an essential role in the psychological adjustment and psychosocial development of adolescents (e.g., communication, assertiveness, risk assessment, and problem solving skills) (Sancassiani et al., 2015). Learning these skills has a special relevance in adolescence since this constitutes a critical stage in psychosocial development, especially in terms of susceptibility to social influence (Steinberg, 2017). In order to reinforce learning and facilitate the incorporation of these strategies in the behavioral repertoire, the skills training of Mantente REAL are drawn from the adolescents’ context, the skills are based on the youths’ communication and decision-making processes. In this regard, the viewing of videos based on the cultural norms of adolescents, scripted, and acted by students, is part of the curriculum of the program (Holleran et al., 2002). Therefore, Mantente REAL is made by young people for young people.

The process to adapt Mantente REAL to Spain focused on modifying the curriculum manuals for teachers and students and the accompanying videos and to make them more representative and realistic for the context of contemporary Spanish adolescents. This process started with a linguistic adaptation of the Spanish version of Mantente REAL that was created for Mexico (Marsiglia et al., 2019), by updating the language to usage in Spain and incorporating gendered language and typical youth idioms and expressions. All new professionally directed videos were created to introduce the program and represent the four REAL drug resistance strategies. The video scripts were developed and enacted by youth from the target population to represent authentic situations where Spanish adolescents confront behavioral health risks, such as drug offers, in identifiably Spanish locations. Examples of behavioral risk scenarios in the curriculum manuals were also adapted and revised to reflect the Spanish cultural context. All the changes in content were supervised and reviewed by independent experts in the field of prevention. The adaptation process maintained fidelity to the contents of the original program by retaining the core elements of the program, such as the mastery of the four drug resistance strategies and training in risk assessment and communication competence.

Figure 1

CONSORT Diagram for Spain Mantente REAL Efficacy Trial

Research thus far has shown the program to be effective for the prevention of substance use in different contexts and countries. Several studies have demonstrated the significant effects of Mantente REAL in reducing adolescent substance use (alcohol, tobacco, and cannabis), preserving or increasing negative attitudes towards substance use, strengthening norms against substance use, and increasing the use of effective drug resistance strategies (Kulis et al., 2005; Kulis et al., 2019; Marsiglia et al., 2015). As regards gender differences in the effectiveness of Mantente REAL, previous research conducted in Mexico indicated that although gender differences in adolescent alcohol use have narrowed to near parity the effects of the program on alcohol frequency are moderated by gender, with females showing stronger desired effects than shown by males (Marsiglia et al., 2015). Despite the evidence in other Spanish-speaking contexts, no research has previously tested the effectiveness of the Spanish version of Mantente REAL adapted to Spain. Previous evidence suggests that the degree to which the curriculum is culturally adapted can affect the program’s effectiveness (Griner & Smith, 2006; Kulis et al., 2005). Therefore, the effectiveness of the culturally adapted version of Mantente REAL to Spain should be tested.

The Current Study

As one more phase of the described process, the current study aims to evaluate at the quantitative level the effectiveness of the culturally adapted version of Mantente REAL with a sample of Spanish adolescents. Although data about tobacco and marijuana use were also collected, this study specifically tests effects on alcohol outcomes. Since it is the most socially accepted and the most prevalently used drug in Spain, this study aims specifically to assess several types of alcohol use, considering different measures that range in severity. Three main hypotheses are proposed. First, that Mantente REAL would be effective in reducing the rates of (a) frequency of alcohol use, (b) amount of alcohol used, (c) heavy drinking episodes, and (d) intoxication episodes. Second, that Mantente REAL would be more effective with students at greater risk, as proposed by the risk argument (MacKinnon et al., 1989), that is, adolescents experiencing higher levels of alcohol use would show stronger desired effects of the Mantente REAL program than those less involved with alcohol use. Third, that Mantente REAL would be equally effective in females and males. Given the pervasive presence of alcohol in the Spanish context and the high rates of consumption among Spanish adolescents, the implementation of a validated program such as Mantente REAL represents important progress in the field of school based prevention. Thus, the findings of this study would contribute to advancing the availability of culturally specific and evidence based alcohol prevention programs in Spain.

Method

Participants

The sample was composed of 755 Spanish adolescents aged 11 to 15 (M = 12.24, SD = 0.56), 47.1% females, residing within two different regions: Santiago de Compostela (Galicia, NW Spain; n = 347) and Seville (Andalucia, S Spain; n = 408); 40.1% of adolescents were born in Galicia, 50.3% in Andalucia, 3.4% in another part of Spain, and 6.1% in another country. Almost a half of students lived with 3 other persons at home (45.5%). Students mostly reported their parents have a level of education equal to or higher than high school (61.2%) and always have enough money at home for food, transportation, utilities, school fees, and clothes (60.9%). Table 1 shows sociodemographic characteristics of the sample by intervention condition. Adolescents were enrolled in the 1^st grade of compulsory secondary education in the school year 2018/2019 in 12 state secondary schools, 6 in the region of Santiago and 6 in the city of Seville. Schools were selected by convenience sampling, but randomly assigned to the experimental or control conditions. Figure 1 displays the study’s CONSORT diagram, showing the school selection process and the flow of participating students (enrollment of subjects, assignment to intervention conditions, and retention of subjects during the study).

Instruments

Recent alcohol use. Four different measures of alcohol use in the past 30 days were evaluated (Graham et al., 1984; Hansen & Graham, 1991). Adolescents reported their frequency of alcohol use (1 item, “In the last 30 days, how many times have you had an alcoholic drink?”), the amount of alcohol used (1 item, “In the last 30 days, how many alcoholic drinks have you had?”), as well as the frequency of heavy drinking episodes (1 item, “In the last 30 days, how many times did you drink five or more alcoholic drinks in a row (– on the same occasion?”) and intoxication episodes (1 item, “In the last 30 days, how many times have you got drunk?”). Responses were scored from 0 to 6 for the frequency measures (none, only once, 2-3 times, 4-9 times, 10-19 times, 20-39 times, 40 times or more) as well as for the amount measure (none, part of one, entire, 2-3, 4-9, 10-19, 20 or more). Questions were phrased identically at the pre-test and the post-test.

Intervention condition. A dummy variable was created to contrast the students who received the Mantente REAL curriculum (1) with the students in the control group who did not receive the Mantente REAL curriculum (0).

Control variables. Intervention effects were estimated while controlling for three variables assessed in the pre-test: gender, intervention site, school belonging, and parenting practices. These variables were included to control for differences between samples from the two regions in the students’ reports on their school and family environments, and the potential influence of these psychosocial factors on student’s responsivity to the program. Gender was assessed by the dummy variable female (0) versus male (1). A dummy variable was created to control for site, contrasting Santiago de Compostela (1) versus Seville (0). Students’ sense of school belonging was assessed by the question “Do you feel part of this school?” answered in a 4-point scale (not at all, a little, some, very much). A principal component factor score was created to control for parenting practices combining a scale of positive relationship with parents (6 items; e.g., “You think your parents care about what you feel”; α = .86; Upchurch et al., 1999), a scale of parent-child conflict (8 items; e.g., “You think you and they get along badly”; α = .87; Robin & Foster, 1989), and a scale of parental knowledge of the child’s whereabouts and activities (3 items; e.g., “They know what you do after school”; α = .81; Kerr & Stattin, 2000), all of which students answered in a 4-point scale (never, few times, sometimes, many times).

Risk moderation interactions. To test whether the Mantente REAL program would be more effective in students already involved with alcohol use (students who showed higher rates in the pre-test), four interaction variables were created by multiplying the dummy variable for the intervention condition by the mean-centered value of each dependent variable at the pre-test (i.e., frequency, amount, heavy drinking, intoxication).

Gender moderation interaction. To test whether Mantente REAL would be equally effective in females and males, an interaction variable was created by multiplying the dummy variable for the intervention condition by the dummy variable of gender.

Procedure

Compliance with ethical standards was taken into account throughout the research and the project was approved by the Bioethics Committee of the University of Santiago de Compostela and the University Pablo de Olavide. The study was first presented to the heads of 46 secondary schools (see CONSORT diagram, Figure 1), who were provided with the necessary information about the study, in the cities of Seville and Santiago de Compostela. Given that Santiago de Compostela is a city with a smaller population than Seville, schools in the outskirts of Santiago de Compostela were also considered to take part in the study. Since the objective was to carry out a pilot study, a sample close to 800 participants was targeted; that is, considering the mean number of students enrolled in Spanish schools, a total of 12 schools were the sample target. Therefore, the recruitment ended once 12 secondary schools agreed to participate in the study, six in each site.

To guarantee the validity of the study, selected schools were randomly assigned to control and experimental groups. Thus, six schools (three in each site) were assigned to the experimental group (i.e., with Mantente REAL implemented in the 1^st grade) and the other six (three in each site) were assigned to the control group (i.e., the program is not implemented). In the schools assigned to the experimental group the program was implemented by the regular tutors (homeroom teachers) in their classrooms during school hours. The program adapted to Spain has a total of 12 lessons that were to be implemented weekly. Program implementation was structured and guided by a teacher’s manual, which detailed all the activities and procedures to follow, and a student’s workbook, which contained the activities to be completed by students. Before starting the curriculum’s lessons, all 1^st grade tutors in the intervention schools received practical training over a two day long session to learn how to deliver the Mantente REAL curriculum with fidelity. In the schools assigned to the control group no other prevention programs were being implemented during that school year, only the regular activities included in the students’ curriculum.

In order to evaluate the effectiveness of the program, information regarding variables of interest was collected in all 12 schools, before (pretest) and after (posttest) the implementation of the program in the experimental group. Posttest surveys were administered approximately four months after the pretest. In addition, two sources of qualitative data were collected: fidelity observations (3 observation sessions) during the program implementation and focus groups with teachers and students after the implementation to assess challenges in delivering the program and the degree of satisfaction with it. Thus, the evaluation process of the program followed the recommendations of some authors (Fernández et al., 2002), who proposed to use a quantitative and qualitative methodology. For quantitative data collection, qualified members of the research teams visited the centers, explained the objectives of the research, and provided proper instructions to the adolescents who answered the self-reported questionnaires. Parental consent was requested and, subsequently, adolescent assent was obtained before questionnaires were administered. Adolescent participation was voluntary, and anonymity and confidentiality of information were completely guaranteed. A confidential unique code created by students themselves was used to match pre-tests and post-tests. This method resulted in a low rate of sample attrition (see CONSORT diagram, Figure 1): 89.9% of the pre-tests had a matched post-test.

Data Analysis

IBM SPSS Statistics 24 and MPLUS 7 were used to conduct the statistical analyses. Baseline (pre-test) equivalence on sociodemographic characteristics was tested across intervention conditions. We assessed equivalence with chi-square tests, t-tests, and standardized effect size measures (Cramer’s V or Cohen’s d), using the value .10 as the threshold of imbalance between groups (Austin, 2009). If variables showed significant standardized differences across conditions (> .10), they were included as control variables in further analyses in order to control for their potential influence on the program’s effectiveness. Second, frequencies, skewness, means, and standard deviations were analyzed and changes in the outcome variables from pre-test to post-test were examined with pairwise t-tests, within and between intervention conditions, to show the direction and statistical significance of the changes. The intervention effect size was estimated using Cohen’s d, dividing the difference-in-differences between experimental and control groups by the pooled standard deviation of the raw scores within groups. Next, three different linear models were analyzed in MPLUS for each outcome: 1) a general model to assess the intervention effects using the dummy variable contrast of intervention condition; 2) the general model including risk moderation interactions for each outcome to test whether the intervention was more effective in the students who were already more or less involved with alcohol use in the pre-test; and 3) the general model including the gender moderation interaction to test whether the intervention was equally effective in female and male students. All tests adjusted statistically for the school-level clustering of data (Mplus “Cluster” command) and type = complex was specified to take non-independence of observations into account. All models controlled for the outcome as measured at the pre-test, gender, intervention site, parental education level, school belonging, and parenting practices. In addition, all models employed full-information maximum likelihood (FIML) estimation to account for attrition to the post-test (10.1%) and item missing data, as well as the robust maximum likelihood (MLR) estimator to adjust for non-normal and zero inflated distributions in alcohol use.

Results

Table 1 displays the preliminary analysis regarding the baseline equivalence in sociodemographic characteristics of participants across intervention conditions. Effect sizes showed that parental education level (Cramer’s V) and students’ sense of school belonging (Cohen’s d) presented imbalance between groups (> .10). Therefore, both were included as control variables in further analyses. No other significant differences in the distribution of gender and birthplace or in the means of age, socio-economic status and household size were found across experimental and control groups. None of them were included as a control variable. Additionally, parenting practices significantly differed across intervention sites, with Sevillian adolescents showing lower means in positive relationship with parents, t(731) = -2.01, p = .045, and parental monitoring, t(718) = -5.17, p < .001, and higher means in parent-child conflict, t(728) = 3.99, p < .001, compared to Santiago de Compostela adolescents. Therefore, intervention site, as well as parenting practices were included as control variables. Regarding baseline equivalence across groups on the four outcome variables (tests not presented in tables), only heavy drinking exceeded the threshold (Cohen’s d = 0.14), with a higher mean in the intervention group than in the control group.

Table 1

Sociodemographic Characteristics of Participants, by Intervention Condition

Note. Cramer’s V is the effect size of chi square differences (categorical variables) and Cohen’s d is the effect size of t-test differences (continuous variables).

Table 2

Sociodemographic Characteristics of Participants, by Intervention Condition

Note. All models used the MLR estimator to address the non-normality of dependent variables and the FIML estimator to account for missing data.

¹Pretest to posttest change in Mantente REAL minus pretest to posttest change in control group

^†p < .10, *p < .05, **p < .01, ***p < .001.

Means, standard deviations, and mean changes in outcome variables from pre-test to post-test are shown in Table 2, separately by intervention condition. The very low means on the outcome variables at pretest (close to zero) reflect the relatively low prevalence of use in the last 30 days and distributions highly skewed toward non-use. At pretest only 11.6% of the students reported any occasion of consuming an alcoholic drink, 10.3% reported consuming some amount of alcohol, 2.7% reported drinking five or more alcoholic drinks on the same occasion, and 2.7% had got drunk in the last month (prevalence rates not presented in tables). These outcomes were positively skewed: alcohol frequency = 2.82, alcohol amount = 3.13, heavy drinking = 5.87, intoxication episodes = 5.47 (skewness statistics not presented in tables). The MLR estimator was used in all models to adjust for this non-normal distribution. In Table 2, the results within conditions indicated that students in the MR group showed significant increases in alcohol frequency and amount, while students in the CG exhibited significant increases in frequency and amount of alcohol use, as well as frequency of heavy drinking and intoxication episodes. The comparison of intervention conditions indicated that students in the MR group showed relatively smaller increases in alcohol use on all four measures, compared to those in the CG, even having shown higher levels of heavy drinking at pre-test. These differences across conditions are in a desirable direction, and statistically significant for heavy drinking (p < .05) and intoxication episodes (p < .01) and marginally significant (p < .10) for alcohol frequency and amount.

Table 2 also displays intervention effects sizes and relative rates of reduction. Cohen’s d effect sizes show small intervention effects on the alcohol frequency, alcohol amount, and heavy drinking outcomes (d = 0.20, small, d = 0.50, medium, d = 0.80, large; Cohen, 1988). In terms of correlation and explained variance, a small effect size means that sample allocation to intervention conditions accounts for up to 1% of the variance of each outcome (Cohen, 1988). Although all of the effect sizes are statistically small, it is clinically relevant to note that the intervention effect on the frequency of intoxication episodes is higher than it is for other outcomes. In this regard, relative rates of reduction indicated that students in the MR group show around 55% less of an increase in alcohol frequency, 48% less in amount used, 80% less in heavy drinking, and 87% less in intoxication episodes than the students in the CG.

Table 3

Unstandardized Results of Baseline-Adjusted General Linear Models Controlling for Gender, Intervention Site, Parental Education, School Belonging, and Parenting Practices

Note. All models used the MLR estimator to address the non-normality of dependent variables and the FIML estimator to account for missing data.

¹In model 2, outcomes at pretest were mean-centered.

^†p < .10, *p < .05, **p < .01, ***p < .001.

Table 3 presents the results from baseline-adjusted general linear models (i.e., controlling for the outcome as measured at the pre-test) assessing the intervention effects on the outcome variables. Three different models were tested. A first model was conducted using the dummy variable contrast of intervention condition to test differences in outcomes between the MR group and the CG (see Table 3, Model 1). The results indicated that the covariates significantly predicted alcohol outcomes, except gender and parental education level. Frequency of the outcome at pre-test positively predicted alcohol outcomes, except heavy drinking; intervention site negatively predicted all outcomes (i.e., students from Santiago de Compostela showed lower rates of outcomes); school belonging positively predicted alcohol outcomes, except heavy drinking; and positive parenting practices inversely predicted all the alcohol outcomes. The results also showed that the MR group shows more desirable changes in alcohol frequency and intoxication episodes. Students in the MR reported a significantly lower frequency of alcohol use (α = -.04, p = .025) and intoxication episodes (α = -.11, p < .001) than the students in the CG.

A second model was conducted to additionally test whether the Mantente REAL program is more effective with students at higher risk for alcohol use (see Table 3, Model 2). In other words, whether adolescents who reported higher levels of substance use in the pre-test show stronger desired effects of the Mantente REAL program than those using less. The results for covariates and intervention effects did not change with respect to the first model. The results indicated that the risk interaction is only significant for alcohol amount. After the intervention, students who reported a higher amount of alcohol use at pre-test reduced the amount of consumption significantly more than those who reported a lower amount of alcohol use at pre-test, as shown in Figure 2. That is, among the students assigned to the MR group those using a higher amount of alcohol before the intervention (see black line with circles in Figure 2) reduced the amount significantly more than those students with a lower amount of alcohol before the intervention (see black line with crosses in Figure 2). The standardized coefficient of the interaction was marginally significant (α = -.21, p = .056).

Figure 2

Estimated Alcohol Amount at Posttest by the Interaction of Pretest and Intervention Condition.

Lastly, a third model was conducted to test whether the Mantente REAL program is equally effective with female and male students. The results for covariates and intervention effects did not change with respect to the first model. The results indicated that the gender interaction was not significant for any alcohol outcome (results not presented in tables).

Discussion

The current study aimed to evaluate the effectiveness, at the quantitative level, of the Mantente REAL program with a version of the curriculum adapted to the Spanish context and tested with adolescents from northern and southern Spain. The main objective was to test whether Mantente REAL was effective in reducing the rates of frequency of alcohol use, amount of alcohol used, heavy drinking episodes, and intoxication episodes. The study also aimed to evaluate whether Mantente REAL was more effective with students at greater risk of alcohol use and whether it is equally effective in females and males.

The results of this study showed very low prevalence rates of alcohol use at pre-test and students in both the intervention and control groups increased significantly the frequency and amount of alcohol from the pretest to the posttest evaluation. However, students who participated in the Mantente REAL prevention program reported lower increases than the control group in their alcohol intake frequency, heavy drinking, and intoxication episodes. Thus, even when use of alcohol increases with time, participating in the Mantente REAL program prevents or reduces students’ alcohol consumption compared with their peers in the control group. Moreover, implementation of Mantente REAL had significant effects on the frequency of alcohol use and intoxication episodes. The effects of the intervention were stronger on intoxication episodes, as indicated Cohen’s d. The results of this study showed that the desired effect of Mantente REAL on reducing the amount of alcohol use was greater in those adolescents who were already involved in higher amounts of alcohol use before the intervention. This result partially supports the risk moderation argument, according to which adolescents at higher risk of using substances would benefit more from preventive interventions (MacKinnon et al., 1989). Lastly, the results indicated that Mantente REAL is equally effective in female and male students and that gender does not moderate the effects of the program on alcohol outcomes. Taken together, the findings of the current study indicate the effectiveness of Mantente REAL in slowing the increase in alcohol use (frequency) at the beginning of adolescence and preventing the development of problematic types of alcohol use (intoxication episodes), both in females and males.

The effects of Mantente REAL, despite presenting low effect sizes, are of great relevance if their potential social impact is considered. The effect size is a robust estimator to analyze intervention effects at an individual level, but it is not the best indicator to analyze the social impact of universal interventions (Greenberg & Abenavoli, 2017). A recent review of meta-analyses concludes that Cohen’s d effect size is not appropriate for interpreting the magnitude of effects of universal prevention programs aimed at school-age youth (Tanner-Smith et al., 2018). Universal prevention programs are aimed at populations that may not have any symptoms or high levels of risk. Therefore, although the program is very effective in reducing the likelihood of the problem, in many cases changes after intervention will not reach a significant level. Moreover, very large sample sizes are needed to have a research design with adequate statistical power in the evaluation of universal prevention programs (Cuijpers, 2003). All these issues lead to the difficulty of observing significant results and obtaining high effect sizes (Greenberg & Riggs, 2015).

According with Tanner-Smith et al. (2018), effects sizes of universal programs with youth are especially low when they are specifically aimed at preventing substance use (P25 = 0.05, median = 0.07, P75 = 0.11) or alcohol use (P25 = 0.05, median = 0.10, P75 = 0.15). Considering these distributions of effects, Mantente REAL shows relatively high effects sizes on alcohol outcomes, above the values associated with the 75th percentile for programs with these characteristics. Another recent meta-analysis of the effectiveness of school substance abuse prevention programs in Spain (Espada et al., 2015) showed that preventive program effectiveness is usually low or moderate and that the global average effect size of drug prevention programs is 0.16 (SE = 0.03). Therefore, Mantente REAL shows similar and even higher effects than the average of the prevention programs in our context. Another alternative to assess the potential social impact of preventive interventions, such as Mantente REAL, may be to consider the percentage of reduction in consumption (Greenberg & Abenavoli, 2017). This study shows that while the increase from pretest to posttest in the frequency of alcohol use in the control group was 0.18, in the treatment group it was only 0.08, a 55% reduction in the increase in the frequency of use. The percentages of reduction in alcohol amount were 48% and for heavy drinking and intoxication episodes the reductions exceeded 80%.

These results are especially relevant in the Spanish context, since previous research in Spain has shown that the average age of onset of substance use in youth is between 14 and 16 years old (Colell et al., 2013). Specifically, more than half of students over 14 years old are involved in recent – within the past 30 days – alcohol use (DGPND, 2017; INE, 2017; OEDA, 2019). Thus, considering that alcohol is the most prevalently used substance among youth in Spain, the implementation of a validated program such as Mantente REAL represents important progress in the field of school based prevention. In addition, according to the indications of various authors (e.g., Gázquez et al., 2009; Gottfredson & Wilson, 2003), the best preventive approach is one that is carried out before the phenomenon of substance use appears. Compared to data at the national level that reported prevalences from 24.3% to 58.5% of alcohol use, heavy drinking, or intoxication episodes in adolescents aged 14-18 (OEDA, 2019), the current results indicate very low rates of alcohol use (from 2.7% to 11.6%) among the considerably younger students enrolled in the 1^st grade of compulsory secondary education, who were, on average, 12-13 years old. These rates were especially low for more abusive types of alcohol use (heavy drinking and intoxication episodes). These findings suggest that the beginning of adolescence is an optimal time to implement this type of program. Early adolescence also constitutes an ideal period for teaching social pressure resistance strategies because the peer group becomes the social group of reference and socialization in this stage (Collins & Steinberg, 2008). Considering the influence that research has consistently found regarding the effects of peers on adolescent behavior, providing adolescents with strategies to cope with peer pressure would prevent not only substance use, but other types of antisocial behavior (Steinberg, 2017).

The results of the current study are consistent with others found in other Spanish-speaking contexts. The desired effects of Mantente REAL on preventing alcohol use were also obtained in Mexico (Marsiglia et al., 2015), Uruguay (Marsiglia et al., 2018), and Mexican American youth, especially when the program was adapted to Latino populations (Kulis et al., 2005). This result suggests that the degree to which the curriculum is culturally adapted can affect the program’s effectiveness, as previous meta-analytic research has indicated (Griner & Smith, 2006). Therefore, the systematic and thorough adaptation and evaluation of Mantente REAL for Spain have contributed to increase its effectiveness in this context. Furthermore, a meta-analysis of the effectiveness of school substance abuse prevention programs in Spain (Espada et al., 2015) concluded that effectiveness increases especially when a combination of oral, written, and audiovisual support materials were used. In this regard, Mantente REAL is a manualized teacher and students program with video support materials for illustrating each strategy, and the sessions were a mixture of writing and oral activities. Previous research has pointed to the beneficial effects of using peer-created videos to communicate with youth in substance use prevention (Holleran et al., 2002; Reeves et al., 2008; Warren et al., 2006). The collaborative and participative perspective of Mantente REAL offers adolescents opportunities for introspection, expression, discussion, and questioning of risks experienced in their daily lives, identifying effective ways to respond to them through the REAL strategies.

In conclusion, the results of this study show not only the effectiveness but the need to implement the Mantente REAL program in Spain in the 1^st grade of mandatory secondary education (12-13 years old). Mantente REAL is an effective program for slowing the developmental increase of alcohol use during adolescence and of preventing involvement in unhealthy behaviors related to alcohol use by Spanish adolescents. These findings support a potential new and efficacious alcohol prevention intervention for adolescents and a promising prevention success for Spanish youth. Alcohol consumption in Spain is incorporated into the culture, is tolerated from an early age, and is normalized especially in local festivals and other mainly outdoors social events. Adolescents who participated in the study had probably experienced that alcohol use is common in local festivities, family celebrations, and peer-group gatherings, and this consumption is associated culturally with joy and having fun. Taking the cultural context into account, the goals of prevention programs with Spanish adolescents cannot focus solely on the elimination or reduction of alcohol use, but should aim to raise awareness of the damage and risks of consumption, as proved youth with assertive life skills and resistance strategies to avoid alcohol use. In this regard, Mantente REAL is an example of a prevention intervention that can be implemented to support the positive psychosocial development of Spanish adolescents.

Limitations and Future Directions

Despite positive results from the implementation of Mantente REAL in Spain, this study is not exempt from important limitations. First of all, the current study included only students in the 1^st grade of mandatory secondary education, which means that the overall prevalence and frequency of substance use tend to be very low. Furthermore, some of the students had not yet experienced the actual situations of social pressure for using alcohol and other drugs. Therefore, future studies should consider the inclusion of different age-range groups in order to delve into potential effects depending on the age and its influence on substance use. This could also be addressed with more longitudinal data through follow-up surveys of the students. In this regard, the current findings should be considered with caution due to the short interval to the immediate post-test. Future follow-up surveys should be conducted to analyze long-term effects of the Mantente REAL program and how they are maintained over time. Secondly, although Mantente REAL is intended to promote the REAL strategies in early adolescence, this study did not include information about intra-group and inter-group differences regarding those strategies. Therefore, this issue should be addressed by future studies in order to assess the potential effect of the program on the psychosocial skills to resist social pressure and their possible mediating role in the intervention’s effects on alcohol use. Further research on potential mediation effects can include the role of mastery of drug resistance strategies, and preservation of anti-drug attitudes. Thirdly, it should be noted that the implementation of Mantente REAL in Seville and Santiago was conducted using the same educational materials and procedures, which had been previously adapted only to the Sevillian context. Northern and Southern Spain present several cultural and social differences which may also influence the effectiveness results of Mantente REAL. Future research projects should consider the adaptation of the program to the Galician context, in which Santiago de Compostela is located, and verify the utility and validity of the adapted version. In addition, following the recommendations for an integral program evaluation, the qualitative data collected, such as fidelity observations and focus groups, should be analyzed to test potential moderation effects on effectiveness and improve future adaptations of the program. Finally, the results are drawn from a convenience sample, composed of schools whose head administrators were most interested in having their students participate in the study. Thus the sample may not be typical or fully represent the range of public secondary school students in the two regions. Despite these limitations, Mantente REAL produced promising results in preventing and reducing alcohol use among early adolescents in two main regions of Spain. These findings not only contribute to the advancement of evidence based prevention interventions in Spain but they also contribute to the overall advancement of prevention science.