The Incredible Years Parenting and Child Treatment Programs: A Randomized Controlled Trial in a Child Welfare Setting in Spain

Introduction

Although the prevalence of child maltreatment is still unknown, there is a broad consensus that it is a widespread phenomenon all over the world (Gilbert et al., 2009). In Spain, official records from Child Protection Services in 2019 showed that, excluding unaccompanied foreign minors, 39,000 children and adolescents (454 per 100,000) had been removed, were at risk of being removed from their homes, or were under Child Protection Services (CPS) investigation due to severe child maltreatment (Fiscalía General de Estado, 2020; Observatorio de la Infancia, 2019). Unfortunately, reliable national data of less severe cases of child maltreatment or children at risk are not available. Prevalence rates from official Spanish records are far from matching real data, as youth victimization studies with national community samples (Pereda et al., 2014) and international population-based surveys suggest. Studies carried out in high-income countries with self-report and parent measures have found overall prevalence rates of 3-17% and 8-31% for sexual abuse among boys and girls respectively, 3.7-29.7% for physical abuse, 4-36.3% for psychological abuse and neglect, and 1.4-16.3% for physical neglect (Barth et al., 2013; Gilbert et al., 2009; Pereda et al., 2009; Stoltenborgh et al., 2015). Rates of maltreatment can be more than ten times the rates of substantiated cases (e.g., Fergusson et al., 2000; Finkelhor, 2008; MacMillan et al., 2003), so, many countries, including Spain, have been involved in legislative changes and global and national efforts to end it (International Society for the Prevention of Child Abuse and Neglect, 2018, 2021).

Child maltreatment substantially contributes to child mortality and is associated with adverse outcomes across the life span. Although these outcomes are not inevitable, maltreatment in childhood is a risk factor for long-lasting negative effects on physical health (e.g., reduced immune system efficiency, abnormalities in the functioning of the endocrine system, chronic pain, obesity), brain structure and functioning, mental health (e.g., behavior problems, depression, suicide attempts, alcohol and other drug misuse), psychosocial adjustment (e.g., difficulties in making and maintaining relationships, maladjustment in school and work, poor impulse control), sexual behavior (teenage pregnancy, unhealthy sexual practices), and criminal behavior (Carr et al., 2020; Child Welfare Information Gateway, 2019; Gilbert et al., 2009; Institute of Medicine and National Research Council, 2014; Lippard & Nemeroff, 2020; National Scientific Council on the Developing Child, 2012, 2020; Teicher & Samson, 2016). Also, experiencing multiple forms of maltreatment is common and has been associated with more severe outcomes (Carr et al., 2020; Hughes et al., 2017; Institute of Medicine and National Research Council, 2014; Lippard & Nemeroff, 2020).

Given the high prevalence and serious consequences of child maltreatment, effective primary and secondary prevention as well as therapeutic programs from early childhood are required (Gilbert et al., 2009). As the etiology of child maltreatment is complex and multidimensional – including a wide range of individual, family and social factors associated with perpetrators, children, and the context where it occurs – and that maltreatment effects are also diverse, a range of services and interventions should be available. But the selection of services and interventions to provide for each child, parent, and family is not easy. When multiple specific problem areas are identified, it is crucial to adequately sequence them, as well as to maximize effectiveness by making use of the smallest number and lowest intensity of services needed to accomplish the intended goals and to produce the largest effects in the shortest timeframe (Barth, 2009; Berliner et al., 2015).

Parenting practices are a central focus of many preventive and rehabilitative programs in the child maltreatment field. Two main reasons explain their relevance. First, although difficulties experienced by families vary, dysfunctional or poor parenting practices by commission or omission (e.g., ineffective, unprotective, or violent) have been identified as a critical risk factor and typically affect many at-risk and maltreating families (Berliner et al., 2015; Temcheff et al., 2018). Second, some studies have found that improvements in parenting practices are associated with positive effects on other family problems or risk factors as parental psychological distress, parental attitudes towards harsh parenting practices, relationships between parents, or child emotional and behavioral problems (Berliner et al., 2015; Chen & Chan, 2016; Pinquart & Teubert, 2010).

Most of the evidence-based parent training programs started out as treatment or preventive strategies focused on child behavior problems (e.g., Incredible Years, Parent-Child Interaction Therapy, Parent Management Training-The Oregon Model, Triple P; see https://www.cebc4cw.org/). These programs have shown efficacy at different ages, countries, and cultures in reducing child behavior problems, producing changes in children’s cognitive and behavioral outcomes, and improving parenting (Furlong et al., 2012; Gardner et al., 2019; Knerr et al., 2013; Mejia et al., 2012; Piquero et al., 2016). According to the reviews of Altafim & Linhares (2016), Branco et al. (2021), Barth & Liggett-Creel (2014), and Temcheff et al. (2018), (a) the main purpose of parent training programs is to improve the relationship and communication patterns between parents and children through the improvement of child-rearing and parenting practices (reinforcement, discipline), the stimulation of a positive and responsive parent-child interaction, the improvement of parental emotional regulation and communication skills, and the promotion of positive and nonviolent techniques to manage child behavior; (b) they are skill focused; (c) delivery techniques usually include modelling, role-playing, video-feedback, and assignment of between-session practice exercises (homework); (d) they often rely on weekly individual or group-based parent training sessions; (e) most of them are delivered at a clinic or service center (e.g., early childhood centers, schools, community, or primary health-care centers), although some programs offer a combination of sessions inside and outside the home; and (f) while some programs involve only the parents and others include joint parent-child interventions, all of them require skill practice opportunities between parents and children.

Many parent training programs have been applied and adapted for at-risk and maltreating parents. Several meta-analyses, from predominantly high income countries, have shown their potential for reducing corporal punishment, unintentional injuries, and child maltreatment, and for preventing the occurrence and recurrence of child maltreatment excluding sexual abuse (Chen & Chan, 2016; Coore-Desai et al., 2017; Euser et al., 2015; Gubbels et al., 2019; Menting et al., 2013; van der Put et al., 2018). Some of the parent training programs with more empirical evidence of effectiveness for the indicated prevention and treatment of child maltreatment are Incredible Years, Parent-Child Interaction Therapy (PCIT), and Triple P (Level4). They have all been rated as empirically well-supported by the California Evidence Based Clearinghouse (https://www.cebc4cw.org/) and the Blueprints for Healthy Youth Development (https://www.blueprintsprograms.org/). These programs share a cognitive-behavioral and theoretical social learning orientation, and, as evidence-based programs, are manualized, provide training to the practitioners who deliver them, use strong ongoing supervision or coaching models, and include procedures and tools to assess and monitor implementation fidelity.

The present study focuses on the Basic Parenting and the Small Group Dinosaur curricula of Incredible Years, a widely researched well-established program designed in the early eighties by C. Webster-Stratton with the goals of promoting young children’s emotional and social competence, preventing, reducing, and treating aggression and emotional problems, and reducing the chance of developing later delinquent behaviors (Webster-Stratton, 2011). The IY program consists of a set of three comprehensive interlocking, multifaceted, structured, and developmentally group-based curricula for parents, teachers, and children that can be used independently or in combination. The curricula focus on the same key outcomes but act through different channels and with different developmental foci. The Parenting program span the age range of 0-12 years, while the child and teacher programs span the age range of 3-8 and 1-8 years, respectively. A minimum number of sessions is required, but clinicians are encouraged to expand on the number of sessions according to group needs. Incredible Years emphasizes sensitivity and adaptation to parents’ and children’s individual needs and goals and to the specific context of the program’s application (for a detailed description of IY’s rationale, theoretical bases, goals, components, and materials, see www.incredibleyears.com; Webster-Stratton, 2011, 2021). The effectiveness of the Incredible Years Program has been evaluated in multiple randomized controlled trials, most of them focused on the Basic Parenting program and particularly the preschool curricula. Although there is promising evidence regarding the benefits of the children and teachers’ curricula, they have been underresearched in comparison to the parenting program. Also, more studies are needed with regard to the efficacy of various combinations of programs (Pidano & Allen, 2015).

The Basic Parenting program has demonstrated extensive evidence of efficacy according to parents, teachers, and observers (Gardner & Leijten, 2017; Kaminski & Claussen, 2017; Leijten et al., 2020; Menting et al., 2013), with some studies suggesting larger effect sizes for treatment vs. preventive and for indicated vs. selective samples (Pidano & Allen, 2015; Scott et al., 2014). The program has shown success with culturally diverse groups in USA, including Hispanic/Latino, Asian American, African American, and migrant families from different countries, and has also been evaluated by independent researchers in many other countries including the United Kingdom, Ireland, Canada, Norway, the Netherlands, Russia, and Portugal (Gardner et al., 2010; Hutchings et al., 2008; Larsson et al., 2009; Pidano & Allen, 2015; Posthumus et al., 2012; Webster-Stratton et al., 2012). Several follow-up studies conducted 1, 3, 8, and 12 years after the end of the intervention have shown the maintenance of its effects (Posthumus et al., 2012; Scott et al., 2014; Webster-Stratton et al., 2011).

Incredible Years has demonstrated positive outcomes with vulnerable families. In Europe, for example, a recently published meta-analysis of 13 selective and indicated prevention and treatment trials done in England, Wales, Netherlands, Ireland, Norway, Portugal, and Sweden has found that the Basic Parenting program has been effective in reducing child behavior problems in ethnic minority and socially disadvantaged families (poverty, lone parenthood, teenage parenthood, household joblessness, or low education), with no significant moderation effects by any social disadvantage indicator or by ethnicity (Gardner et al., 2019). The meta-analysis included baseline and post-intervention parents’ reports of child behavior problems of 1,696 children aged 2-10 years old, measured through the Eyberg Child Behavior Inventory Intensity scale (ECBI-I; Eyberg & Pincus, 1999).

The IY Basic Parenting program has also demonstrated positive outcomes with maltreating parents. Hughes and Gottlieb (2004) examined program effects on observational measures of parenting skills and child autonomy in a sample of maltreating mothers and their 3-8 years old children from eastern Canada. Twenty-six mothers were randomly assigned to an 8-week version of the IY program, or to a waitlist control group. The IY Parenting program was provided as an additional service, as almost 70% of the mothers in both groups were engaged in other mental health-related services. Low attrition (7%) and high attendance (92%) rates were found for the IY intervention. Compared to the control group, IY mothers experimented a significant improvement in observed involvement toward their children – that is, parenting behaviors that praised, nurtured, and showed appreciation – and a marginally significant improvement in observed autonomy-support – that is, parenting behaviors that enhanced the child’s sense of value and personal control. No differences were found in parenting behaviors that enhanced a child’s mastery by setting limits and boundaries, or in observational measures of child autonomy. In another randomized control trial carried out by Hurlburt et al. (2013) in seven Head Start centers in Seattle (USA), 361 mothers who received an 8-week version of the IY Basic Parenting program were compared to 156 mothers assigned to a control group. In addition to finding that IY participants improved more than those in the control group in observational measures of positive parenting practices, nurturing/supportive parenting, and discipline competence, and that their children improved more on observed child behavior, it was found that IY intervention benefits were similar for mothers with and without a reported history of child maltreatment. Although both studies (Hughes & Gottlieb, 2004; Hurlburt et al., 2013) implemented and evaluated abbreviated versions of the IY Parenting program, their findings led them to recommend more intensive and prolonged interventions for parents in contact with child welfare. Such a more prolonged intervention was implemented and evaluated by Letarte et al. (2010) and Karjalainen et al. (2019). Letarte et al. provided a 16-week version of the IY Basic Parenting program to 35 families with children aged 5-10 years old monitored in a child protection service in Montreal (Canada). Families were assigned to the intervention group who received IY plus regular services, or to a waitlist control group who received regular services during the study period. Self-reported measures of parenting practices, parents’ self-efficacy, and parents’ perception of child behavior problems were compared at baseline and post-intervention. Results showed that parents who participated in IY reported less harsh discipline, more praise and incentives, more appropriate and positive verbal discipline, better monitoring strategies, and perceived fewer and less frequent disruptive behaviors in their children than parents in the control group. No effects of IY participation were found on parents’ reports of expectations toward their children and self-efficacy. In another study carried out in Finland, Karjalainen et al. (2019) implemented and evaluated a 19-20-week version of the IY Basic Parenting program supported by four additional structured home visits. A sample of 122 parents with 3-7 year-old children with behavioral problems referred to child protection services or receiving other parenting support from social services at the time of the study were randomized into intervention and control groups after baseline assessment. Parental self-reported measures of parenting practices, child behavior problems, parenting stress, and psychological distress were used. Results showed a greater decrease over time of parents’ reported child behavior problems and harsh discipline, and a greater increase of reported positive parenting practices (praise and incentives) in the IY intervention group when compared to the control group. No significant differences were found in the use of inconsistent and appropriate discipline. Also, and contrary to expected, no significant effects of IY intervention were found on parenting stress and parental psychological well-being, a finding attributed to the fairly good initial levels of mental health in both the intervention and control groups, and the access of the parents in the control group to high-quality social and mental health services if needed. An additional study by the same authors reported that the parents from child protection services were committed and reasonably well engaged in the IY program, showing similar rates of attendance and satisfaction to non-referred parents (Karjalainen et al., 2020).

As well as more prolonged parenting interventions (at least 18-20 sessions; Webster-Stratton & Reid, 2010), other recommendations have been formulated for the IY Basic Parenting Program in order to overcome the particular challenges and barriers that may arise when working with families involved in the child welfare system. These recommendations include the addition of two components: individual home visits – in order to set up parent-child experiential practices, to provide support and reinforcement to parents for their efforts, and to make up missed sessions – and the IY Small Group Therapeutic Child Treatment program (Small Group Dinosaur curriculum) – to treat maltreated children’s problems with attachment, emotional regulation, social skills, and cognitive development (Webster-Stratton, 2014; Webster-Stratton & Reid, 2010). These are mainly clinical recommendations, as the empirical evidence for the benefits of parent training plus child therapy over parent training alone is still scarce and mixed (Larsson et al., 2009; Webster-Stratton & Hammond, 1997).

In Spain, some experiences have been made with the implementation and assessment of evidence-based programs in Child Welfare and Child Protection Services, such as the Strengthening Families Program (competenciafamiliar.uib.eu) or Safe Care (Arruabarrena et al., 2019). Recent years have also seen a strong push towards the implementation and evaluation of preventive positive parenting programs (Rodrigo, 2016; familiasenpositivo.org). However, implementation of evidence-based programs is still scarce, and further efforts are needed to test and scientifically evaluate them in order to improve outcomes for vulnerable children and families.

This study is the first to evaluate the implementation of the Incredible Years program in Spain. Our aim was to test through a randomized control trial the effectiveness of the IY Basic Parenting Program (IY-Parent) and the Small Group Therapeutic Child Treatment Program (Small Group Dinosaur Program, IY-Child) in a sample of maltreating and at-risk families referred to Child Welfare and Child Protection Services. We hypothesized that IY-Parent and IY-Child programs will be effective in improving parenting skills, reducing child behavior problems, and consequently reducing the risk of child abuse. Also, IY effects on related variables such as parenting stress and parents’ psychological distress were explored because, although they have been identified as relevant risk factors for child maltreatment (Barnhart & Maguire-Jack, 2016; Schaeffer et al., 2005; Stith et al., 2009), evidence about the effects of parenting programs on such variables is mixed, with some studies showing such effects (Barlow et al., 2014; Berliner et al., 2015; Furlong et al., 2012; Hutchings et al., 2007, 2012; Pinquart & Teubert, 2010; Weber et al., 2019) and others failing to confirm them (Chen & Chan, 2016; Dedousis-Wallace et al., 2021; Leijten et al., 2017). Finally, we explored whether post-intervention changes were maintained after the intervention ended, and the influence of family sociodemographic characteristics, parent participation in the program (couple vs. only one parent), and program attendance on intervention effects. We also explored the mediating mechanisms for parenting practices and parenting stress as predictors of child abuse potential and child behavior problems.

Method

Participants

One hundred and eleven families with 4- to 8-year-old children living at home were recruited from Child Welfare (CW) and Child Protection Services (CPS) of the region of Gipuzkoa (Spain). CW/CPS caseworkers recruited families with the following inclusion criteria: (1) there was a substantiated report or significant risk for child maltreatment, (2) children displayed significant behavior problems, and (3) parents had significant difficulties managing their children’s behavior. Sexual abuse cases, parents with severe mental health disorders, severe cognitive limitations or drug addiction, and children in temporary care, with diagnosis of neurodevelopmental disorders (e.g., autism), severe developmental delays, or undergoing psychotherapeutic or psychiatric intervention were excluded from the study.

Procedure

Participants (111 families) were randomized to Incredible Years or to a control group after the parents gave written consent to their CW/CPS caseworkers to receive parenting support services and to participate in the study. Families did not receive any financial or other type of compensation for participating. The Ethics Committee of the University of the Basque Country UPV/EHU approved the study protocol.

The unit of randomization was the child. It was controlled that at least one third of the children assigned to the Incredible Years group were girls. After consent, participants were blindly allocated using a computer-generated random number sequence by an independent researcher, to Incredible Years (IY; n = 62 families, 85 parents) or to the control group (CG; n = 49 families, 61 parents). Baseline (Time 1), post-intervention (Time 2; 6 month), and follow-up (Time 3; 12 month) assessments were conducted at families’ homes by an independent, trained evaluator. Although the evaluator should be blind to participants’ group membership, in many cases masking was not possible because families disclosed informative details. Between allocation and baseline assessment, 17% (n = 21) of participants dropped out the study: 9.6% (n = 6) in the IY group and 22.4% (n = 11) in the control group.

Socio-demographic characteristics of participants who completed the baseline assessment are shown in Table 1. No statistically significant differences were found between IY and control groups. Most of the children were boys (IY = 60.7%, CG = 71.1%), with a mean age of 6.61 years in the IY group (SD = 1.29) and 6.64 years in the control group (SD = 1.58). Most of the participants were mothers (IY = 73.7%, CG = 72.0%), although there were a significant percentage of fathers (IY = 26.3%, CG = 28.0%). Approximately one third of the parents (IY = 28.9%, CG = 36.0%) had only primary education. There were high percentages of immigrant parents (IY = 31.6%, CG = 36.7%), single-parent or separated/divorced families (IY = 60.7%, CG = 55.3%), and families with economic difficulties (IY = 35.7%, CG = 44.7%) in both groups. Most of the families (IY n = 33, 58.9%; CG n = 26, 68.4%) had at least one substantiated child maltreatment report, while the remaining families were at-risk (IY n = 23, 41.1%; CG n = 12, 31.6%).

Between baseline (T1) and post-intervention assessment (T2), families from the control group dropped out more frequently from the study (n = 9, 23.7%) than those from the IY group (n = 5, 8.9%). The difference was statistically significant, χ²(1) = 7.66, p = .006. Comparison between retained and lost families showed no differences in sociodemographic characteristics or dependent variables at baseline, with the exception of economic difficulties: parents who dropped out reported greater difficulties, χ²(1) = 4.28, p = .039. Between post-intervention (T2) and follow-up (T3) assessments, 19.6% of the families in IY (n = 10) and 17.2% of the families in the control group (n = 5) dropped out the study. No differences in sociodemographic characteristics or dependent variables at post-intervention were found between retained and lost families (see Figure 1).

Intervention

Families in the Incredible Years group received the Preschool Basic Parenting program and the Small Group Dinosaur program, delivered following their original format and content (Webster-Stratton, 2011). It was provided in 19 weekly 2-hour sessions (5-6 months) to groups of 10-12 parents and 6 children (with at least 2 girls per group). As recommended for child welfare populations (Webster-Stratton, 2014; Webster-Stratton & Reid, 2010), four 1- to 1.5-h one-to-one structured home visits (IY Home Visiting Coach Model) following group sessions 5, 9, 13, and 17 were added. In the IY-Parent program, parents view videotapes depicting parent models interacting with their children in various situations. In collaboration with two group leaders, who used an empowering approach, parents discussed these video vignettes, identified parenting principles, and put learned principles and techniques into practice through role-plays. In addition, home assignments and between-session telephone calls were used as part of the usual procedure of the program. Parenting skills emphasized included how to play with children, social, emotional, academic and persistence skills coaching, effective praise and use of incentives, establishing predictable routines and rules and promoting responsibility, effective limit-setting, and strategies to manage misbehavior and teach children to problem solve. For the IY-Child program, skills emphasized included emotional literacy, empathy or perspective taking, friendship skills, anger management, interpersonal problem-solving, and school rules. Teachers and parents received weekly information about the behaviors and concepts taught to children and suggestions for strategies they could use to reinforce skills taught. Children were assigned home activities to complete with their parents and received weekly good behavior-charts that parents and teachers completed. Parents’ and children’s group sessions took place at the same time in independent rooms in a family center. Supervised free childcare for other children in the family was provided when needed.

Ten IY-Parent and IY-Child groups were run during a two-year period. The intervention was delivered by four parent-leaders and three child-leaders previously trained over 12 months by accredited Incredible Years trainers. The training included attendance at two independent 4-day workshops for the parent-group and the child-group leaders, attendance of parent-group leaders at 1-day workshop for the Home Visiting Coach Model training, and monthly clinical support, supervision, and consultation sessions. To participate in the trial, the group leaders must have received a positive evaluation by IY trainers, and be accredited or undergoing the accreditation process (for detailed information about the previous phase of preparation of the pilot implementation, see De Paúl, Arruabarrena, et al., 2015). All leaders had backgrounds in psychology. During the trial, group leaders received two-monthly clinical support, supervision, and consultation sessions from an IY-accredited mentor, and attended monthly coordination meetings. To ensure fidelity, they adhered to standard program manuals, protocols, and teaching methods (video vignettes, homework, role-plays), and completed protocol checklists after each session. All group sessions were video recorded and subsequently reviewed.

Percentages of parents and children who attended thirteen or more group sessions were high: 74.3% of the parents and 83.9% of the children. The percentage of families who dropped out of the IY program was low (5.6%). Some families (11.7%) received additional supportive services during the trial (e.g., counselling or home visiting from Child Welfare or Child Protection Services workers, stimulation for children with neurodevelopmental delays).

Families in the control group received standard services from Child Welfare and Child Protection Services. Seventy percent of them (71.9%) received parent counselling or parent training, in individual or group formats, at home or outside. These interventions were non-structured and highly variable in their procedure, frequency, and content. The remaining 28.1% received CW/CPS caseworker follow-up, also with a non-structured format. Almost thirty percent (28.1%) of the children received direct non-structured and diverse therapeutic or supportive services in individual or group format. Half of the families received two or more services (number of services per family M = 1.25, SD = 0.92). No information was available about intervention dropouts in the control group, where families received standard services as long as they needed according to CW/CPS caseworker assessment. During the study, families in the control group were not offered participation in the IY intervention after Time 3 (12 month) assessment.

Instruments

Families in the IY and control groups were assessed at home, using standardized instruments by a trained clinical psychologist. Parent reports (at baseline, post-intervention Time 2, and follow-up Time 3) and an observational measure of parent-child interaction (at baseline and post-intervention Time 2) were used. Procedure and measures were the same for each group and at each time point. Participants in the IY-Parent program also completed a satisfaction questionnaire when finished.

Parenting Practices Interview (PPI; Webster-Stratton et al., 2001). The PPI consists of 64 items rated by parents of children aged 3 to 12 years old on a seven-point scale (1 = never/totally disagree to 7 = always/totally agree) that assesses seven dimensions: appropriate discipline, positive verbal discipline, praise and incentives, clear expectations, monitoring, harsh and inconsistent discipline, and physical punishment. For the present study, a recent adaptation of the PPI with a Spanish sample (Rivas et al., 2021a, submitted for publication) was used. The PPI adaptation consisted of 25 items assessing four dimensions: appropriate discipline (7 items, e.g., “Take away privileges like TV, playing with friends”), verbal praise and incentives (7 items, e.g., “Give your child a hug, kiss, pat, handshake for a good behavior”), inconsistent discipline (5 items e.g., “Threaten but do not punish”), and physical punishment (6 items e.g., “Give your child a spanking”). Cronbach’s alpha coefficients with the present sample ranged from moderate to good: appropriate discipline (.77), verbal praise and incentives (.70), inconsistent discipline (.77), and physical punishment (.87).

Parenting Stress Index/Short Form (PSI-SF; Abidin, 1995). The PSI-SF is a 36-item, self-report measure of parenting stress. It includes three subscales: parental distress (PD, e.g., “I feel lonely and without friends”), parent-child dysfunctional interaction (PCDI, e.g., “Sometimes I feel my child doesn’t like me and doesn’t want to be close to me”), and difficult child (DC, e.g., “My child gets upset easily over the smallest thing”). Each subscale consists of 12 items rated from 1 (strongly disagree) to 5 (strongly agree), with scores ranging from 12 to 60. A Total score is calculated by summing the three subscale scores, ranging from 36 to 180. Abidin (1995) reported Cronbach’s alpha coefficients of .91 for the PSI-SF total score, and .87, .80 and .85 for the PD, PCDI, and DC subscales, respectively. The PSI-SF version validated with Spanish population (Rivas et al., 2020) was used in the present study, with satisfactory internal consistency indexes for the total score (α = .93) and all three dimensions (Cronbach’s alphas of .86, .91, and .85).

Beck Depression Inventory-II (BDI-II; Beck et al., 1996). The BDI-II is a 21-item, self-report measure of depressive symptomatology appropriate for both psychiatric and normative populations. Responses are given using a four-point scale from 0 to 3 (e.g., 0 - “I do not feel like a failure”; 1 - “I have failed more than I should have”; 2 - “As I look back, I see a lot of failures”; 3 - “I feel I am a total failure as a person”), with scores ranging from 0 to 63 and higher scores indicating higher levels of depressive symptomatology. The BDI-II has been shown adequate reliability (between .92 and .93 for internal consistency) as well as adequate construct validity (Beck et al., 1996). The BDI-II has been validated for its use with Spanish population (Sanz et al., 2003). In the present study, internal consistency was also satisfactory (Cronbach’s alpha of .87).

Brief Child Abuse Potential Inventory (B-CAP; Ondersma et al., 2005). The B-CAP is a self-report screening questionnaire with 34 items. It is composed of the Abuse scale, measuring the risk of a parent physically abusing their children, and two validity scales: a three-item random response scale and a six-item lie scale. The Abuse scale of the Spanish version of the B-CAP was used in this study (Rivas et al., 2021b). Responses are on a binary scale (agree-disagree), so scores range from 0 to a maximum of 22. Ondersma et al. (2005) indicated good internal consistency for the Abuse scale (KR20 = .89). In the present study the internal consistency for the Abuse scale was also good (KR20 = .83).

Eyberg Child Behavior Inventory (ECBI; Eyberg & Pincus, 1999) is a parent-rating scale covering 36 child disruptive behaviors with two subscales. The Intensity subscale measures the frequency of the child’s behavior (e.g., “Acts defiant when told to do something”, “Refuses to go to bed on time”) on a seven-point scale, ranging from 1 to 7 with a minimum score of 36 and a maximum of 252. The Problem subscale measures the extent to which the parent finds the child’s behavior troublesome, rated on a binary scale (0 = no, 1 = yes) with a score range from 0 to 36. Eyberg and Pincus (1999) reported high internal consistency for both Intensity and Problem subscales (α = .95 and KR20 = .94, respectively). The ECBI has been translated and validated with Spanish population (García-Tornel et al., 1998). In the present study, both Intensity and Problem subscales showed high internal consistency (α = .91 and KR20 = .88).

Dyadic Parent-Child Interaction Coding System-IV (DPICS-IV; Eyberg et al., 2014). The DPICS-IV is an observational instrument that requires videotaping 25 minutes of semi-structured parent-child interaction of three standardized situations with varying parental control levels. The procedure starts with a Child-Led Play (CLP) situation of 10 minutes, where the child plays freely, and the caregiver is expected to follow the child. In the next 10 minutes, Parent-Led Play (PLP), the caregiver is encouraged to choose the activity and lead the play. In both situations, the first five minutes are for warming-up, and only the second five minutes are coded. The last five minutes includes the Clean-Up (CU) task, where the caregiver informs the child that it is time to pick up the toys. Therefore, the codification takes 15 minutes of the total videotaped time. For the present study, a Spanish adaptation of the DPICS-IV clinical version was used (Cañas et al., 2021) and two dimensions of parent behavior were analyzed: Praise (e.g., “The flower you drew is amazing”) and Negative Talk (e.g., “The flower you drew is a mess”). Interrater reliability in DPICS items was completed by two PhD candidates with certified training in DPICS, based on the double coding of 15% of randomly selected videotapes from the total sample. The interclass correlation coefficients (ICC) for both Praise and Negative Talk were above .95, indicating good interrater reliability. One of the coders was blind to participants’ group membership, whereas the other – who was the same person who conducted the observation at home – was aware of the group membership of some families.

Incredible Years Parenting Program Satisfaction Questionnaire (www.incredibleyears.com/for-researchers/measures/). The IY-Parent Program Satisfaction Questionnaire was developed by the IY program and uses a seven-point scale at the end of the program to measure parental satisfaction with the overall program, the usefulness of the teaching format and the parenting techniques used, and the parent and child group leaders’ skills. Parents could also express their feelings and opinions about the program in an open-response question.

Data Analysis

Differences between groups at baseline were analyzed with chi-square for categorical data and t-tests for continuous variables. Analyses were performed using SPSS 26.0. Regardless of their actual participation, data from every parent allocated to IY or to control groups were included in the analyses. Only participants who completed every instrument at each assessment time were included in the respective analysis. In the ECBI, each parent was considered independently even if they participated as a couple, because individual perception of child behavior problems was the focus of interest of the study. For families with more than one child participating in the study, the child with the highest score in the ECBI Intensity scale at baseline was selected.

To evaluate differences between IY and control groups in Time 1 (baseline), Time 2 (post-intervention, 6 month), and Time 3 (follow-up, 12 month) assessments, univariate and multivariate analysis of covariance (ANCOVAs) were used, including previous Time scores as covariates. Effect sizes (ES) were calculated with partial eta square (η_p²) and classified according to Cohen’s principles: .01 for a small effect, .06 for a medium effect, and .14 for a large effect size. Paired samples t-test were also calculated between Time1-Time2, Time2-Time3, and Time1-Time3 assessments for each group. Cohen’s d was used to calculate effect sizes, d ≥ 0.20 was considered a small effect, d ≥ 0.50 a medium effect, and d ≥ 0.80 a large effect. These analyses were also used to analyze post-intervention intragroup differences based on severity of child behavior problems (low, medium, and clinical range) and on IY attendance (less or more than 13 sessions).

Moderation and mediation analyses were performed using the SPSS Macro Process (Hayes, 2013). For moderation analyses, the baseline score of the outcome variable was controlled by including it in the regression. For the mediation analysis, new variables (amount of change) were built based on baseline and post-intervention scores. For negative parenting practices, parenting stress, child abuse potential and child behavior problems, the amount of change was calculated from baseline minus post-intervention scores (T1-T2). For positive parenting practices, it was calculated from post-intervention minus baseline scores (T2-T1). Bootstrap procedures with 10,000 samples were used to test the significance of the mediating effects, with mediation considered to be occurring when the indirect effect was significant with 95 % confidence intervals not containing zero (Hayes, 2013).

Results

Differences at Baseline and Post-intervention between Incredible Years and Control Groups

No differences at baseline between IY and control groups were found in any of the outcome measures included in the study (p > .05).

Comparisons between baseline and post-intervention scores along with results of paired t-test and ANCOVAs are presented in Table 2.

Self-reported parenting practices. Regarding PPI positive parenting practices, paired t-tests between baseline and post-intervention showed that only parents in the IY group reported a significant increase with a medium effect size, t(68) = -3.45, p < .001, d = 0.53, in the use of verbal praise and incentives. No significant changes were observed in any group in parent reports of appropriate discipline. ANCOVA showed that the difference in verbal praise and incentives between IY and control group was significant at post-intervention. Parents in IY reported a larger improvement (medium-large size) in the use of verbal praise and incentives (p < .001, η_p² = .12) than parents in the control group.

In negative parenting practices, parents in both groups reported significant decreases in PPI scores of inconsistent discipline, IY t(68) = 5.55, p < .001, medium effect size d = 0.66; CG t(33) = 2.18, p < .05, small-medium effect size d = 0.42, and physical punishment, IY t(68) = 6.15, p < .001, medium effect size, d = 0.68; CG t(33) = 3.04, p < .005, small-medium effect size, d = 0.49. There were significant post-intervention differences between groups (ANCOVA) for inconsistent discipline: parents in the IY group reported a larger decrease (medium size) in the use of inconsistent discipline (p < .005, η_p² = .07) than parents from the control group.

Observed parent-child interaction. Unlike the PPI self-report measure, a paired t-test did not show significant changes between baseline and post-intervention assessments in the DPICS Praise dimension in the IY group. Contrary to expectations, in the control group a significant decrease in DPICS Praise, t(19) = 2.70, p < .05, small-medium size d = 0.47, was found. ANCOVA showed that the difference at post-intervention between IY and control groups was significant with a medium effect size (p < .005, η_p² = .10): parents in the IY group demonstrated a greater improvement in their observed use of praise than parents in the control group.

In line with the negative parenting dimensions of PPI self-report, results indicated a significant reduction in DPICS Negative Talk dimension only for the IY group, t(59) = 4.04, p < .001, medium effect size d = 0.57. However, no significant differences were observed between IY and control groups at post-intervention.

Parenting stress. Paired t-test showed that parents in the IY group reported significant decreases in their perception of parenting stress, t(67) = 4.43 p < .001, small-medium effect size d = .48, feelings of parental distress, t(67) = 3.47, p < .001, small effect size d = 0.35, and their perception of having a difficult child, t(67) = 6.18, p <.001, medium effect size d = 0.68. In the control group, only a significant decrease of parental perception of having a difficult child was found, t(32) = 2.08, p < .05, small effect size d = 0.27. ANCOVA confirmed significant differences between groups in parenting stress at post-intervention: parents in the IY group reported larger decreases (small-medium and medium sizes, respectively) for both parenting stress (PSI-SF total score; p < .05, η_p² = .04) and perception of having a difficult child (PSI-SF DC; p < .05, η_p² = .06) than parents in the control group.

Parental depressive symptomatology. Paired t-test showed that only parents in the IY group reported a significant decrease between baseline and post-intervention in BDI-II scores, t(67) = 4.40, p < .001, medium effect size d = 0.53. No significant differences were found in the control group. ANCOVA confirmed a small-medium effect (p < .05, η_p² = .04), indicating that IY parents reported a greater decrease at post-intervention in their depressive symptomatology than parents in the control group.

Child abuse potential. Paired t-test showed that only parents in the IY group reported a significant decrease with a small effect size between baseline and post-intervention in BCAP scores, t(68) = 3.56, p < .001, d = 0.30. No significant changes were found in the control group. No significant difference between IY and control group was observed at post-intervention.

Parental perception of child behavior problems. Paired t-test showed that parents in both groups reported significant decreases in the intensity of perceived child behavior problems, IY t(65) = 8.81, p < .001, medium-large effect size d = 0.73; CG t(28) = 4.19 p < .001, small effect size d = .36, and in the level at which those behaviors were troublesome for them, IY t(65) = 4.99, p < .001, medium effect size d = 0.59; CG t(28) = 5.99, p < .005, small effect size d = 0.33. ANCOVA confirmed significant differences between groups at post-intervention, with parents in the IY group reporting larger decreases (with medium and small-medium effect sizes, respectively) in both measures (p < .005, η_p² = .08; p < .05, η_p² = .05).

Additional analyses explored patterns of change between baseline and post-intervention assessment according to the initial severity of perceived child behavior problems. Children were classified in three levels according to ECBI Intensity scores at baseline: low (≤ 90), medium (91-129), and clinical range (≥ 130). Chi-square tests did not show significant differences between IY and control groups in the percentage of children in each severity level at baseline. As can be seen in Table 3, parents in both groups reported significant large size decreases of ECBI scores for children in the clinical range, IY t(22) = 6.36, p < .001, d = 1.53; CG t(7) = 5.09, p < .001, d = 2.41. The percentage of children in the clinical range decreased 20% in both groups between baseline and post-intervention. However, only parents in the IY group also reported significant decreases of ECBI scores for children with low, t(20) = 4.84, p < .001, and medium, t(23) = 5.89, p < .001, severity behavioral problems at baseline. Such decreases were also of large size (d = 0.96 and d = 1.37, respectively).

Moderator effect of sociodemographic variables. Sociodemographic variables such as children’s age and gender, parents’ gender, educational level and country of origin, and family economic difficulties were tested at baseline as possible moderators of post-intervention measures. No significant effects were found.

Effect of the intervention with both parents or with only one parent. Moderation analyses were performed with the Incredible Years group to explore the effect of the intervention being done with the couple (both parents participate, n = 19 families) or with only one parent (only the mother participate, n = 31 families). No moderation effects on IY outcomes were found. Also, results for ANCOVAs comparing IY and control groups measures at post-intervention excluding fathers (that is, including only one parent – the mother – per family) were similar to results found including fathers, except for PPI Inconsistent Discipline dimension, F(1, 74) = 1.21, p = .275, η_p² = .02, and parental depressive symptomatology, F(1, 74) = 1.59, p = .211, η_p² = .02, where differences between groups were no longer significant.

Mediational Models of Change

Two mediation models of change were conducted with Condition (IY group = 1; control group = 0) as the predictor variable, changes in parenting practices (self-report PPI and observation DPICS) and parenting stress (PSI-SF) as serial mediator variables, and change in child abuse potential (BCAP) and perception of child behavior problems (ECBI-Intensity Scale) as the two predicted variables.

The mediational models were tested for both positive parenting (PPI Verbal Praise and Incentives dimension and DPICSs Praise category) and negative parenting (PPI Inconsistent Discipline dimension). Mediation analyses were conducted separately for each self-reported (Verbal Praise and Incentives, and Inconsistent Discipline dimensions from the PPI) and observed (DPICS Praise category) variable. Therefore, three mediational models were tested for each predicted variable (BCAP and ECBI). Because significant results were observed only for self-reported positive parenting measures (PPI Verbal Praise and Incentives), only these findings will be presented.

Mediational model of change in child abuse potential (BCAP). As can be seen in Figure 2, intervention had a fully mediated effect in the change of child abuse potential via changes in positive parenting and in parenting stress. The mediating effect of positive parenting was observed only when it was measured through parents’ self-report (β = .12, SE = .07, 95% CI [.002, .278]). When compared to the control group, parents who participated in IY reported a greater change in PPI positive parenting at post-intervention (β = .45, SE = .19, p = .019), which furthered a greater change in PSI-SF parenting stress (β = 4.27, SE = 1.71, p = .014), which in turn led to a greater change in BCAP child abuse potential (β = .06, SE = .02, p = .004).

Mediational model of change in child behavior problems (ECBI). As can be seen in Figure 3, the treatment had a fully mediated effect in the change of child behavior problems via changes in positive parenting and in parenting stress. The mediating effect of positive parenting was observed only when it was measured with self-report (β = 1.08, SE = .68, 95% CI [.005, 2.56]). When compared to the control group, parents who participated in IY reported a greater change in PPI positive parenting at post-intervention (β =.45, SE = .19, p = .019), the change in positive parenting promoting greater changes in PSI-SF parenting stress (β = 4.27, SE = 1.71, p = .014), which in turn led to a greater change in ECBI child behavior problems (β = .56, SE = .14, p = .000).

Maintenance of Post-intervention Effects at Follow-up

From post-intervention (T2) to follow-up six months later (T3), paired t-test showed that parents in both IY and control groups reported significant additional decreases in their perception of child behavior as troublesome, IY t(58) = 6.44, p < .001, large effect size d = 0.89; CG t(26) = 3.54, p < .001, medium effect size d = 0.67. Also, parents in the IY group continued to report significant additional small size decreases in parental distress, t(58) = 2.76, p < .001, d = 0.28, and child abuse potential, t(58) = 2.44, p < .05, d = 0.21. However, no significant differences between IY and control groups were found between post-intervention and 12-month follow-up in such measures.

Neither significant differences between post-intervention and 12-month follow-up nor significant differences between IY and control groups were found for the remaining variables in which post-intervention effects were observed (PPI verbal praise and incentives, PPI inconsistent discipline, PSI-SF total stress, PSI-SF difficult child, BDI-II parental depressive symptomatology, and ECBI intensity), indicating that the effects were maintained over time.

Table 4 summarizes the effect/change sizes observed in the self-reported outcome measures between assessments: from baseline (T1) to post-intervention (T2 - 6 month), from post-intervention (T2 - 6 month) to follow up (T3 - 12 month), and from baseline (T1) to follow up (T3 - 12 months).

Relationship between Incredible Years Attendance and Post-intervention Effects

The relationship between IY-Parent attendance and post-intervention effects was analyzed dividing parents into two groups: those who attended less than 13 sessions (n =16), and those who attended 13 or more sessions (n = 53). No significant differences were found between groups (p > .05) in sociodemographic characteristics or any outcome variable at baseline. ANCOVAs did not indicate significant differences at post-intervention between groups on any outcome measure except for PPI inconsistent discipline: parents who attended 13 or more sessions reported lower scores at post-intervention (M = 2.23, SD = .83) than parents with lower attendance (M = 3.01, SD = 1.42), and a larger decrease in the use of inconsistent discipline, F(1, 66) = 5.13, p < .05, with a medium effect size (η_p² = .07). Mean attendance of couples was higher (M = 80.5%, SD = 21.44) than parents who assisted alone (M = 71.7%, SD = 29.01).

Parent Satisfaction with the Incredible Years Program

At post-intervention, 86.5 % of the parents who participated in the IY program reported that they were satisfied or very satisfied with their children’s progress, 100.0% would recommend or highly recommend the program to a friend or relative, 98.6 % had positive or very positive feelings about the program, and 94.5 % were confident or very confident in their ability to manage future behavior problems in the home.

Discussion

This study presents the results of the first randomized controlled trial carried out in Spain to test the effectiveness of the Incredible Years (IY) program. The Basic Parenting (IY-Parent) alongside the Small Child Dinosaur treatment (IY-Child) curricula were provided by previously trained practitioners to a sample of families with children aged 4-8 years in child welfare due to substantiated or risk for child maltreatment.

Baseline (T1), 6-month post-intervention (T2) and 12-month follow-up (T3) assessments were compared between two groups of families, those who participated in Incredible Years, and a control group who received standard services. Larger positive changes were expected from T1 to T2 in the group of parents and children who participated in the IY-Parent and IY-Child programs in terms of parents’ self-reported and observed parenting practices, parenting stress, depressive symptomatology, child abuse potential, and perception of child behavior problems. We also analyzed whether post-intervention changes were maintained six months after post-intervention (T3), as well as the influence of sociodemographic characteristics and program attendance on IY intervention effects. Finally, mediating mechanisms for parenting practices and parenting stress as predictors of child abuse potential and child behavior problems were explored.

Our results showed that, when IY and control groups were compared, the IY-Parent plus IY-Child interventions made a significant positive difference from baseline to T2 in parents’ reported and observed use of positive parenting practices such as praise, accompanied by a significant reduction of reported inconsistent discipline, parenting stress, parental depressive symptomatology, and perception of child behavior problems. No significant differences between the IY and the control groups were found in parents’ reports of appropriate discipline, physical punishment, child abuse potential, and observed negative talk.

Our IY effects on the increase of parental praise and the reduction of parents’ perception of child behavior problems were similar to those found by Letarte et al. (2010) and Karjalainen et al. (2019) with child welfare referred parents who received the IY Basic Parenting program alone. In contrast, while Karjalainen et al. (2019) did not find any difference between the IY Basic Parenting and control groups, we found significant IY effects on parenting stress and depressive symptomatology. Maybe in our case the addition of the IY-Child curricula contributed to the differences, or maybe they were related to Karkalainen’s control group having access to high quality mental health services (which is not always the case in Spain). Also noteworthy in our study was the concordance between self-reported and observed measures of IY effects on parental praise, which strengthens this finding. This is important, and in line with one of the main focuses of the Incredible Years program: the promotion of positive parent-child communication and interaction patterns. Also, IY parents showed a significant medium-sized reduction in observed negative talk toward their children, although the difference with the control group was non-significant.

Focusing on within-group effect sizes, families who participated in the IY-Parent and Child programs experienced significant medium and large-sized positive changes from baseline to post-intervention (T2) in self-reported parental measures of parenting practices (increase of verbal praise and incentives, and reduction of inconsistent discipline and physical punishment), observed negative talk toward their children, depressive symptomatology, and perception of child behavior problems, as well as significant small and small-medium positive changes in child abuse potential and parenting stress. Parents in the control group also reported significant positive changes in parenting practices (specifically, a reduction of inconsistent discipline and physical punishment) and perception of child behavior problems, although with small and small-medium effect sizes. Such findings suggest that, in our context, standard parent training and supportive services provided by Child Welfare and Child Protection Services to maltreating and high-risk families can be effective in promoting some positive changes, although fewer and less intense than the IY program. Also, attention should be paid to the significant reduction in parent praise observed in the control group from baseline to T2 as this may indicate a worsening in some indicators of the parent-child relationship.

Two full serial mediation effects were found between participation in IY-Parent plus IY-Child programs, positive changes in parenting practices, subsequent reduction of parenting stress, and final reduction of both perception of child behavior problems and child abuse potential. This finding provides support to the importance of intervening in parenting practices and parenting stress when the goal of the intervention is the reduction of child behavior problems and the prevention or reduction of child maltreatment. More studies are needed along these lines to explore which specific components of parenting practices are related to changes in child behavior problems, as investigated by Altafim et al. (2021). In their study with 143 Brazilian socioeconomically disadvantaged mothers of children aged 3-8 years, mothers’ emotional and behavioral regulation – that is, avoidance of negative practices such as spanking, hitting or yelling – emerged as a core mechanism in explaining parenting program effects on reducing children’s internalizing and externalizing behavioral problems, particularly for children with high levels of behavioral difficulty at baseline. On the other hand, our results suggest that the combination of IY-Parent plus IY-Child programs might contribute to preventing and reducing child maltreatment recurrence in this sample of at-risk and maltreating families. The small size changes observed in parent self-reports of child abuse potential and the larger changes observed in variables acting as risk factors for maltreatment, such as negative or dysfunctional parenting practices, parenting stress, parents’ psychological distress, or child behavior problems (Austin et al., 2020), should be considered positive signs. However, it is important to keep in mind that these changes do not necessarily reveal or reflect a real prevention or reduction of child maltreatment, and that we assessed child abuse potential through a self-report measure (Brief Child Abuse Potential Inventory BCAP; Ondersma et al., 2005). Only objective measures – such as Child Welfare and Child Protection Service reports – can really show whether IY has proved effective preventing the onset or recurrence of child maltreatment.

In the present study, no moderating influence on IY effects was found for child gender and age, parent gender, educational level and country of origin, and economic difficulties in the family. Studies carried out in other countries with the IY-Parent program have also found no evidence of moderating effects of family characteristics, such as single parenthood, ethnic minority, and parental educational level. This finding has been attributed to different reasons, for example, methodological issues of the studies, the capacity of IY to be tailored to specific characteristics and needs of families, or the reduction of differences between families due to the group format (Menting et al., 2013). The above, however, does not mean that the IY-Parent program is necessarily a valid approach for all families with child behavior problems receiving child welfare. Some parents may need to address other problems (e.g., severe mental health problems or substance addition, intimate partner violence) before participating in a parent training program, or they have problems which prevent them from participating in a group-based intervention, thus benefiting more from an individual approach. The number of parents participating in the program (couple vs. only one parent) did not moderate IY effects, even if we found that parent attendance was higher for couples. Such finding differs from from other studies that have found that fathers’ involvement increased and sustained intervention outcomes (Bagner, 2013; Bagner & Eyberg, 2003; Lundahl et al., 2008; Panter-Brick et al., 2014; Webster-Stratton, 1985). Further analysis is needed, as the effects of one parent’s involvement on the other and on intervention outcomes, and the effects of couple vs. only one parent intervention involvement on short- and long-term outcomes. Parents’ involvement measures should go beyond attendance to include participation in program activities during and between sessions.

In the present study, the post-intervention effects on the explicit targets of the IY program (parenting behaviors and child behavior problems) were extended to other family characteristics such as parenting stress and parents’ psychological wellbeing. As mentioned previously, there is mixed evidence regarding the effects of parenting programs on these two variables. In our case, findings aligned with other studies that have found such associated or cascading effects (Barlow et al., 2014; Berliner et al., 2015; Furlong et al., 2012; Hutchings et al., 2012; Hutchings et al., 2007; Pinquart & Teubert, 2010; Weber et al., 2019), and support Barth (2009) in arguing that:

The evidence that parent education cannot succeed unless other family problems are also addressed is anecdotal and weak—at least as much evidence suggests that first helping parents to be more effective with their children can help address mental health needs and help improve the chances of substance abuse recovery. [….] sources of family adversity as marital conflict and depression can be alleviated in two different ways: by directly treating partner social support and depression through direct interventions aimed at parenting problems and by improving parenting skills. […] rather than deciding who gets mental health interventions to reduce depression based on parents’ entry characteristics, it may be more cost-effective to offer an initial standard parent training program. Practitioners can track how successfully parents progress through the program and continue to monitor other family risk variables, such as continuing marital conflict, depression, and stress, that may interfere with treatment success. Only when program managers see no improvement in child behavior or in measures of the parental or family distress that interferes with the parenting program should they add interventions targeting the specific risk factors of ongoing concern. (p. 109)

This suggestion by Barth (2009) was adopted by the APSAC Task Force on Evidence-Based Service Planning Guidelines for Child Welfare in its recommendation that the priority focus of the intervention in child maltreatment cases should be the improvement of parenting skills and the parent-child relationship, along with the consequences of maltreatment on the child (Berliner et al., 2015). The APSAC Task force recommended pursuing few targets in depth and with intensity, avoiding supplemental services unless essential. As found in some studies, more is not always better and in some cases such ancillary services “may present an overwhelming burden or impede parents’ ability to focus on and master parenting skills” (Kaminski et al., 2008, p. 581).

Regarding other major findings of the present study, overall post-intervention effects remained stable over time in the IY and control groups, as suggested by non-significant differences between T2 and T3 assessments. Additional improvements were even found in both groups regarding child behavior problems, and in the IY group in self-reported measures of parental distress and child abuse potential. This maintenance – and in some cases improvement – of intervention effects on child behavior problems is in line with the findings of van Aar et al. (2017), who reviewed evidence of 40 trials for three patterns of long-term effects: sustained (maintenance of improvements, with no further support provided), fade-out (undoing of some of the improvements and fallback to previous problems), and sleeper effects (gradually increased intervention effects over time). They found evidence that changes in children’s disruptive behavior following parent training interventions remained stable at least until three years follow-up. However, they cautioned that, although less frequently, fade-out and sleeper effects also occurred. Thus, although it can be expected that positive parent training outcomes persist once the intervention has finished, more knowledge is needed to identify those families likely to show sleeper effects who might need more time to change, and those families likely to show fade-out effects who might benefit from booster sessions or additional support to prevent fallback (van Aar et al. 2017). This may apply to economically disadvantaged families, who, although benefitting as much as non-disadvantaged families from parent training in the short term, might experience more trouble maintaining positive outcomes in the medium-long term (Leijten et al., 2013).

In the present study, the percentage of children in the clinical range according to their parents’ reports decreased 20% in both Incredible Years and Control groups between baseline and post-intervention. The finding that children with more marked levels of behavior problems demonstrated greater intervention effect sizes is common in parenting programs (e.g., Altafim et al., 2021; Hautmann et al., 2011; Lundahl et al., 2006; Nowak & Heinrichs, 2008). The meta-analysis of Menting et al. (2013) found that initial severity of child behavior problems was a significant predictor of the IY-Parent program outcomes, with larger effect sizes found for studies which included more severe cases as well as for treatment vs. prevention studies. This has been explained by children with more severe behavior problems having greater scope for improvement, and/or their parents potentially being more motivated to accept help, modify their own behavior, and attend sessions (Kaminski et al., 2008; Menting et al., 2013). Based on these findings, it has been suggested that the IY-Parent program might be more suitable for treatment and indicated prevention than for universal and selective prevention purposes (Gardner & Leijten, 2017; Scott et al., 2014). In the present study, it is remarkable that children with initial lower levels of behavior problems also demonstrated large effect sizes in the IY group, which was not the case in the control group, where no differences between ECBI scores were found from baseline to post-intervention.

Another interesting topic explored in the present study was the relationship between parents’ IY attendance and intervention effects. Although some studies with child welfare families have found a dose-response relationship (Hurlburt et al., 2013), we did not find any evidence of such a relationship. This may be due to our high attendance rates, with 74.3% of the parents and 83.9% of the children attending thirteen or more sessions, and because our program curricula included four additional home visits to provide make-up sessions for parents who had missed group sessions and to enhance the parent group learning. In any case, the meta-analysis of Menting et al. (2013), as well as the guidelines of the IY developers (Webster-Stratton, 2014; Webster-Stratton & Reid, 2010), provide empirical evidence and clinical support for the recommendation that a minimum number of sessions need to be attended to obtain positive outcomes. For high-risk and maltreating parents, 18 sessions are recommended, a figure which according to the meta-analysis by de Euser et al. (2015) lies inside the range for producing higher effect sizes in reducing or preventing child maltreatment. This meta-analysis found a curvilinear association between program effect sizes on parenting behavior and program duration and number of sessions: while higher effect sizes were found for programs of moderate duration (6-12 months) or number of sessions (16-30 sessions), shorter or longer duration or number of sessions did not improve intervention outcomes. Again, such studies support the argument that more is not always better.

The present study contributes to the emerging experiences and literature on evidence-based parenting programs for Spanish families, and offers preliminary support for the benefits of a new well-researched program in our country. Moreover, the high level of engagement of the families in the IY-Parent and Child programs (low dropout and high attendance rates) as well as the high degree of parental satisfaction, reinforce the program’s transportability to Spain. Two main general conclusions can be drawn from our findings. First, they strengthen the evidence based on the effectiveness of Incredible Years in bringing about significant positive changes in parenting practices and child behavior problems in real-world settings, with different populations and in countries and sociocultural contexts different from those of its origin (Gardner & Leijten, 2017; Menting et al., 2013; Pidano & Allen, 2015). As described in a previous paper, the adaptation of the IY-Parent and Child programs for implementation in Spain did not need more than surface adaptations (translation and modification of vocabulary and replacement of cultural references) and additional training for practitioners in the use of positive reinforcement towards parents and children (De Paúl, Arruabarrena, et al., 2015). Second, in line with other studies (Hurlburt et al., 2013; Karjalainen et al., 2019; Letarte et al., 2010), our findings provide additional support for the benefits of the IY model in changing parenting practices and reducing child behavior problems among parents and children receiving child welfare because of substantiated reports or risk of child maltreatment. Such benefits were obtained following the adaptations recommended by Webster-Stratton (2014) and Webster-Stratton and Reid (2010) for applying the program to these families: increased program dosage (minimum of 18 two-hour sessions); addition of four home visits to coach parent-child interaction patterns and make up for missed group sessions; addition of the Small Group Dinosaur program; provision of practical assistance to facilitate group attendance (e.g., childcare, transportation); increased efforts in alliance-building techniques; increased focus on key topics (such as parent-child attachment, emotion and social coaching, parental attributions and self-talk, positive discipline, monitoring and self-care); and coordination with child protection service caseworkers. Although the Advanced program is also recommended alongside the IY-Parent program for maltreating families, it was not applied in the present study. Further studies are needed to test the additional benefits of the Advanced program, as well to explore whether the combination of IY components (parents, children, and classroom-based components) increases the effect sizes for intervention outcomes – particularly the combination of the IY-Parent and Child programs over the IY-Parent program alone – as well as the conditions in which these outcomes are produced (moderator variables). Research in this respect in scarce.

Several limitations of the present study should be taken into consideration. First, the high number of drop-outs after trial allocation, which substantially reduced the sample size across successive assessments, thereby limiting the strength and generalizability of results as well as intergroup and intragroup comparisons. It is possible that offering some kind of compensation to families (e.g., financial) would have resulted in fewer drop-outs. Second, given the highly time-consuming nature of the observational measures of parent-child interaction, these were only used in baseline and post-intervention assessments, not for follow-ups. Despite this limitation, the use of observational measures should be valued as a notable feature of the present study since these measures are less prone to biases than self-report measures. Although observation may of course also be affected by biases (e.g., parents who receive parent training may be aware of the specific behaviors that would be socially desirable during the in-home observations), such reactivity does not seem to pose a substantial problem (Hurlburt et al., 2013). Third, it was not possible to guarantee that evaluators were blind to the participants’ group membership. Fourth, although the study provides evidence regarding the impacts of the IY intervention on potential risk factors for child maltreatment, it does not provide evidence regarding its direct impact on maltreatment, which is an important area for further research. Fifth, in some analyses of the present study the same child was included twice. This was because the perception of each parent about himself/herself, his/her child’s behavior problems, and their observed behavior toward their children were the focus of the assessment. Since the intervention can have differential effects on parents, each was analyzed independently even if they participated as a couple.

The findings of the present study are promising and encourage testing the IY-Parent and Child programs with new Spanish populations, both in child welfare – e.g., children with different ages, foster families (Bywater et al., 2011; Linares et al., 2006; McDaniel et al., 2011; Nilsen, 2007) – as in other fields – e.g., children receiving mental health services for conduct problems and ADHD. Longitudinal studies are needed with larger samples and longer follow-ups, which would make it possible to increase the evidence regarding long-term results of all IY interventions for various ages, diagnoses, and demographic populations. Furthermore, it is crucial to expand knowledge of which components of the IY program produce more benefits, for which type of families and under which conditions, as well as whether the combination or addition of IY components (parents, children and classroom-based components) increases effect sizes of intervention outcomes.

Implications for Practice

The group-based Incredible Years approach merits the attention of policymakers, agencies, and practitioners as a particularly relevant preventive and rehabilitative evidence-based approach in the field of child welfare because it has been demonstrated to be efficient, can be cost-effective, and can promote the participation of parents who might be reluctant to individual approaches (Hurlburt et al., 2013). The present study provides evidence that transporting IY-Parent and Child programs with fidelity to Child Welfare and Child Protection Services in Spain is feasible, that it is a well-accepted approach by practitioners and families, that it promotes positive outcomes similar to those found in other Western countries, and that its benefits are greater than those of current standard services.

It is well known that full implementation of evidence-based programs in real-world settings is not easy. It requires a sustained commitment of personnel and resources, as well as ongoing support and monitorization of fidelity. There are few experiences and studies in the Spanish child welfare field to indicate which specific challenges need to be tackled in this process in Spain, although they are probably similar to those in other Western countries and fields (Fixsen et al., 2005). The limited number of studies carried out in Spain indicate that although practitioners might report a generally positive attitude toward evidence-based programs (De Paúl, Indias, et al., 2015), there may be impediments to their implementation, such as the belief that structured interventions and remaining true to the original program do not allow adaptation to meet individual needs and to respond to the cultural particularities of families (Pascual et al., 2020). Such beliefs are erroneous, at least in the case of Incredible Years, as showed by the solid evidence on its transportability to different countries and culturally diverse groups. In spite of being a manualized program, Incredible Years uses a collaborative and culturally sensitive model involving explicit tailoring to the needs of the individual families (Gardner & Leijten, 2017; Hutchings et al., 2008; Larsson et al., 2009; Pidano & Allen, 2015; Posthumus et al., 2012; Webster-Stratton et al., 2012). As Hutchings et al. (2011) stated:

[…] ensuring fidelity does not mean that the programme must be delivered in the same way every time. While there are essential core components of content and delivery, there is scope for leaders to make informed clinical adaptations of the IY programme to match the needs of a particular population or family, and the barriers to participation that they may encounter without affecting core components of the programme fidelity. Such proactive adaptations may be considered to complement, rather than compete with, efforts to maintain fidelity. (p. 137)

Of course, new homegrown interventions designed to be tailored to the cultural values and norms of Spanish families should be supported and developed. Although it is a time consuming and costly process, such innovation is necessary. Nevertheless, although it might appear to be the case that homegrown interventions will be more effective, it does not necessarily seem to be so. The systematic review and meta-analysis carried out by Leijten et al. (2016) of evidence-based parenting interventions based on behavioral/social learning theory found that the outcomes of homegrown interventions were similar to those of transported programs in terms of reducing disruptive child behavior. According to the empirical evidence, it was concluded that, when policymakers and service providers must choose between implementing imported evidence-based interventions versus developing or nurturing one locally, they should select interventions according to their evidence base rather than their cultural specificity. The present study provides preliminary evidence to endorse the choice of the Incredible Years program in the Spanish context.