Methodological Challenges in Group-based Randomised Controlled Trials for Intimate Partner Violence Perpetrators: A Meta-summary

Introduction

Thirty years after the United Nations Assembly declared the need to eliminate violence against women, the prevalence of violence against this population remains high, with more than a quarter of women aged 15 to 49 experiencing physical and/or sexual violence performed by their intimate partner (World Health Organization, 2021). Intimate partner violence (IPV) has a severe impact on the mental health of victim/survivors as well as an economic and social impact, which can exert an overload on health systems (Peterson et al., 2018; Sardinha et al., 2022; World Health Organization, 2013). In addition to its impact on the development of mental disorders such as post-traumatic stress disorder (PTSD) and depression, IPV is associated with chronic pain, headache, gynecological or gastrointestinal problems in victim/survivors (Baker et al., 2021; Campbell et al., 2018; Daugherty et al., 2021; Gibson et al., 2019).

The implementation of IPV interventions is fundamental to reducing the victim/survivor’s risk of suffering future violence. IPV perpetrator programmes were first developed over 40 years ago with the realisation that providing services to survivors of domestic violence and abuse without addressing the behaviour of perpetrators, was an inadequate response to this damaging violation of human rights (Hamel & Nicholls, 2006; Healey et al., 1999). Group treatment was favoured instead of individual therapy/counselling as it was believed that it offered abusers: more opportunities to expand their social networks to include others who are supportive of being non-abusive (Barner & Carney, 2011; Krishna et al., 2011; National Research Council et al., 1996); provided opportunities for peer learning (Yalom & Leszcz, 2020); and be potentially more cost effective by being able to cover a wide range of different treatment approaches (Karakurt et al., 2019). Group programmes do not work for all perpetrators and the individual approach is necessary with high harm individuals, who are difficult to engage and unlikely to attend groups (Tsantefski et al., 2021). Eckhardt et al. (2013) note that the social and historical analysis of IPV has been instrumental in formulating the philosophy, structure, and goals of perpetrator intervention. However, while such programmes in the United Kingdom at least share the goals of reducing levels of violence and promote victim/survivor safety, there is considerable variability in intervention methods and approaches (Murphy et al., 2020; Phillips et al., 2013). Many programmes operate an open group format, with one or two facilitators managing group discussion and activities. Participation may either be on a ‘voluntary’ basis or mandatory; the latter may be required by criminal courts as part of a pre-trial diversion programme, ordered by judges as part of a sentence, or may be imposed by probation agencies (Hamberger & Hastings, 1993). Even when IPV programmes are referred to as ‘voluntary’, there are likely to be pressures to participate from partners and ex-partners or social services.

Historically, most IPV male perpetrator programmes were grounded in a feminist analysis of abuse, informing psychoeducational models or the Duluth model (Barner & Carney, 2011; Phillips et al., 2013). Programmes operating on the Duluth model (Pence et al., 1993) are based on an understanding of the patriarchal nature of society and institutions that support and perpetuate male power, control and privilege within the domestic sphere (Dobash & Dobash, 1979), experienced as a sense of entitlement by male perpetrators of IPV. Such programmes assume that to reduce violence it is necessary to expose patriarchal or misogynistic behaviour and encourage perpetrators to take responsibility for adopting coercive and abusive tactics in relationships. In this way, more egalitarian attitudes and behaviours can be promoted (Eckhardt et al., 2013). Cognitive behavioural therapy (CBT) is another explicit model for perpetrator programmes. CBT approaches aim to change a person’s behaviour by developing a therapeutic relationship, exposure and dissolution of distorted cognitions, behaviour change strategies, working on core beliefs and trying to prevent relapse and further abuse (Nesset et al., 2019). Currently, the work with IPV perpetrators is quite complex, and most group programmes use a mixture of interventions that include feminist approaches and psychotherapeutic approaches such as CBT (Lilley-Walker et al., 2018). Most programmes focus on IPV involving partners or ex-partners rather than the broader focus on domestic violence and abuse involving wider family violence (Eckhardt et al., 2013; Westmarland & Kelly, 2013).

There has been a proliferation of empirical studies evaluating IPV group perpetrator programmes over the last four decades. Reviews have attempted to summarise the evidence for these programmes, which remains uncertain (Cluss & Bodea, 2011; Eckhardt et al., 2013; Nesset et al., 2019). Generally, these reviews report uncertainty about programme effectiveness due to heterogeneity of populations and interventions, limited or poorly measured outcomes, short follow-up or more biased research designs with longer follow-up, and insufficient attention to the context of the interventions (Akoensi et al. 2013; Arias et al., 2013; Gondolf, 2011).

Conflicting results reported by evaluations of IPV perpetrator programmes fuel fierce debate about appropriate intervention models and outcomes. Mixed findings from studies leave practitioners and service providers with conflicting guidance on how best to deliver programmes (Akoensi et al., 2013). Additionally, such findings are challenging to funders of perpetrator programmes in criminal justice, local authority or health care sectors. For example, the UK National Institute of Care Excellence guidelines do not recommend the commissioning of perpetrator programmes outside the context of further evaluation (National Research Council [NICE, 2013]).

In the light of uncertainty about IPV group perpetrator programme effectiveness, at least in part driven by problems in research design, treatment modalities and outcome measurement, in this review we aim to articulate the methodological challenges in the conduct of randomised controlled trials (RCTs) of IPV perpetrator programmes, to inform more robust evaluation. Previous meta-analysis has been performed to investigate the effectiveness of intervention programs for perpetrators (Karakurt et al., 2019). However, the authors included individual and group interventions without considering structural differences between such interventions. Furthermore, they only considered pre- and post-intervention results and did not differentiate between varied experimental conditions. Other recent systematic reviews have focused on different aspects of IPV interventions: a) effectiveness in different cultural contexts (Satyen et al., 2022); b) effectiveness for male perpetrators or male victims in health settings (Tarzia et al., 2020); c) comparison of experimental group versus group without any intervention (Cheng et al., 2021); d) risk-need-responsivity framework (Travers et al., 2021); e) motivational interviewing (Pinto E Silva et al., 2022 ; Santirso et al., 2020); and f) substance abuse (Stephens-Lewis et al., 2021).

As a research team developing and trialing our own group IPV programme for perpetrators considered to be in the mild to moderate risk category, we were particularly keen to gather and critically reflect on the key methodological challenges. While other reviews have tended to focus on the mixed or relatively weak evidence from RCTs or other designs (group format and individual programmes) with some consideration as to how methodological challenges might have made drawing conclusions so difficult, our review concentrates solely on the methodological challenges. Assessing these within RCTs is an opportunity to address these limitations and thereby improve the testing and the effectiveness of interventions. Focusing specifically on IPV programmes, our objectives were to identify and describe:

• randomised controlled trials measuring the effects of IPV group male perpetrator programmes.
• methodological challenges (contextual or procedural), as identified by the trialists in the included studies, that may explain the heterogeneity of the findings and why interpretation is difficult.
• articulate common challenges involved in conducting evaluations of IPV group perpetrator programmes.

Method

Eligibility Criteria

To be included, studies had to meet the following criteria: (1) individual or cluster-randomised controlled trials (RCTs); (2) the study population comprised male perpetrators of IPV, where participants attended the treatment programme voluntarily or were court-ordered to participate (primary studies where the focus was on women who abuse their partner or spouse were excluded and trials having a mixed population of men and women perpetrators); (3) group format IPV programmes of any duration delivered in any setting employing any psychotherapeutic approach (for example based on Duluth, CBT or strengths-based or a combination of these); (4) any outcomes.

Search Methods for Identification of RCTs

RCTs were identified through published systematic reviews. Literature searches were conducted to identify systematic and meta-analytic reviews on the effectiveness of group interventions with IPV perpetrators.

Eight online databases, including Web of Science, Google Scholar, Scopus, PsycINFO, Cochrane Library, ERIC, PubMed, and DARE were searched using Boolean operators and the following search and Mesh terms in the title and the abstract: “systematic review” OR “meta-analysis” AND “domestic violence” OR “batterer” OR “abuse” OR “intimate partner violence” OR “partner-violent men” OR “intervention program” OR “group” OR “offend*” AND “intervention” OR “treatment” OR “evaluation” OR “assessment” OR “effect*”.

These searches initially covered the period from January 2000 to March 2020; top-up searches were run again in February 2022 and March 2023. Reference lists of selected reviews were searched. The search was limited to systematic reviews and meta-analyses.

Selection of Studies

After mapping the studies (number and type of studies, i.e., experimental or quasi-experimental) included in the systematic and meta-analytic reviews, titles and abstracts of potentially relevant trials were screened by the first author and HC. We then retrieved the full text of all potentially relevant studies. Full-text articles were assessed for inclusion by WT, HC and GF independently against the eligibility criteria, with discrepancies resolved through discussion. Please see Figure 1 for a flow diagram showing the selection process for RCTs from the systematic reviews and meta-analytic evaluations.

Data Extraction and Management

The following data were extracted:

A) Study characteristics. Year of publication, country and setting where the study was conducted.

B) Participants. Number of participants assigned to intervention and control condition(s), referral status (e.g., voluntary and/or court-mandated), previous history of violent behaviour and treatment, mental health status, current substance abuse, additional problems/disorders.

C) Intervention. Content, components, duration/time, profession of person delivering the programme (or intervention), gender and number of therapist(s) or group leader(s), support to (ex)partner, risk management, the degree of mandatory delivery, attrition, adherence, type of comparison group (no intervention, other intervention).

D) Type of outcome measure(s). Recidivism or abuse outcomes (e.g., physical violence, aggression), attitudinal outcomes, interpersonal outcomes (e.g., managing anger) and psychological outcomes (e.g., self-esteem).

E) Source of outcome data. Routinely collected clinical or administrative data; self-reports, partner report, or other forms for gathering outcome data.

F) Length of follow-up time. Months and/or years.

G) Author narratives. For all included studies we also extracted: a) text providing the authors’ own assessment of the findings and b) text from the discussion and conclusion sections of each trial report (and, where available, its linked publications) that referred to methodological challenges in conducting IPV male group perpetrator programme evaluations. Methodological challenges were defined as: Any text within the discussion/conclusion section of the paper that provides authors’ observations/reflections/insights in relation to methodological issues implicated in the conduct of the trial. This includes any reference to the context for IPV group perpetrator programmes and any narrative that provides authors’ observations/ reflections/insights related to the process of conducting such programmes (i.e., what factors should be considered in the conduct of IPV evaluation programmes) and that attempted to offer any explanations for the positive (or negative) findings.

H) Risk of bias. We used the Cochrane Collaboration Risk of Bias tool (RoB2) to assess the methodological quality of all eligible randomised controlled trials (Higgins et al., 2011). The RoB2 is comprised of six domains and assesses each study’s risk of bias as ‘high risk’, ‘low risk’, or ‘some concerns’.

Data Analysis

All data collection and analysis were conducted by one reviewer (WT) and checked for accuracy by members of the author team (HC, MH, KM & GF), with disagreements resolved through discussion. Qualitative meta-summary was conducted using Microsoft Word and Excel, starting with identification of eligible text from the relevant discussion section of the included studies. We extracted text that referred to methodological challenges of evaluating IPV group perpetrator programme effects. We separated text referring to methodological challenges from all other text in the papers and edited the statements on methodological challenges to ensure that it was presented in a way that was accessible to readers while preserving their underlying content and meaning (Sandelowski & Barroso, 2003). Methodological challenges were then grouped, abstracted, and considered separately.

Pre-selected methodological characteristics were used to group similar findings together. For example: context of study (e.g., setting, community or prison); sample characteristics (e.g., participants’ referral route [voluntary and court-mandated]); mental health status and substance abuse history; intervention (modality, implementation, duration & evaluation); and outcomes (type, duration of outcome measurement).

We abstracted findings (e.g., authors’ narrative) to summarise the content of each group of methodological challenges. For each study characteristic, we reworked our lists of methodological challenges until we developed a new list of abstracted statements that captured the overall meaning of the original statements. This was done by eliminating redundancies, refining statements to ensure they were inclusive of the authors’ interpretations, preserving contradictions and ensuring clarity and accessibility.

Lastly, we provided a narrative summary of the abstracted methodological issues to identify whether any specific methodological challenges (pertaining to study/intervention characteristics) were implicated in positive (or negative) programme effects.

Frequency of methodological challenges were also used to extract more meaning from the narrative summaries by numerically describing the magnitude of the abstracted findings (Sandelowski & Barroso, 2003). Frequency of methodological challenges were calculated by dividing the number of studies contributing data on specific methodological challenges by the total number of studies.

Results

We identified 23 systematic reviews/ meta-analytic evaluations of trials on the effects of male perpetrator programmes (Akoensi et al., 2013; Aos et al., 2006; Arce et al., 2020; Arias et al., 2013; Babcock et al., 2004; Cheng et al., 2021; Feder et al., 2008; Feder & Wilson, 2005; Ferrer-Perez & Bosch-Fiol, 2018; Karakurt et al., 2019; Miller et al., 2013; Nesset et al., 2019; NICE, 2013; O’Connor et al., 2021; Pinto E Silva et al., 2022; Santirso et al., 2020; Satyen et al., 2022; Smedslund et al., 2011; Stephens-Lewis et al., 2021; Tarzia et al., 2020; Travers et al., 2021; Vigurs et al., 2016; Wilson et al., 2021). These 23 systematic reviews and meta-analyses were used to identify primary studies evaluating IPV perpetrator programmes, which is the main focus of this paper. After removing duplicates, we identified 15 randomised controlled trials from the 23 review papers. We additionally identified quasi-experimental and pre/post-test design studies which informed our thinking on methodological challenges in RCTs but are not included here.

Description of Randomised Controlled Trials

The 15 RCTs on the effects of male perpetrator programmes span the period 1992 – 2020 (see Table 1). Eleven were conducted in the United States (Alexander et al., 2010; Dunford, 2000; Easton et al., 2007; Feder & Dugan, 2002; Labriola et al., 2005; Murphy et al., 2020; Saunders, 1996; Taft et al., 2016; Taylor et al., 2001; Taylor & Maxwell, 2009; Tollefson & Phillips, 2015), one in Canada (Palmer et al., 1992), one in Spain (Lila et al., 2018) and the remaining two in Norway (Nesset et al., 2020; Palmstierna et al., 2012).

Setting

Eleven RCTs were conducted in community settings (e.g., family advocacy centres and/or agencies serving perpetrators and victims/survivors of domestic violence); two in outpatient hospital facilities (e.g., substance misuse treatment facility in the Easton et al. (2007) study and specialised mental health services in the Palmstierna et al (2012) study). The sheriff department jail was the setting for the Taylor & Maxwell (2009) study while the Nesset et al (2020) study was conducted under the auspices of a hospital-affiliated forensic research department.

Population

In total, the trials had a population of 4,216 men; in most cases men were court-mandated to treatment. Only a small minority of men were voluntary referrals to the IPV programme (Nesset et al., 2020; Palmstierna et al., 2012).

Intervention Model

The Duluth and the cognitive-behavioural model were the two dominant theoretical paradigms that informed the interventions tested in the studies. The study by Palmer et al. (1992) tested the effects of a brief psychoeducational programme with limited information provided about its content and the study by Tollefson and Phillips (2015) tested a programme using mind-body bridging techniques to prevent explosive states of mind.

Research Design

Five studies compared Duluth-informed interventions with no treatment (Feder & Dugan, 2002; Labriola et al., 2005; Taft et al., 2016; Taylor et al., 2001; Taylor & Maxwell, 2009) and three compared CBT with a control (Dunford, 2000; Palmstierna et al., 2012; Taft et al., 2016). The trial by Palmer et al. (1996) compared the effects of a psychoeducational programme to a control group. The study by Tollefson and Phillips (2015) compared a mind-body programme with a standard psychoeducational IPV perpetrator programme. Six were comparative effectiveness trials involving group-based CBT compared with either psychodynamic treatment (Saunders, 1996), twelve-step facilitation (TSF) (Easton et al., 2007), Mindfulness-based stress reduction (MBSR) (Nesset et al., 2020), individually delivered CBT (Murphy et al., 2020) or enhanced CBT treatment incorporating motivational interviewing (Alexander et al., 2010; Lila et al., 2018). None of the studies were cluster-randomised trials.

Duration

Group-based sessions lasted between 60 minutes and 3 hours and were in most cases delivered weekly. The exception is the Taylor & Maxwell (2009) study where sessions were held daily (prison setting). The duration of interventions ranged from 8 weeks (Taylor et al., 2001) to a year (Dunford, 2000).

Length of Follow-up

There were variations in the length of follow-up. The longest follow-up period of 54 months was noted for the Saunders (1996) study. Except for the Easton et al. (2007) study which incorporated weekly and monthly follow-ups, most studies included 6 and 12 months follow- up.

Outcome Measures

Table 1 lists the range of measures used to assess outcomes across studies. There is considerable heterogeneity in the type and number of measures researchers have employed to assess outcomes with a particular focus on recidivism data (based on preparators’ and/or victim/survivors’ self-reports and official/police data), psychological profile data (based on men’s, victim/survivors’ self-reports and therapists’ assessments) and in some instances other proximal outcomes (e.g., treatment compliance (Lila et al., 2018).

Overview of Findings

We did not perform a meta-analysis due to heterogeneity between studies in terms of setting, outcome and follow-up. To facilitate further appreciation of the results of the meta-summary approach relating to methodological challenges, we considered the authors’ own assessments of their primary statistically ‘significant’ findings which are summarised below.

We identified five studies (Lila et al., 2018; Palmer et al., 1992; Palmstierna et al., 2012; Taft et al., 2016; Taylor et al., 2001) reporting a significant reduction in official (police recorded) and/or men’s self-reported rates of physical violence towards partners. Testing a brief psychoeducational programme, Palmer et al. (1992) reported significantly higher recidivism rates for controls than men in treatment (31% vs. 10% respectively). Taylor et al. (2001) tested the effects of a Duluth-informed programme and noted a 50% reduction in new violence at 6-months M = 0.38 for controls and M = 0.10 for treatment) and at 12 months (M = 0.55 for controls and M = 0.28 for treatment) based on official records. Palmstierna et al. (2012), testing a 15-week CBT-informed programme, reported that most of the men had reduced their self-reported violent behaviour. Taft et al. (2016), testing a 12-week trauma-informed intervention, noted greater reductions in self-reported physical and psychological IPV use particularly related to controlling behaviour involving isolation and monitoring. Finally, Lila et al. (2018) showed that adding motivational strategies to a standard intervention (i.e., cognitive-behavioural) programme can produce greater reductions in self-reported violence and recidivism risk of IPV relative to the standard programme alone.

Among the ten studies reporting negative results, only four compared the control and intervention groups (Dunford, 2000; Feder & Dugan, 2002; Labriola et al., 2005; Taylor & Maxwell, 2009), whilst the other six studies evaluated the effectiveness of two or more distinct types of intervention to reduce IPV (Alexander et al., 2010; Easton et al., 2007; Murphy et al., 2020; Nesset et al., 2020; Saunders, 1996; Tollefson & Phillips, 2015). Six studies used some type of CBT approach and one of them compared CBT versus psychodynamic therapy (Saunders, 1996). In addition, session lengths in hours were shorter among RCTs that reported negative results (M = 29.72) compared to those that were successful in preventing IPV (M = 35.8).

Methodological Challenges

The methodological challenges articulated in discussion and/or conclusion sections of the 15 RCTs fell into three broad categories: contextual, research design specific, and procedural. The last category was further subdivided into three topic areas: sample, intervention (modality, duration, implementation) and measurement (type and duration of outcome measurement). These challenges are summarised below.

Frequency of Methodological Challenges

All 15 RCTs identified at least one challenge involved when conducting IPV group perpetrator programmes (Table 2). The main challenges reported, as measured using frequency of methodological challenges, were related to source of outcome measure used (12/15), treatment modality (11/15), sample (9/15), duration of follow-up (7/15), and challenges due to study or intervention attrition (8/15). Challenges relating to the implementation of the research design (3/15) and contextual challenges were the least discussed (3/15).

Furthermore, among the 15 RCTs, only eight reported how they tackled study attrition (numbers of participants giving feedback and filling in questionnaires) or intervention attrition (numbers of participants continuing to attend the intervention or treatment, Table 3) (Alexander et al., 2010; Feder & Dugan, 2002; Lila et al., 2018; Nesset et al., 2019; Taft et al., 2016; Taylor et al., 2001; Taylor & Maxwell, 2009; Tollefson & Phillips, 2015). Among them, seven used different statistical techniques to control for loss to follow-up (study attrition) and two studies used statistical methods and contacted participants via phone calls or reminders to avoid loss to follow-up (Nesset et al., 2020) or to avoid loss to intervention (Taft et al., 2016). Study attrition and intervention attrition can be related such as when a participant may be unwilling or unable to continue participation in the trial because they become frustrated with the intervention in some way and/or other aspects of their life makes involvement difficult. For example, a male participant’s partner might leave the relationship and he may blame this on the intervention, stop attending the groups and refuse to complete any further follow-up questionnaires.

In relation to risk of bias, ten studies were classified as at high risk of bias, two at low risk, and three studies as having some concerns (Figure 2). There was a slight tendency for the studies with the lower risk of bias rating to identify and report more methodological challenges (Table 2). The mean number of challenges reported for lower risk of bias studies (Alexander et al., 2010; Easton et al., 2007; Lila et al., 2018; Palmer et al., 1992; Taylor et al., 2001) was 5 compared to a mean of 3.4 for higher risk studies (Dunford, 2000; Feder & Dugan, 2002; Labriola et al., 2005; Murphy et al., 2020; Nesset et al., 2020; Palmstierna et al., 2012; Saunders, 1996; Taft et al., 2016; Taylor & Maxwell, 2009; Tollefson & Phillips, 2015).

Contextual

There is acknowledgment that these intervention programmes operate within specific judicial systems which have an ethical, as well as legal, responsibility to facilitate their smooth implementation. Three studies highlight context-related challenges. Palmer et al. (1992) refers to the legal system’s responsibility to address attrition and ensure treatment attendance by pursuing group participants who fail to attend with whatever sanctions are available in achieving this aim (Ford & Regoli, 1993). Labriola et al. (2005) note that research sites should ideally be engaged in rigorous monitoring practices (rather than simply checking-in) facilitated by clear and frequent communication of court policies and expectations from perpetrators. Tollefson and Phillips (2015) recognise that it might be hard to generalise the findings to other areas as usual care for convicted domestic abuse offenders in their area (Utah) was a minimum of a 16-week IPV perpetrator programme.

Research Design Specific

Dunford (2000) suggests that the risks of conducting RCTs to assess interventions are likely to be fewer than the consequences of doing so. This observation underscores the need to use rigorous designs to evaluate interventions for domestic violence. Aspects of randomised trial design were difficult to implement in two studies. Taylor et al. (2001) needed to make substantial concessions to court officials to gain their cooperation, allowing judges to override assignment to the control condition. Feder and Dugan (2002) highlighted random assignment as essential for reducing bias but acknowledged this could be problematic; some of the agencies involved in this study were resistant to the experimental design and would not help in contacting the victim/survivors thereby directly contributing to low victim/survivor response rates.

Procedural

Individuals’ Needs. Among included studies, six of them refer to ‘sample’ as a contributing factor in explaining treatment effects (Alexander et al., 2010; Easton et al., 2007; Feder & Forde, 2000; Palmer et al., 1992; Saunders, 1996; Taylor et al., 2001). There was explicit reference to the need to identify and tailor the content of interventions to match types of participants to improve outcomes (Dunford, 2000; Labriola et al., 2005; Murphy et al., 2020). Two studies explicitly question the assumption of the ‘one size fits all’ approach (Dunford, 2000; Saunders, 1996). Study authors have articulated the heterogeneity of trial participants especially with reference to: a) their mental health status (Palmer et al., 1992); b) their childhood traumas (Saunders, 1996); c) motivation to change their behaviour even if they have been mandated to treatment (Alexander et al., 2010; Taylor et al., 2001); d) their stakes in conforming with court decisions (Feder & Forde, 2000); and e) their substance abuse histories (Easton et al., 2007).

Intervention

Intervention or programme challenges featured prominently in the discussion section of the included studies. Challenges were organised by whether they referred to: a) treatment modality; b) duration and length of treatment; and c) treatment implementation. Each of these are examined below.

Treatment Modality

Taylor et al. (2001) were the first to explicitly question the dominance of Duluth-informed treatment and proposed that treatments informed by a client-centred approach that seek to engage clients as much as possible rather than confront participants (as some Duluth programmes may do) may be most useful. A similar observation was shared by Feder and Dugan (2002) advocating the need for new and innovative IPV perpetrator programmes to reduce IPV to help victim/survivors. Similarly, Labriola et al. (2005) encouraged the development of intervention programmes with different conceptual bases and curricula considering the typology of participants and adopting a more direct behavior modification approach. Tollefson and Phillips 2015 had concerns about cross contamination of the interventions as the same facilitator delivered both the mind-body bridging programme and the usual care programme, and a small dose of mind-body techniques were delivered in the usual care programme. The trial by Easton et al. (2007) that targeted substance (alcohol and illicit drugs) abuse and co-occurring IPV and the studies by Alexander et al. (2010) and Lila et al. (2018), which incorporated stages-of-change motivational interviewing, provide good attempts to explicitly target participants’ characteristics. However, the combination of several intervention strategies makes it difficult to disentangle the extent to which the interventions’ effects are attributable to each of these strategies (Lila et al., 2018; Murphy et al., 2020; Nesset et al., 2020).

Treatment Implementation

We found little reference to treatment implementation problems in the included trials. Easton et al. (2007) note that the findings may be confounded by therapist effects (only one therapist delivered either the SADV or TSF conditions) and Alexander et al. (2010) highlighted the low number of audiotaped sessions which precluded analysis of the differential impact of therapist adherence on treatment modality.

Measurement

We found extensive reference to various measurement problems in the included trials.

Source of Outcome Measurement

Palmer et al. (1992), Saunders (1996) and Taylor and Maxwell (2009) questioned the reliance on police reports as a measure of recidivism and highlighted the importance of cooperation of partners or ex-partners in assessing treatment outcomes, an observation echoed by other researchers (Alexander et al., 2010; Easton et al., 2007; Feder & Forde, 2000; Lila et al., 2018; Nesset et al., 2020; Taft et al., 2016; Taylor et al., 2001). While victim/survivor reports are more credible in assessing continuation of abuse (Lila et al., 2018; Nesset et al., 2020; Palmstierna et al., 2012), low victim/survivor response rates (below 25%) were common and particularly problematic due to potential bias from getting responses from a small sample of victim/survivors (Alexander et al., 2010; Feder & Dugan, 2002; Labriola et al., 2005; Taylor et al., 2001). Easton et al. (2007) highlighted the importance of independently assessing outcomes with reliability scores being provided. That said, Easton et al. (2007) only managed to collect collateral data from female partners of 55% of the male participants; and Lila et al. (2018) did not collect any as Spanish legislation prevents the services referring IPV perpetrators to intervention programmes from providing information that would allow access to their victims.

Duration of Follow-up

The short duration of follow-up was noted in six trials (Alexander et al., 2010; Easton et al., 2007; Lila et al., 2018; Palmstierna et al., 2012; Taft et al., 2016; Taylor & Maxwell, 2009). Davis et al. (2000) suggested that both short- and long-term outcomes need to be included; short-term to assess transient programme effects and long-term outcomes to determine whether the intervention leads to permanent changes.

Attrition

There was extensive reference to the effects of treatment attrition on data quality (Feder & Dugan, 2002; Nesset et al., 2020; Palmer et al., 1992) and the initial randomisation (Murphy et al., 2020; Saunders, 1996). Taylor et al. (2001) highlighted the need for researchers to find ways of minimising study attrition while maximising interview response rates when interviewing victim/survivors about continuing abusive behavior from their partners.

We found little explicit reference to the effects of treatment duration and length of treatment in the included trials. Taylor and Maxwell (2009) refer to the short duration of the prison-based Duluth-informed intervention. In a re-analysis of the Brooklyn experiment, Maxwell et al. (2020) concluded that ‘there is little support that longer programs are more effective than shorter ones’ (p. 493). This finding runs contrary to the earlier summary observation that session length was shorter among the randomised controlled trials that reported negative results compared to those that were successful in preventing IPV. This assertion also goes against the more recent conclusions of Arce et al. (2020) who in their meta-analysis found that longer programmes (those lasting over 16 weeks) produced more effective outcomes.

Discussion

Through extensive searching and mapping of studies reported in systematic reviews, we identified 15 RCTs examining the effects of group-based IPV perpetrator programmes. Seven studies were comparative effectiveness trials. With two exceptions, Duluth or cognitive-behavioural models informed the group-based interventions which included mostly court-referred participants. A range of measures were employed, including psychological and treatment-related, with recidivism as the primary outcome substantiated by official/police records, self-reports from perpetrators and/or their victim/survivors followed over a period of 15 weeks to 12 months in the majority of cases.

Five studies reported statistically significant findings relating to violence reduction either substantiated by police records or self-reported. The findings relate to the effects of a psychoeducational programme for men convicted of domestic assault (Palmer et al., 1992), a Duluth-informed programme for men charged with domestic abuse (Taylor et al., 2001) and two CBT-informed programmes for voluntary/clinician -referred men (Palmstierna et al., 2012; Taft et al., 2016).

The use of the qualitative meta-summary approach facilitated the emergence of important insights on several methodological (i.e., contextual, research design and procedural) challenges which may contribute to the heterogeneity of findings from trials and the persistent uncertainty about effects of perpetrator programmes. Evidenced by the frequency of methodological challenges, major challenges include the source of outcome measurement (whether self-report and/or victim/survivor-collaborated), treatment modality, sample (i.e., participants’ characteristics), and duration of follow-up. Challenges identified with less frequency relate to the context of the trial and implementation of the research design, although identification of these problems by trialists in fewer studies does not mean they are irrelevant.

There was agreement on the importance of corroborating men’s self-reports of violence with both official (police, e.g. report and/or arrest data) and victim/survivor data (Nesset et al., 2020; Taylor et al., 2001). Police arrest data (where behaviour is deemed a criminal offence) as a measure on its own may exclude information on repeat perpetration/recidivism where behaviour, such as non-physical IPV, is not considered a criminal offence. Police reports may therefore provide a more useful measure of recidivism but are dependent on quality and depth of detail in police recording. Comparing recidivism data across trials from different countries can also create problems as arrest and criminal offences related to IPV behaviour may be defined and framed differently across countries. Additional sources for substantiating treatment effects can be particularly problematic due to generally low victim/survivor response rate (Alexander et al., 2010; Feder & Dugan, 2002; Labriola et al., 2005; Taylor & Maxwell, 2009). These observations echo findings from quasi-experimental research on the effects of IPV perpetrator programmes (Babcock & Steiner, 1999; Bloomfield & Dixon, n.d.; Coulter & VandeWeerd, 2009).

Generally, we noted investigators questioning the ‘one size fits all’ intervention approach (Dunford, 2000; Saunders, 1996). Treatment effects appear to be greater when interventions consider perpetrators’ motivations for treatment attendance and adherence (Labriola et al., 2005; Lila et al., 2018; Taylor et al., 2001). Perpetrators are not a homogenous group and their typology (e.g., age, employment status, offence type, educational level, mental health status and/or substance abuse histories) is an important methodological consideration that can mitigate treatment effects (Labriola et al., 2005; Saunders, 1996) and restrict the generalisability of findings (Taft et al., 2016).

Our analysis indicates that participants, especially in court-mandated programme evaluations, need to be sufficiently integrated into a coordinated judicial and/or community response. The observation is not unique to the trialists considered in this review (Chen et al., 1989; Harrell, 1991). While some investigators question whether systemic factors are as important as participants’ characteristics in influencing outcomes (Tollefson & Gross, 2006), the importance of agency collaboration, information sharing and the creation of strong links of the agencies with the local communities in which they operate cannot be underestimated (Kelly & Westmarland, 2015).

Implications for Research

RCTs of IPV perpetrator programmes are an important vehicle to increase the uptake and application of knowledge by clinical and policy decision-makers (Feder & Boruch, 2000). Our analysis highlighted consensus among trialists about the need to evaluate treatment effects through rigorous research designs, namely RCTs. While most quasi-experimental studies in the area acknowledge lack of randomisation as a threat to internal validity (Hendricks et al., 2006; Morrel et al., 2003; Tollefson & Gross, 2006), the conduct of RCTs is not without its difficulties.

Recent systematic reviews, partly because they employ different inclusion criteria, continue to provide conflicting findings relating to the effects of different interventions models (Arce et al., 2020; Karakurt et al., 2019). In particular, those examining the effects of the dominant cognitive-behavioral model (Nesset et al., 2019) note insufficient evidence to confirm its positive effects on men’s IPV reduction. However, promising effects from some of the included RCTs are trauma-informed interventions (Karakurt et al., 2019; Taft et al., 2016), more integrated interventions such as psychological therapies in conjunction with substance abuse treatment (Tarzia et al., 2020) and CBT (Palmstierna et al., 2012). Nevertheless, the generalisability of findings remains limited, pointing to the need for more evidence to evaluate the effectiveness of group perpetrator interventions. Future studies should aim to test well-articulated and innovative treatment approaches (Romero-Martínez et al., 2022) specifically tailored to address the complex social, psychological and substance misuse needs of IPV perpetrators. When possible, investigators should consider how to circumvent difficulties arising from random allocation relating to a) ethical concerns (e.g., victim/survivor safety); b) differential retention of participants (due to group allocation) and c) keeping to a minimum judicial interference (in cases of perpetrators mandated to treatment) which can negatively influence the conduct of the study. With regard to ethical concerns it has, for example, been noted that attending an IPV programme may give some victim/survivors false hope about the future relationship, increasing their risk of harm from abuse (Feder & Dugan, 2002).

Furthermore, intervention attrition is an important barrier found in these programmes which potentially impacts the findings. A meta-analysis showed that dropout rates were similar regardless of the type of programme target audience, i.e. sex offenders or perpetrators of domestic abuse (Olver et al., 2011). The factors most associated with intervention attrition were age, criminal history, personality, psychological concerns, and motivation for treatment. Those who did not complete treatment were higher-risk offenders, and treatment abandonment predicted recidivism (Olver et al., 2011). The authors suggest that providers should build a strong therapeutic bond with clients and develop strategies to avoid intervention attrition. In this way, clients can be more motivated to participate in the programmes (McMurran, 2003; Olver et al., 2011). However, there may be particular problems mixing low/moderate risk of harm and high harm perpetrators and Olver and colleagues’ recommendation is probably too optimistic.

Other ideas for tackling intervention attrition are to separate out early non engagement and no shows at the first sessions from participants who engage initially in IPV perpetrator programmes but then drop out later. Richards et al. (2021) found that early no shows and non-engagers were more likely to have mental health problems whereas dropouts were more like to have substance use problems. Donovan and Griffiths’ (2015) study focused on the pre-IPV programme phase and suggested that motivational interviewing should be used more in the early stages with a bigger role for social work practitioners and health practitioners supporting better initial engagement. Early screening of participants to identify factors that might make them more likely to not engage early on or drop out later may be helpful. In a meta-analysis of intervention attrition variables Jewell and Wormith (2010) recommend the early screening approach to facilitate the provision of better support having found younger, unemployed perpetrators who had not been court mandated to attend to be at greater risk of drop out. Another advocate for the use of screening, Priester et al. (2019) found that adverse childhood experiences, especially experiences of household dysfunction such as a loss of a parent, having a household member in prison or having substance use or mental health problems, predicted IPV perpetrator programme intervention attrition. Addressing the problems of later programme attrition, Richards et al. (2021) suggested that monitoring employment throughout an IPV perpetrator programme would be useful as this may be a red flag and precursor to dropping out. Other suggestions for tackling attrition from Richards and colleagues are to invest in intensive case management and where probation staff are involved, for their role to be focused on longer term supporting goals such as programme completion and engagement, rather than just early signs of engagement.

Future trials should therefore consider pre-treatment screening (Dunford, 2000, Jewell & Wormith, 2010; Priester et al., 2019); careful monitoring of perpetrator behaviour during treatment and signposting men with previous abusive histories and severe psychological disorders to more intensive interventions (Alexander et al., 2010; Richards et al., 2021); and to identify and a priori match participants (in terms of motivation/needs) to interventions to improve attendance, engagement and outcomes (Alexander et al., 2010; Dunford, 2000; Jewell & Wormith, 2010; Saunders, 1996). As these are complex interventions with multiple factors affecting, for example, intervention engagement, attrition and outcomes, embedding routine and more comprehensive data collection and evaluation in all IPV perpetrator programmes would help support the ongoing improvement of understanding and practice. The standards against which the treatment is being evaluated would need to be explicitly defined (Davis et al., 2000) and both short (up to 6 months) and long term (preferably up to or longer than two year) follow-ups need to be included. Outcomes need to be assessed, preferably by independent assessors and reliability scores provided when independent raters are used (Easton et al., 2007). Careful consideration and planning are required to overcome low victim/survivor response rate during these evaluations. All outcome measures are open to different forms of bias. Perpetrator accounts might be unreliable and too invested in their own change narrative (Dobash & Dobash, 1998; Henning & Holdford, 2006); partners might be reluctant to report abuse because, amongst other reasons, of fear of retaliation, shame, or concern about professional intervention (Boethius & Åkerström, 2020; Overstreet & Quinn, 2013); and police are more likely to record and make arrests in relation to incidents of physical abuse than coercively controlling behaviour (McPhee et al., 2022). We do not think that a combined outcome reduces the bias in any of these measures or that alternative outcome measures are preferable, such as risk scores (Turner et al., 2019). We recommend reporting all three outcomes as they are of potential value to different audiences such as people experiencing abuse, service commissioners and practitioners (Westmarland & Kelly, 2013).

The current review aimed to identify and collate all published RCTs on group IPV male perpetrator programmes. As primary studies were identified through systematic reviews, due to their various inclusion/exclusion criteria, it is possible we may have missed some primary studies. We might not have exhaustively covered all possible database search terms and we did not, for example, use the term “perpetrator” although we did use the terms “batterer” and “partner-violent men”. To maximize retrieval of systematic reviews and meta-analyses, our search utilised multiple complementary methods in addition to database searching. We searched broadly using eight electronic databases. Previous systematic reviews and/or meta-analyses used between two and six databases except for one article that used 13 databases to assess cultural differences in IPV interventions (Satyen et al., 2022). Although our review used multiple databases it does not fulfil the search criteria for a systematic review (Uman, 2011). This does not weaken the validity of our methodological meta-summary based on the systematic reviews (and their primary trials) that we identified in our search. We then used a rigorous process for identifying, extracting, and analysing statements referring to methodological challenges from the discussion section of these documents. Importantly, we were interested in any reference to methodological challenges from trialists assessing the effectiveness of IPV male perpetrator interventions. It is important to note, however, that the methodological challenges summarised here were extracted from the Discussion section of the reports only, it is possible that additional reference to methodological challenges may have been identified should we have included the whole report (Gondolf, 2015). Our review contributes by pointing out the limitations and methodological challenges of previous randomised controlled trials, which we hope will be helpful to the development of more rigorous study designs and further development of feasibility and effectiveness studies of IPV interventions.

Conclusion

This review provides a systematic summary of existing methodological challenges, as identified by trialists, in the conduct of randomised controlled trials on IPV male perpetrator programmes. The present review will serve as a useful resource for authors wishing to conduct IPV group perpetrator programmes, as well as researchers wishing to conduct empirical research on IPV group programme effectiveness. It is also a necessary first step to developing a cohesive methods guidance document that addresses relevant issues and areas of uncertainty when planning studies on the effects of IPV group perpetrator interventions. Accordingly, the results of this study were used to inform the design of a randomised controlled trial of a community-based intervention for male perpetrators of mild to moderate risk in the Southwest of England and South Wales (http://www.bristol.ac.uk/primaryhealthcare/researchthemes/reprovide/). The results of this REPROVIDE study are due at the end of 2024.