Predictive efficacy of violence risk assessment instruments in Latin-America

Two conditions decreased the risk of ethics faults: a) the observational design maintains the confidentiality of the individuals of the sample; b) the assessments were part of the activities that had been ordered by the Court, with clear knowledge of all parties involved in each case, such as defense attorney, curators, and prosecutors. The data obtained was coded and entered in a database from the Prosecutor General Bureau of the Supreme Court of Justice of the Province of Buenos Aires. The project was approved by the Bioethical Committee of the Institute of Bioethics and Humanities of the Mainetti Foundation, which is independent from the Judicial Power and the National University of La Plata.

During the follow-up period, 99 (64.7%) of the subjects had at least one general relapse and 91 (59.5%) subjects had at least one violent relapse. The incidence rate of violent recidivism was 16.8 per 100 person-years.

During the period in which information on recidivism was collected via interviews with individuals and their families, as well as by checking official records, recidivism events were reported by subjects in 14 (14.1%) of the 99 recidivating cases; in 50% of those 14 cases, official information of a second offense was also obtained.

Table 1 shows the mean scores obtained at pre-release assessment.

The prevailing diagnoses were related to substance abuse and antisocial personality disorder ( Table 2).

The association of the VRA baseline scores with dichotomous and interval outcome measures was estimated. In general, the point bi-serial correlations for the total values were higher for violent recidivism than general recidivism: HCR 20 (.207, p = .005 vs. .152, p = .03); VRAG (.269, p < .001 vs. .240, p = .001); PCL-R (.206, p = .005 vs. .179, p = .01); SPJ (.248, p = .248 vs. .236, p = .002). For each outcome measure the association with VRAG scores was the strongest, followed by SPJ. With respect to the HCR-20 subscales, the subscale H had the strongest point bi-serial correlation (.189, p = .02). In the case of PCL-R, the antisocial facet (Facet 4) was outstanding ( r = .254, p = .001). The results highlight the relevance of static factors associated with the transgressor lifestyle, which are measured by these scales. Similar results were obtained with the estimate of the AUCs ( Tables 3 and 4 ). The VRAG and the antisocial facet of the PCL-R were best at discriminating violent recidivating individuals.

For each VRAG category, a regular growing trend of the proportions of recidivating individuals was identified. While no subject was classified in the lowest category of risk, the percentage of recidivating individuals was 0%, 33%, 50%, 46%, 51%, 73%, 70%, and 100% for the categories 2, 3, 4, 5, 6, 7, 8, and 9 respectively.

Predictive validity was also determined by dichotomizing the total pre-release assessment scores. In the case of the PCL-R a cutoff of 30 was used (as recommended by the Technical Manual for the categorical diagnosis of psychopathy) and for the HCR-20 and the VRAG the means were used ( Table 5 ). Of the three instruments, the VRAG had the best predictive power.

Logistic regression models were estimated with “violent recidivism” as the dependent variable. When controlling by age ( OR = 0.94; 95% CI .89, 99; p = .012), VRAG higher than the mean (VRAG > 12) doubled the risk of violent recidivism ( OR = 2.14; 95% CI 1.07, 4.35; p = .04), as did PCL-R higher than the mean (PCL-R > 20; OR = 2.13; 95% CI 1.07, 4,24; p = .03) and SPJ measured as an interval variable (minimum 1; maximum 3; OR = 2.13; 95% CI 1.08, 4.21; p = 0.3). However, HCR 20 higher than the mean and PCL-R > 29 showed no significant association when controlling for age. The associations had a marginal statistical significance and no associations were found when total values were entered as interval variables: VRAG ( OR = 1.04; 95% CI .1, 1.09; p = .03); PCL-R (OR = 1.06; 95% CI 1.01, 1.11; p = .01); HCR 20 (OR 1.06; 95% CI 1.01, 1.11; p = .03).

Estimations of Kaplan-Maier function for the VRAG stratified by the mean value and for the SPJ factor stratified in low, moderate, and high scores showed that the greater the risk factor, the higher the accumulated probability of general recidivism during the follow-up time, log rank (Mantel-Cox) = 7.6 (1); p = .006 and log rank (Mantel-Cox) = 7.9 (2); p = .02, respectively. On the other hand, the Kaplan-Meier Function for the HCR-20 stratified by the mean and the PCL-R total stratified by the mean or by the cutoff score of 30 showed no significant differences.

The trend was even more noticeable when calculating the Kaplan-Meier with violent recidivism as the outcome: VRAG, log rank (Mantel-Cox) = 9.5 (1); p = .002; and of those having greater risk according to the SPJ, log rank (Mantel-Cox) = 8.44 (2); p = .015 (Figures 1 and 2 ). Likewise, Kaplan-Meier function with the HCR-20 stratified by the mean and the PCL-R total, either stratified by the mean or by the cutoff score of 30 showed that those that had higher scores in the instruments also had the higher accumulated probability of recidivism with violent acts, but the differences were not significant: HCR-20, log rank = 2.99 (1); p = .08; PCL-R stratified by the means, log rank = 3.31 (1); p = .07; PCL-R stratified by cutoff score of 30, log rank = 0.77 (1); p = .38.

Figure 1

VRAG Kaplan-Maier Cumulative Survival Function for VRAG and Violent Recidivism.

Figure 2

Kaplan-Maier Cumulative Survival Function for Structured Professional Judgment and Violent Recidivism.

Taking into account the significant result of the function of Kaplan-Meier, Cox regression was estimated with age at release and the dichotomized VRAG factor as co-variables. Controlling for age, RR = .951 (95% IC 0.917, 0.985); p = .006, the VRAG value higher than the mean increased the rate of violent recidivism 57%, RR = 1.573 (95% IC 1.023, 2.419), p = .039.

This study, as part of a program created at the beginning of the millennium in Argentina, has helped to disseminate a systematic style of assessment of risk of violence over traditional non-systematic practices in Latin America. The systematic methodology resulted in a technically improved model with respect to the previous model based on “beliefs and thoughts” that had various vulnerabilities ( Monahan, 1981a, 1981b; Quinsey et al., 1999; Rice, Harris, & Quinsey, 2002 ) and that, due to the non-systematic way of working, created barriers to explore its efficacy. It should be noted that, with independence of results, this study made it possible to test the reliability and predictive validity of a battery of instruments to assess the risk of violent recidivism, for the first time in Latin America. Thus, the program achieved its objective and inaugurated a period of diffusion so that other experts would add evidence to the complex area of violence risk assessment, which sits in the interface of law and mental health.

This study analyzed the predictive validity of various VRA assessment tools in a cohort of released convicted males in Argentina. It provides valuable information about which VRA tool was most reliable, and the results can be used for comparisons with other populations. The long recruitment period of the consecutive candidates for early release enabled us to obtain a representative profile of the prison population at the moment of “leaving” the prison system.

Evidences from several studies support integrated evaluation and intervention according to the model of Risk-Need-Responsivity ( Andrews & Bonta, 2010; Andrews, Bonta, & Wormith, 2006 ). In Argentina, common day practice is far away from the systematic use of the model, but this study shows its plausibility. As found in other countries ( Rodríguez et al., 2011 ), substance abuse disorders was an outstanding health problem. The program established a thorough diagnostic system for substance abuse disorders, which lead to referral to treatment in the community according to the risk and needs principles. On the other hand, a biochemical monitoring system was implemented. Drug metabolite testing in different substrates (urine, hear, nails) reinforced the monitoring and re-intervention in those cases with a decreased predisposition to withdrawal ( Folino, Arado, Ferrari, & Marengo, 2002 ).

General and violent recidivism rates (65% and 59% respectively) reached in a 3.5-year follow up were much higher than was estimated with a 1991 cohort study from a single province in Argentina with a longer follow-up time ( Folino & Marchiano, 2002 ). The higher rates are worrisome for public security and from a preventive point of view. In addition to their own original vulnerabilities, people released from prison also face the stressors of social re-incorporation ( Folino et al., 2005 ). Thus, there is a clear need for the state to create programs for secondary prevention for this population. The recidivism rates in the present study might be useful as baseline data to compare the impact of future programs aimed at diminishing recidivism rates.

This study found that VRA instruments improve prediction of recidivism rates, but with low efficiency. The results show a lesser efficiency than those from studies in other countries included in recent meta-analyses ( Singh et al., 2011; Yang et al., 2010 ). The four instruments predicted violent recidivism better than general recidivism, contrary to the recent findings with other instruments and with bigger populations ( Zhang, Roberts, & Farabee, 2011 ). The percentage of subjects with a recidivism event increased according to the risk classifications in the nine categories of the VRAG, and the VRAG was the only instrument that reached an AUC of .65 for violent recidivism. But the index is still below .70, which is considered clinically significant.

Predictive efficiency was greater when the probability of accumulated recidivism in the follow-up time was considered. Kaplan-Meier function showed that individuals who had been exposed to greater risk according to VRAG classification and the SPJ had a significantly greater probability of both general and violent recidivism. The results suggest that preventive services should be provided mostly during the first period after release. They also add clinical relevance to the use of instruments.

Age has traditionally been significantly and inversely associated with violent recidivism ( Hill, Habermann, Klusmann, Berner, & Briken, 2008; Looman & Abracen, 2010; Lund, Hofvander, Forsman, Anckarsäter, & Nilsson, 2013; Quinsey et al., 1999; Webster et al., 1997 ). In multivariate analyses controlled for age, the VRAG maintained its predictive value.

The results of the PCL-R were not in line with early studies that reported a high violence predictive power for the diagnosis of psychopathy ( Harris, Rice, & Cormier, 1991; Salekin, Rogers, & Sewell, 1996 ). The slightly better performance found with facets 3 and 4 suggests that preventive measures aimed at addressing criminogenic factors differently than interpersonal or affective personality factors could be more beneficial ( Andrews & Bonta, 2003; Bonta, Law, & Hanson, 1998; Yang et al., 2010 ). There is also the possibility that the effect of personality factors on the risk for violent recidivism is surpassed by socioeconomic and political instabilities, such as those experienced in Argentina from the turn of the century. A recent study in England and Wales has also shown predictive difficulties in psychopathic subjects ( Coid, Ullrich, & Kallis, 2013 ). The authors suggest that one possible reason is that the items of the instruments show little variation in psychopathic offenders and, therefore, are not adequate to differentiate those at high and low risk for violent reoffending. Further research is needed to find better indicators of social performance during the reincorporation period and to identify the most sensitive personality factors.

One limitation of the study is the decrease in information sources in the follow-up period since the last quarter of 2004, when the interviews with released subjects and their families were stopped. The strengths of the study include that the social workers were blinded to the pre-release assessment, the representativeness of the sample, and the wide range of sources of official records for new charges.

The present study's findings complement the results of previous studies in Latin America. While previous Latin American reports supported the predictive validity of the instruments in forensic population ( De Borba Telles et al., 2012; Folino, 2006 ), this study suggests caution about risk assessment when the targeted population is made up by convicts. Future studies should identify local risk factors and look for any interactions with regional socioeconomic conditions.

The performance of these instruments is far from optimal and there is a need to focus on preventive actions in cases that are classified as high risk, as well as in those that are low risk ( Large, Ryan, Callaghan, Paton, & Singh, 2014 ). A future study is planned to explore the distributions of individual risk factors in the different categories formed by the omnibus measure, with the hypothesis that there are different factors that influence different subgroups. This would allow for a refinement in the assessment process and in the planning of interventions.

Unfortunately, the knowledge and diffusion of structured and actuarial systems of assessment is very new in Latin America, as is the acknowledgement of the importance of research on the subject. The lack of dissemination of this kind of systematic assessment prevents studies with larger samples and makes it difficult to adapt the VRA instruments as it occurs elsewhere in the world ( Ragusa-Salerno, Ostermann, & Thomas, 2013; Rettenberger, Haubner-Maclean, & Eher, 2013; Zhang et al., 2011 ;). Further, the penitentiary institutions in Latin America remain disconnected to the precarious “out” services and institutional actions are carried out inconsistently as short-term policies or in reaction to media comments. We will have to wait for the use of evidence-based practices and for bridging the gap between science and practice in penitentiary institutions.

The present study had no specific financing and was conducted as part of the official work for the Prosecutor General Bureau of the Supreme Court of Justice of the Province of Buenos Aires in a covenant with the Master Course in Forensic Psychiatry at the School of Medicine of the National University of La Plata.

The author translated the instruments jointly with an official translator. The results of the present study were provided in a work report to the judicial institution.

Previous results of the program were published in articles mentioned in the bibliography.