Among the statistical techniques for single-case designs, the effect size indices based on nonoverlap of data between phases stand out for their usefulness in clinical practice (Manolov & Rochat, 2024; Parker et al., 2011; Sanz & García-Vera, 2015). These indices, which we will refer to as NES (non-overlap effect size) indices, do not attempt to test the statistical significance of the therapeutic effect or change, but do allow that change to be quantified and its magnitude to be assessed, thereby overcoming some of the problems associated with visual data analysis. To do this, NES indices express as a percentage the number of treatment data points that represent a reduction in dysfunctional variables (or an increase in functional variables) relative to baseline.

Moreover, compared to other statistical analysis techniques, NES indices appear particularly useful for analyzing the single-case designs typically carried out in clinical practice. For example, time series analysis requires a very large number of data points in both baseline and treatment phases and is complex in terms of statistical knowledge and computational work. Randomization-based nonparametric tests, on the other hand, require delaying the start of treatment until a moment determined by a random procedure, without considering the ethical, practical, and clinical reasons that may call for the treatment to begin as soon as possible or at another more appropriate time. Unlike the other two types of analysis, NES indices can be calculated with very few data points in either the baseline or treatment phases. They are extremely simple to compute, even manually from data graphs, and do not require randomizing the timing of the treatment. Instead, treatment can be applied based on clinical or practical considerations.

Several NES indices have been proposed (Parker et al., 2011), but among them the percentage of nonoverlapping data, the percentage of data exceeding the median, and the nonoverlap of all pairs index stand out, known by their English acronyms as PND, PEM, and NAP, respectively. PND and PEM are outstanding for their ease of calculation and popularity (Escudero et al., 2018; Peltier et al., 2024; Scruggs & Mastropieri, 2013), while NAP stands out for its strong statistical performance (Manolov et al., 2011; Parker et al., 2011).

PND is defined as the percentage of treatment phase data points that surpass the most extreme data point from baseline, and is calculated by counting the number of treatment phase data points that exceed (above, in the case of functional variables, or below, in the case of dysfunctional ones) the most extreme data point from baseline, and dividing this number by the total number of data points in the treatment phase (Mastropieri & Scruggs, 1985-1986). Despite its limitations, it is one of the oldest and most widely used indices and, therefore, the one that most easily allows comparison of a given study’s results with those in the scientific literature (Scruggs & Mastropieri, 2013).

PEM is defined as the percentage of treatment phase data points that exceed (above for functional variables or below for dysfunctional ones) the median of the baseline data (Ma, 2006). It is calculated by dividing the number of treatment phase data points that exceed the baseline median by the total number of data points in the treatment phase.

Finally, NAP, developed by Parker and Vannest (2009) to address limitations of other NES indices such as PND and PEM, considers all possible overlaps between baseline and treatment data. As such, it can be interpreted as the percentage of nonoverlapping data between the two phases. NAP is calculated by pairing each baseline data point with each treatment data point, then: (a) counting all nonoverlaps, meaning pairs in which the treatment data point exceeds the baseline point in the functional direction; (b) counting all ties, which are pairs where the values are equal; (c) adding the number of nonoverlaps to half the number of ties; and (d) dividing this sum by the total number of possible pairs between baseline and treatment data points.

Each NES index has its advantages and disadvantages (Dowdy et al., 2021; Parker et al., 2011), but they all share the limitation that a therapeutic change of large magnitude does not necessarily imply that such a change is clinically significant. It is true that, generally, therapeutic effects of large magnitude tend also to be clinically significant, but it is still possible to obtain NES indices equal to 100% in a single-case design without the therapeutic changes necessarily having a clear impact on the patient’s daily functioning or being large enough to assume that the patient has recovered from their psychological disorder and returned to normal functioning.

To overcome this limitation, Sanz and García-Vera (2015) proposed combining NES indices with the evaluation of the clinical significance of the therapeutic change in order to create clinically significant effect size indices, which will be referred to from now on as CSES indices, based on the initials of their name in English (clinically significant effect size). Specifically, the CSES indices involve combining the statistical approach to clinical significance of Jacobson and Truax (1991) with the NES indices, in such a way that the data overlap between baseline and the treatment phase is assessed based on whether the treatment data point represents a clinically significant improvement or recovery compared to the baseline data point.

Jacobson and Truax’s (1991) method assumes that a clinically significant change would mean that a patient’s score on an instrument measuring a relevant psychological construct (e.g., depression, anxiety, quality of life) no longer belongs to the distribution of scores on that instrument for a dysfunctional population (e.g., Spanish patients with psychological disorders), but rather to the distribution of a functional population (e.g., the general Spanish population). Consequently, to determine the existence of a clinically significant change in a patient, the Jacobson and Truax’s (1991) method involves, first, establishing a cutoff score (C) on the reference instrument that the patient must reach in order to move from a dysfunctional to a functional distribution. To this end, the authors propose three operational definitions of C, of which, if information from both distributions is available, the best is the weighted midpoint between the means of the functional and dysfunctional distributions, given by the following formula:

In this formula, SD_n and SD_p represent the standard deviations of the instrument in the normal (or general) and patient populations, respectively, and M_n and M_p are the means of the instrument in the normal and patient populations, respectively.

Based on this first criterion from the Jacobson and Truax’s (1991) method, a CSES recovery index could be proposed, defined as the percentage of treatment phase data points that indicate a clinically significant recovery. It would be calculated by counting the number of treatment data points that exceed the cutoff value C (above in the case of functional variables or below in the case of dysfunctional ones) and dividing this number by the total number of data points in the treatment phase. Its calculation formula would be the following:

Secondly, the Jacobson and Truax’s (1991) method involves estimating whether the change indicated by the scores on an instrument is not due to measurement error but instead reflects a reliable, real change in the psychological construct of interest. To do this, the authors propose a “reliable change index”, or RCI, which takes into account the standard error of the difference between two scores on the instrument (s_dif) and is calculated using the following formulas:

In these formulas, x₂ is the patient’s score on the instrument at a given time (e.g., posttreatment), x₁ is the score at an earlier time (e.g., pretreatment), s_x is the standard deviation of the instrument scores in the patient population, and r_xx is the internal consistency reliability of the instrument in that population.

Based on this second criterion, a treatment phase data point could be considered a clinically significant improvement compared to the baseline data if it represents a change in the direction of functionality that exceeds the standard error of the difference between two scores on the instrument (s_dif). Therefore, if the comparison point from baseline is the most extreme value, a PND for improvement could be calculated; if the comparison point is the baseline median, a PEM for improvement could be calculated; and if all possible comparisons between baseline and treatment data are taken into account, a NAP for improvement could be calculated. The formulas for calculating these three indices would be the following:

The standard error of the difference between two scores (s_dif) describes the range of the distribution of change scores that would be expected if no real change occurred, such that an RCI greater than 1.96 would be very unlikely (p < .05) to occur without a real change having taken place. Consequently, in the original method by Jacobson and Truax (1991), the difference between the treatment and baseline data must exceed the value obtained by multiplying s_dif by 1.96 in order to ensure, with a 95% confidence level, that the change is not due to measurement error of the instrument. However, this is a very strict criterion compared to the NES indices, which only require that the treatment and baseline data do not overlap, that is, that they are not equal, even if the difference is only one point. In the present proposal, an intermediate solution is chosen by relaxing the overly strict criterion of Jacobson and Truax (1991) and requiring that the difference between the treatment data point and the baseline data point exceed s_dif, but not necessarily the product of s_dif and 1.96.

The aim of this study was to examine the validity of the CSES indices proposed by Sanz and García-Vera (2015). To that end, data were analyzed from a group of patients who had received psychological treatment for posttraumatic stress disorder (PTSD) resulting from a terrorist attack they had experienced many years earlier. For each patient, NES indices (PND, PEM, and NAP) and CSES indices (CSES recovery, PND improvement, PEM improvement, and NAP improvement) were calculated based on the measures of posttraumatic stress symptomatology obtained during baseline and treatment phases. These groups of indices were then correlated with each other to assess their convergent validity and also with various posttreatment criteria of therapeutic success or effectiveness in order to assess their criterion validity.

Data from two published studies were reanalyzed to examine the efficacy and clinical utility of trauma-focused cognitive behavioral therapy (TF-CBT) in victims of terrorism suffering from very long-term emotional disorders (Gesteira et al., 2025; see also Gesteira et al., 2018; Moreno et al., 2019). For this study, all participants from those two studies were selected who had received a DSM-IV diagnosis of PTSD related to a terrorist attack based on a structured diagnostic interview administered at pretreatment, had completed the instrument for assessing posttraumatic stress symptomatology at all measurement time points before, during, and after treatment, and had also completed the structured diagnostic interview at posttreatment.

The final sample of the present study consisted of 30 adults, 12 men and 18 women, aged between 32 and 65 years (mean age = 46, SD = 10.5). The ages at which they had experienced the terrorist attack ranged from 3 to 57 years, with a mean of 27.6 years, and an average of 18.4 years had passed from the time of the attack to their participation in the studies. Of the patients, 62.7% had been injured in the terrorist attack, 20.3% were relatives of individuals injured in an attack, 10.3% were relatives of individuals killed in an attack, and the remaining 6.7% had been injured in an attack and were also relatives of someone who had been killed or injured in the attack.

After providing verbal informed consent, all participants voluntarily completed a psychological assessment by telephone, during which the PCL-S was administered. Subsequently, after signing a written informed consent form, they completed an in-person psychological assessment in which both the SCID-I-CV and the PCL-S were administered. After receiving a PTSD diagnosis, participants voluntarily began—after signing a new informed consent form—an individual and free of charge trauma-focused cognitive behavioral therapy (TF-CBT) program consisting of 16 sessions, each lasting 60-90 minutes and held weekly. Before the start of each odd-numbered treatment session, patients completed the PCL-S. After the 16 weeks of treatment, patients were reassessed with the SCID-I-CV and the PCL-S in the posttreatment evaluation. All assessments and treatments were carried out by licensed general health psychologists with postgraduate training in clinical psychology and in providing psychological care to victims of terrorism, including specific training in the assessment protocol and the TF-CBT program. Further details of the procedure can be found in Gesteira et al. (2018), Gesteira et al. (2025), and Moreno et al. (2019).

The PCL-S scores from the pre-treatment telephone assessment, the pre-treatment in-person assessment, and the beginning of the first treatment session were considered as baseline measures, and the PCL-S scores from treatment sessions 3, 5, 7, 9, 11, 13, and 15 were considered as treatment phase measures. Based on these baseline and treatment measures, and for each patient, the PND, PEM, and NAP indices were calculated using the web application by Pustejovsky et al. (2024), and the CSES indices—CSES recovery, PND improvement, PEM improvement, and NAP improvement—were calculated manually and using the individual patient graphs.

To calculate these indices, we used the cutoff value (C = 29.17) and the standard error of the difference (s_dif = 6.06) for the PCL-S, as reported by Sanz and García-Vera (2015) based on data from Reguera et al. (2014), who studied a sample of 589 victims of terrorist attacks. Accordingly, for each patient, a PCL-S score in the treatment phase that represented a decrease of more than 6 points compared to the most extreme baseline PCL-S score (PND improvement), compared to the median of the baseline PCL-S scores (PEM improvement), or in a pairwise comparison between baseline and treatment PCL-S scores (NAP improvement) was considered an improvement data point, whereas a PCL-S score below 29 in the treatment phase was considered a recovery data point (CSES recovery).

Descriptive statistics were calculated for all indices, and to examine the convergent validity of the new indices, Pearson correlation coefficients were calculated between the NES and CSES indices. To examine the criterion validity of the NES and CSES indices, their Pearson correlation coefficients were calculated with the following three therapeutic success or effectiveness criteria assessed at posttreatment: the total score on the PCL-S, the absence of a PTSD diagnosis, and clinically significant recovery defined as a PCL-S score below 29, with the latter two criteria being dichotomous.

Table 1 presents the means, standard deviations, and ranges of the NES and CSES indices. Taking into account the conventional values for interpreting NES indices (Ma, 2006; Parker & Vannest, 2009; Scruggs & Mastropieri, 1998), the means of the PND, PEM, and NAP indices indicated that, in the patient group of the present study, treatment effectiveness was questionable (50% ≤ PND < 60%), the treatment was moderately effective (70% ≤ PEM < 90%), and the treatment effect size was medium (66% ≤ NAP < 92%), respectively.

As for the posttreatment therapeutic success criteria, the mean PCL-S score was 26.8 (SD = 9.1, range = 17-57), 86.7% of patients no longer had a PTSD diagnosis, and 66.7% showed clinically significant recovery—that is, a PCL-S score below 29.

Table 2 presents the correlations between the NES and CSES indices. The CSES indices showed significant correlations with the NES indices which, according to Cohen’s (1988) standards, were large in size (r ≥ .50), with values ranging from .58 to .91, except for the correlation between the CSES recovery index and the PEM, which was .48. On the other hand, indices based on similar calculations showed not only significant but also very large correlations, such that correlations among the NES indices ranged from .67 to .91, and correlations among the CSES improvement indices (PND improvement, PEM improvement, and NAP improvement) ranged from .73 to .90. Interestingly, the CSES recovery index showed significant correlations with the CSES improvement indices (ranging from .52 to .58) that were very similar to the significant correlations it had with the NES indices (ranging from .48 to .59).

Table 3 presents the correlations of the NES and CSES indices with the therapeutic success criteria assessed at posttreatment. All indices showed statistically significant, negative, and large correlations with PCL-S scores at posttreatment, with correlations ranging from -.47 to -.65. Similarly, all indices, except for the PEM improvement index, showed statistically significant, positive, and medium-sized correlations with the recovery criterion at posttreatment, with correlations ranging from .35 to .49. However, no index, except for PEM, showed a statistically significant correlation with the absence of a PTSD diagnosis at posttreatment; in the case of PEM, the correlation was significant and medium to large in size (r = .40).