Measuring Cyber Dating Violence: Reliability and Validity of the Escala de CiberViolencia en Parejas Adolescentes (Cib-VPA) in Spanish Young Adults

Introduction

The terms “cyber dating violence” or (with slightly different nuances) “cyber dating abuse” are understood to refer to a set of repeated behaviours aimed at controlling, undermining or causing harm to one’s partner (Borrajo et al., 2015; Reed et al., 2016), and include behaviours such as frequently visiting said partner’s social media profile, sending insulting or threatening messages, spreading negative information about them, or stealing or misusing their passwords (Borrajo & Gámez-Guadix, 2016; Borrajo et al., 2015; Darvell et al., 2011). Cyber dating violence therefore happens quickly and is easy to perpetrate. It is usually public in nature and can occur anywhere and anytime, even after the relationship has ended (Bennett et al., 2011; Stonard et al., 2017; Zweig et al., 2014), which increases its potential to accentuate the experience of victimisation (Korchmaros et al., 2013; Stonard et al., 2017; Zweig et al., 2014).

The prevalence studies carried out to date report very diverse data, with percentages ranging from 12% to 54% (Stonard et al., 2017) or even from 6% to 91% (Brown & Hegarty, 2018), although they are better understood when a comprehensive assessment of the specific types of behaviour involved is carried out (see the study by Borrajo et al., 2015). Depending on the measurement instruments used, there are many disparities also in terms of gender differences, although most studies report no significant or only very slight differences between men and women in relation to these behaviours (Leisring & Giumetti, 2014; Reed et al., 2016).

Cyber dating violence has been clearly linked to other types of violence (Zweig et al., 2013), including ‘offline’ violence (Leisring & Giumetti, 2014; Viejo et al., 2016). These two types of violence (online and offline) are considered to be co-occurring experiences and each is viewed as a risk factor for the emergence of the other (Temple et al., 2016).

In recent years, many different measures have been designed to assess this phenomenon. These instruments vary widely in terms of the factors considered (e.g., the digital means used for perpetrating violence), the way in which they define the term ‘relationship’, their target age group, whether they take into account past relationships or only focus on current ones, the type and number of violent behaviours, and the time frame included in the assessment. Another relevant factor given the bidirectional nature of cyber dating violence is whether the instruments assess victimisation, perpetration, or both (for more information, see the review study by Brown & Hegarty, 2018). Their selection will therefore depend on the aims of the research being conducted and the sample to which they are to be administered, although some are simply not suitable for assessing such a complex phenomenon. Moreover, the main limitation of these instruments is that few of them provide evidence of their psychometric properties (not only reliability, but validity also) (Borrajo et al., 2015; Rodríguez-Domínguez et al., 2020).

One of the main difficulties linked to researching cyber dating violence in Spanish-speaking contexts is the scarcity of instruments that have been developed and validated with the Spanish population. One exception to this is the Cyberdating Q_A scale (Sánchez et al., 2015). However, although this scale offers useful data on adolescent romantic relationships and how they are influenced by the social media, it does not allow a joint assessment of both perpetration and victimisation (Cava & Buelga, 2018). The Cyber Dating Abuse Questionnaire (CDAQ; Borrajo et al., 2015) is another instrument worth considering due to its good psychometric properties. The questionnaire aims to evaluate cyber dating abuse (both perpetration and victimisation) in young adults, and covers a wide range of behaviours (control and direct aggression) to be scored on a 6-point Likert-type scale (1 = never, 2 = not in the last year, but it occurred before, 3 = rarely: 1 or 2 times, 4 = sometimes: between 3 and 10 times, 5 = often: between 10 and 20 times, and 6 = always: more than 20 times).

It should be noted that the Escala de Ciber-Violencia en Parejas Adolescentes (Cib-VPA; Cava & Buelga, 2018) has also been shown to be a suitable instrument for assessing cyber dating violence among adolescents in the Spanish context. The questionnaire evaluates a range of conducts including both excessive control behaviours and direct aggression. Moreover, unlike some other instruments (Bennett et al., 2011; Fox & Warber, 2014), it measures both perpetration and victimisation, which helps researchers gain a more precise understanding of the phenomenon, as previous studies have shown that the two are often reciprocal.

The Cib-VPA comprises 20 items evaluating two subscales of cyber dating violence. The first, ‘Perpetrated cyber dating violence’ (10 items), refers to behaviours directed towards one’s partner, whereas the second, ‘Cyber dating violence victimisation’ (10 items), refers to those perpetrated by one’s partner towards oneself. Each subscale includes the same item-content and is divided into two factors: ‘cyber aggression’, which focuses on actively aggressive online behaviours, such as threatening or humiliating, and ‘cyber control’, which refers to monitoring, checking or surveillance behaviours.

Participants are asked to respond to each item on a four-point Likert-type scale (1 = never, 2 = sometimes, 3 = usually, and 4 = always).

The original version of the Cib-VPA was found to have good psychometric properties, with evidence being provided not only of its reliability, but of its validity also (Cava & Buelga, 2018). Indeed, the original version was found to have alpha coefficients of .97 and .94 for cyber aggression and cyber control (respectively), within the Perpetrated cyber dating violence subscale; and of .97 and .92 for cyber aggression and cyber control (respectively), within the Cyber dating violence victimisation subscale. For this purpose, first, an exploratory principal axis factor analysis with Oblimin rotation was conducted, that yielded the aforementioned two-subscale structure, in which each subscale comprised two factors. After this, a CFA was carried out, testing both a possible two-factor model and a possible one-factor model. This was applied to each of the two sub-scales. Since both factors (cyberaggression and cybercontrol) were found to have high correlations with each other in the two subscales (Perpetrated cyber dating violence and Cyber dating violence victimisation) both a bifactor model and a unifactor model were tested separately in both subscales. In addition, in the case of the Perpetrated cyberbullying subscale, a two-factor model was also tested excluding item 4.

The Cib-VPA was also found to have concurrent validity, as evident in the correlations observed with the Conflict in Adolescent Dating Relationship Inventory (CADRI; Wolfe et al., 2001), a widely-used measure of offline dating violence.

Despite its positive psychometric properties, however, this version has some aspects that require improvement. First, to date, no other validation study has been conducted that replicates or supports the data obtained in the original one (either in Spain or in any other country). It would therefore be interesting to verify whether this structure and the results obtained are maintained in other samples. Specifically, and given the scarcity of instruments for evaluating the construct in our country, the scale needs to be validated with a sample of young adults so that it can be used with a wider age range. Second, it is important to assess possible correlations between the Cib-VPA and other measures of intimate partner violence (in both its online and offline forms), since the literature indicates that said correlations are usual. The original validation study did not include other online dating violence instruments. Third, it would be useful to explore possible gender differences in cyber dating violence, since many other studies have found that such differences may not exist (Bennett et al., 2011; Reed et al., 2016).

On the basis of the information outlined above, the aims of the present study were as follows:

1. To analyse the factor structure of the Cib-VPA in a sample of Spanish adults, using the confirmatory factor analysis (CFA) method to determine whether the structure proposed by Cava and Buelga (2018) is replicated.

2. To evaluate the internal consistency of Cib-VPA scores using the Cronbach’s alpha coefficient and convergent and discriminant validity, by correlating Cib-VPA scores with other measures of offline and online dating violence.

3. To explore gender differences in Cib-VPA scores.

Method

Design of the Study

This is a cross-sectional study for the evaluation of the metric properties of the Escala de Ciber-Violencia en Parejas Adolescentes (Cib-VPA).

Subjects

The sample comprised 298 undergraduate students, 24.8% men (n = 75), 74.5% women (n = 222), and 0.7% who identified themselves as ‘other’ (n = 2), enrolled in the Education Studies Degree at the University of the Basque Country (Spain). Participants ranged in age from 19 to 40 years (M = 21.02, SD = 2.59). In terms of family structure, 76.8% (n = 229) came from two-parent families, 15.1% (n = 45) had divorced parents, 5% (n = 15) came from a family in which one of the parents had died, and 3% (n = 9) from other types of family. With regard to parental education level, 1.7% (n = 5) of participants’ mothers and 1.3% (n = 4) of their fathers had no qualifications; 13.4% (n = 33) of participants’ mothers and 13.4% (n = 40) of their fathers had primary-level qualifications; 52.1% (n = 155) of mothers and 62.7% (n = 187) of fathers had secondary-level qualifications; and 35.2% (n = 105) of mothers and 22.5% (n = 67) of fathers had university qualifications.

Measurements

Escala de Violencia de Pareja en las Redes Sociales en Adolescentes (e-VPA; Cava & Buelga, 2018), which has been extensively described above.

Conflict in Adolescent Dating Relationship Inventory (CADRI; Wolfe et al., 2001) was used in its Spanish version (Fernández-Fuertes et al., 2006). Although it was originally designed for use with an adolescent population, it has also been used with university students (e.g., Benítez & Muñoz, 2014; Cascardi et al., 2019; Muñoz-Ponce et al., 2020; Rojas-Solís, 2011). The CADRI is a self-report instrument comprising 34 items with bidirectional questions (victim/perpetrator). There are four response options: never, seldom (one-two times), sometimes (three-five times), and often (six times or more). The CADRI assesses five different types of abusive behaviours: physical abuse, threatening behaviour, sexual abuse, relational abuse, and verbal/emotional abuse. Cronbach’s alpha values in the current sample were as follows: relational aggression, α = .25; verbal aggression, α = .79; physical aggression, α = .48; relational victimisation, α = .63; verbal victimisation, α = .87; and physical victimisation, α = .06. Overall, internal consistency was α = .87 for perpetration and α = .89 for victimisation. Given their lack of internal consistency, relational aggression, physical aggression, and physical victimisation will not be considered in the statistical analyses.

Controlling Partners Inventory (CPI; Burke et al., 2011) comprises18 items evaluating four dimensions or factors of cyber dating abuse among adults: (1) Photos, camera, and SpyWare refers to using hidden webcams or SpyWare to monitor a partner’s behaviour or threatening to or posting inappropriate, nude or embarrassing photos of a partner; (2) Excessive communication evaluates whether the respondent makes excessive numbers of cell phone calls or texts; (3) Threatening involves items about sending threatening phone calls, text messages, or emails; and (4) Checking behaviours refers to using a partner’s password to check their computer and checking mobile phone call and email histories. For each of the monitoring behaviours, participants are asked to respond to two items: “I have done this” (‘CPI-Self’) and “partners have done this to me” (‘CPI-Partner’) on a five-point Likert-type scale ranging from never to four or more times. The original version of the CPI was found to have good psychometric properties (Borrajo et al., 2015; Brown & Hegarty, 2018). In the current sample, the internal consistency values were as follows. For CPI-Self: Photos, camera, and SpyWare α = .49; excessive communication α = .79; threatening α = .70; and checking behaviours α = .70; For CPI-Partner: Photos, camera, and SpyWare α = .73; excessive communication α = .73; threatening α = .75; and checking behaviours α = .63. Given its lack of internal consistency, Photos, camera and SpyWare will not be considered in the statistical analyses.

Procedure

All students read and signed a written consent form prior to participation. They were also told that they could withdraw from the study at any point during the assessment and that participation was completely voluntary. Surveys were administered in paper-and-pencil form, collectively, and outside class time. The study was presented as a survey to explore dating relationships among university students. Students did not receive anything in return for their participation.

A total of 410 undergraduates participated in the study, of which 72.68% (n = 298) had previous dating experience or were currently in a relationship. Dating experience was assessed by asking participants if they were currently in a dating relationship or if they had been in the past.

Statistical Analyses

First, the descriptive statistics for the items, the hypothesised factors (means, standard deviations, asymmetry, and kurtosis) and the psychometric properties of the Cib-VPA were analysed. A confirmatory factor analysis (CFA) was performed to verify the factor structure of the test, as the instrument had already been created and validated, and was therefore based on a solid theoretical framework. Our study therefore assumed the hypothesis of replicability of the original structure. In the original study, the authors conducted two CFAs (one for each dimension of the questionnaire), so these two models were tested firstly. As both models were over-fitted, we performed a single CFA with the four subscales. Thus, all the items with their respective factors were introduced at the same time in the model, as cyber perpetration and cyber victimisation are constructs that, although different, nevertheless form part of the same theoretical model (the same questionnaire, in this case). By doing this (i.e., including all the factors in the same model), it is possible to control for the shared and non-shared variance between factors and items.

The CFA was carried out following the unweighted least squares estimation method of the parameters, as the data did not meet multivariate normality according to Mardia’s test (Bollen, 1989). Although in the original study the authors used the WLSMV, which is the best method in the case of SEM based on ordinal non-normally distributed data (Muthén et al., 1997), the ULS has also been proposed as a valid method for non-normal data, and provides very similar outputs (Forero et al., 2009).

The model was assessed using several goodness of fit indices, including the χ²/Satorra-Bentler ratio, with values below 2 indicating a good fit (Brooke et al., 1988); the non-normed fit index (NNFI), for which values must be .90 or higher (Bentler & Bonnet, 1980), and the root mean squared error of approximation (RMSEA), with values lower than .08 indicating an acceptable fit (Browne & Cudeck, 1993). Moreover, the Langrange multiplier test was performed in order to verify the improvement of the fit of the model by establishing correlations between pairs of errors (Breusch & Pagan, 1980). The EQS 6.1 software package (Bentler, 2005) was used for the analyses. After the structure of the questionnaire had been obtained, the internal consistency (McDonald’s ω) of the resulting factors was analysed. McDonald’s (1999) omega was used to estimate internal consistency, since it is a better estimator of reliability than Cronbach’s alpha (Dunn et al., 2014). It was interpreted following the EFPA guidelines (Evers, et al., 2008), so values under .70 are insufficient, between .70 and .80 are sufficient; between .81 and .90 are good; and above .90 are excellent. Once the optimal model had been found, the convergent and discriminant validity of the Cib-VPA were analysed. To this end, a bivariate correlation matrix (Pearson’s r) between the Cib-VPA factors and the other measures used in the present study (CADRI and CPI) was calculated, analysing the association between the factors obtained and other offline and online dating violence measures.

Finally, gender differences in the Cib-VPA factor scores were calculated using t-tests, with significance levels being reported in all cases. These analyses were carried out despite the fact that factorial invariance (Byrne, 1989; Chen, 2007) by gender could not be computed due to the small sample size of the group of male (n = 75). Moreover, it was verified that item 8 and item 13 had a SD of 0.

Ethical Considerations

The study procedure was approved by the Research Ethics Board at the University of the Basque Country (CEISH-UPV/EHU, M10/2018/2018).

Results

First, the factor structure of the questionnaire was analysed. The results did not support the original structure of the questionnaire, as both victimization model (χ²/df = 0.15, NNFI = 1.10, RMSEA < .001) and perpetration model (χ²/df = 0.18, NNFI = 1.12, RMSEA < .001) were over-fitted, since the χ²_{Satorra-Bentler} and the degrees of freedom ratio were less than 1 (Jöreskog, 1970).

Then, other CFA including all 20 items resulted in an over-fitted model, since the χ²_{Satorra-Bentler} and the degrees of freedom ratio were less than 1: χ²/df = 0.01, NNFI = .93, RMSEA < .001. Given that the χ² statistic was so low, Lagrange multiplier test did not suggested any pair of correlations in order to improve the fit of the model. When analysing the factor loadings of the items in this model, two items in the cyber aggression factor of the perpetrated cyber dating violence subscale (item 7: “I have insulted or threatened my boyfriend/girlfriend in private” and item 8: “I have told my boyfriend/girlfriend that, if he/she breaks up with me, I will tell people about or publish it”) were found to be close to 0. Both items were therefore eliminated and the CFA was run again with the remaining 18 items. This new analysis revealed a satisfactory fit of the model to the data according to the three indices considered (χ²/df = 1.48, NNFI = .93, CFI = .96, RMSEA = .040 CI 90% [.028, .052]). The Lagrange multiplier did not suggest any pair of correlation between pairs of errors of items loading in the same factor. The results are presented in Figure 1, including the factor loadings of the items (l coefficients) and the errors of estimation (δ coefficients). As shown in the figure, the subscales correlated in both parts of the questionnaire, and the factor loadings ranged from .26 (item 4, cyber aggression factor in the perpetrated cyber dating violence subscale) to .93 (item 7, cyber control factor in the perpetrated cyber dating violence subscale). Finally, internal consistency was calculated. McDonald’s omega coefficients were .60 and .86 for cyber aggression and cyber control (respectively), within the perpetrated cyber dating violence subscale; and .79 and .63 for cyber aggression and cyber control (respectively) within the cyber dating violence victimisation subscale. Total cyber perpetration attained .72, and total cyber victimization attained .77. Then, both cyber aggression and cyber control attained lower coefficients than the criterion. Then, it could be more recommendable to work with the total scores.

Once the structure of the instrument had been obtained, descriptive analyses were conducted of the items and the resulting factors. The results are outlined in Table 1. As shown in the table, the mean values of the items were low, while the distributions were markedly asymmetrical (positive) and leptokurtic. This may be due to the nature of the construct being evaluated, since violence is a variable whose distribution does not conform to normality.

Next, the convergent and discriminant validity of the instrument were analysed (see Table 2). In general, both the convergent and discriminant validity of the Cib-VPA were confirmed, given that in the case of variables that were analogous to those from the other two instruments used (CADRI and CPI), correlations with the Cib-VPA factors were positive, significant and of greater magnitude than those observed with the variables pertaining to the opposite role (e.g., perpetration and victimisation).

Finally, gender differences in the Cib-VPA subscales and factors were analysed (see Table 3). No statistically significant differences were observed in relation to any subscale or factor, indicating that men and women scored similarly for both perpetration and victimisation.

Discussion

The main aim of the present study was to obtain validity evidence the Escala de Ciber-Violencia en Parejas Adolescentes (Cib-VPA; Cava & Buelga, 2018) among the young adult population in Spain. This makes a relevant contribution to the study of cyber dating violence, first, because few instruments to date have provided psychometric evidence of reliability and validity, as this scale does, and second, because there are very few instruments in Spanish that specifically assess this type of dating violence (Borrajo et al., 2015; Rodríguez-Domínguez et al., 2020).

So far, this instrument has only been applied to adolescent samples (Cava & Buelga, 2018; Muñiz & Monreal, 2017). However, cyber dating violence needs to be evaluated further among older age groups, since as adolescents grow older and reach young adulthood, they are more likely to engage in more couple relationships. Furthermore, it is also important to extend the age range since older adults may differ in their answers to the same items, and some items may not even be applicable to them.

The CFA provided evidence for an 18-item model, divided into 2 subscales with 2 factors per subscale. As in the original instrument (Cava & Buelga, 2018), two subscales were distinguished also in the validated version, namely ‘Perpetrated cyber dating violence’ and ‘Cyber dating violence victimisation’, and within each subscale, two factors were maintained: ‘cyber aggression’ and ‘cyber control’. All were found to have sufficient internal consistency values. Carrying out a single CFA including all items in the same model may be considered a more accurate analysis than the one carried out by Cava and Buelga (2018), who performed two independent CFAs, since while cyber perpetration and cyber victimisation are considered different factors, they nevertheless form part of the same framework, thus making it possible to control for the shared and non-shared variance between the factors and items, something which could not be tested in the original study.

However, in this studyby thisby , two items from the cyber aggression factor of the ‘Perpetrated cyber dating violenc’e subscales had to be removed, as their loadings were close to 0. In other words, very few participants (almost none) responded affirmatively to these items. The social desirability bias is likely to play an important role in this issue, prompting people not to acknowledge that they engage in this type of behaviour towards their partner. Indeed, overall, studies have reported a negative relationship between social desirability and intimate partner violence scores, and research has shown that violence perpetration reports appear to be more prone to the influence of social desirability than reports of victimisation (Bell & Naugle, 2007; Sugarman & Hotaling, 1997). Although it may seem that eliminating these items from only one subscale and not from its analogue would affect content validity, the truth is that the perpetrated cyber dating violence and cyber dating violence victimisation subscales are independent constructs, even though they constitute two parts of the same questionnaire (as reflected in the SEM model). Moreover, in the construction of the questionnaire, the original authors did not intend to enable an item-by-item comparison using the test, suggesting that it is not strictly necessary for the items in the ‘Perpetrated cyber dating violence’ and ‘Cyber dating violence victimisation’ subscales to be totally equivalent. Nevertheless, future validation studies should continue to analyse the role of these two items in the test, and if necessary, reformulate or eliminate them definitively.

The analysis of the correlations between the Cib-VPA factors revealed that all correlations (regardless of whether they referred to perpetration or to victimisation) were significant and of moderate magnitude. This is consistent with most research carried out to date in this field, the results of which point to bidirectionality as a core characteristic of cyber dating violence. In other words, they suggest that young people are both aggressors and victims in their dating relationships (Gámez-Guadix et al., 2018; Leisring & Giumetti, 2014; Reed et al., 2016).

The positive and significant correlations found between the Cib-VPA and the CADRI and CPI further demonstrate the convergent and discriminant validity of the instrument. It should also be noted that in both cases (CADRI and CPI) the correlations between the subscales of the tests and the factors of the Cib-VPA were higher when they both referred to the same subscale (perpetrated violence or victimisation). In other words, correlations were higher when both the variables in question assessed either perpetrated violence or victimisation than when one assessed perpetration and the other victimisation.

Correlations between the Cib-VPA and the CADRI were also found in the original validation study (Cava & Buelga, 2018), and many others have reported a relationship between offline and online dating violence (Leisring & Giumetti, 2014; Viejo et al., 2016). Some authors explain this in terms of continuity between the real and the virtual worlds, with adolescents and young people reproducing the behaviours they engage in outside the social media in the virtual context (Cava & Buelga, 2018). Others suggest that, rather than being distinct forms of abusive behaviour, offline and online dating violence may in fact be experienced as a continuum (Stonard et al., 2017). However, the novel contribution made by this study is to administer a questionnaire that evaluates exactly the same construct as the CPI. As expected, correlations between the two instruments were observed, which suggests that both may be assessing the same construct.

As regards gender differences, none were observed, with both men and women being aggressors and victims of cyber dating violence. This is consistent with most previous studies (Leisring & Giumetti, 2014; Reed et al., 2016), and may indicate a tendency for men and women in this age group to be equally involved in this particular type of violence, which takes place through the social media and mobile phone apps.

The present study is subject to certain limitations, the principal one being the fact that the sample selection process was not systematically randomised, which may have resulted in possible biases. In order to improve the representativeness of the sample, future studies should broaden the scope to include young people who are working or studying outside the university field, and should try to balance the proportion of men and women. Furthermore, the inclusion of clinical samples characterised by high levels of violence in their relationships would also be of great interest in this regard.

Finally, the Cib-VPA is a self-report measure of cyber dating violence. Such measures may be influenced by conscious or unconscious attempts by participants to respond as socially expected. It is therefore advisable to use them in combination with interviews or other implicit or indirect measures of dating violence, in order to reduce the response bias.

Despite these limitations, however, the Cib-VPA seems to be an adequate instrument for measuring cyber dating violence among young Spanish-speaking adults. It would therefore be interesting to obtain validity evidence it in other countries, in order to enable cross-cultural and comparison studies.