ES EN
Head of The European Journal of Psychology Applied to Legal Context
Vol. 8. Num. 2. - 2016. Pages 69-77

Validity and reliability of the Cyber-aggression Questionnaire for Adolescents (CYBA)

[Validez y fiabilidad del Cuestionario de Ciberagresión para Adolescentes (CYBA)]

David Álvarez García1 , Alejandra Barreiro-Collazo1 , José Carlos Núñez Pérez1 , Alejandra Dobarro1
1Univ. Oviedo, Fac. Psicología, Asturias, España


https://doi.org/10.1016/j.ejpal.2016.02.003

Abstract

Cybercrime is a growing and worrisome problem, particularly when it involves minors. Cyber-aggression among adolescents in particular can result in negative legal and psychological consequences for people involved. Therefore, it is important to have instruments to detect these incidents early and understand the problem to propose effective measures for prevention and treatment. This paper aims to design a new self-report, the Cyber-Aggression Questionnaire for Adolescents (CYBA), to evaluate the extent to which the respondent conducts aggressions through a mobile phone or the internet and analyse the factorial and criterion validity and reliability of their scores in a sample of adolescents from Asturias, Spain. The CYBA was administered to 3,148 youth aged between 12 and 18 years old along with three self-reports to measure aggression at school, impulsivity, and empathy. Regarding factorial validity, the model that best represents the structure of the CYBA consists of three factors (Impersonation, Visual-sexual Cyber-aggression, and Verbal Cyber-aggression and Exclusion) and four additional indicators of Visual Cyber-aggression-Teasing/Happy Slapping. Regarding criterion validity, the score on the CYBA correlates positively with aggression at school and impulsivity and negatively with empathy. That is the way cyber-aggression correlates with these three variables, according to previous empirical evidence. The reliability of the scores on each item and factor of the CYBA are adequate. Therefore, the CYBA offers a valid and reliable measure of cyber-aggression in adolescents. 

Resumen

Los ciberdelitos constituyen un problema creciente y preocupante, sobre todo si están involucrados menores. Las ciberagresiones en adolescentes, en concreto, pueden generar consecuencias legales y psicológicas muy negativas para los implicados. Es importante, por tanto, disponer de instrumentos que permitan detectar precozmente estos hechos, así como comprender el problema de cara a plantear medidas eficaces para su prevención y tratamiento. El objetivo de este trabajo es diseñar un nuevo autoinforme, denominado Cuestionario de Ciberagresión para Adolescentes (CYBA), para evaluar en qué medida el informante ejerce agresiones a través del teléfono móvil o Internet y analizar la validez factorial y de criterio y la fiabilidad de sus puntuaciones en una muestra de adolescentes de Asturias (España). Para ello, se aplicó el CYBA a 3.148 jóvenes de 12 a 18 años, junto con tres autoinformes para medir agresión escolar offline, impulsividad y empatía. Respecto a la validez factorial, el modelo que mejor representa la estructura del CYBA es el compuesto por tres factores (suplantación, ciberagresión visual-sexual y ciberagresión verbal/exclusión) y otros cuatro indicadores de ciberagresión visual-burlas/happy slapping. Respecto a la validez de criterio, la puntuación en el CYBA correlaciona de manera positiva con agresión escolar offline e impulsividad y negativa con empatía, tres variables con las que la evidencia empírica previa indica que correlaciona la ciberagresión. La fiabilidad de las puntuaciones en cada ítem y factor del CYBA son adecuadas. Por todo ello, se concluye que el CYBA ofrece una medida válida y fiable de ciberagresión en adolescentes. 

The rapid development of electronic devices and applications for communication has recently changed the way adolescents socialise. The use of such communication methods is practically universal among youth in technologically developed countries. In Spain, 90.3% of 15-year old children have a mobile phone ( Instituto Nacional de Estadística–INE [National Statistics Institute], 2014 ), and 90% have a profile on at least one social network ( Ministerio del Interior [Home Office], 2014 ). These percentages are similar to those from other European countries, such as France ( Pouria & Brousse, 2011), Ireland (O’Neill & Dinh, 2015), or Sweden (Findahl, 2014 ). Despite the great advantages that these tools offer for establishing new friendships or maintaining contact with family or friends, they also present some significant potential dangers.

Sometimes adolescents take advantage of the anonymity offered by these media to intentionally harm, offend, or hurt other adolescents. The term cyber-aggression is typically used to refer to this type of behaviour ( Corcoran, McGuckin, & Prentice, 2015 ). Aggression through electronic devices can be even more serious than face-to-face aggression. It can occur at any time of the day and can be spread immediately to many people and, because it occurs at a distance, the perpetrators do not see the negative effects of aggression on the victim, hindering empathy and promoting recurrence ( Álvarez-García, Dobarro, & Núñez, 2015 ).

Cyber-aggression among adolescents occurs in many countries and constitutes an international problem ( Li, Cross, & Smith, 2012 ). However, the estimated percentage of adolescents involved in such acts varies depending on the definition of cyber-aggression, the sample, and the methodology used for the analysis. A recent study in Spain based on a large nationwide sample estimates that 4.2% of students in compulsory secondary education (CSE) admit to having recorded a peer through the use of a mobile phone audio or video application to use it against him/her; 3.4% admit to having recorded a peer to later force him/her with threats to do something that he/she did not want to do; 4.6% have sent messages over the internet or by mobile phone in which a peer was insulted, threatened, offended, or frightened; and 3.8% have spread photos or images over the internet or by mobile phone of a peer to use them against him/her ( Díaz-Aguado, Martínez, & Martín, 2013 ).

Cyber-aggression can lead to significant negative consequences in adolescent development, not only in victims but also in perpetrators. In victims, it can lead to depressive symptoms and suicidal ideation in the most severe cases ( Arce, Velasco, Novo, & Fariña, 2014; Gini & Espelage, 2014 ). In perpetrators, their behaviour can be reinforced through peers, favouring a more general pattern of antisocial behaviour. Previous studies suggest that cyber-aggression forms part of a larger pattern of behaviour characterized by low empathy ( Brewer & Kerslake, 2015 ), deficits of self-control ( Vazsonyi, Machackova, Sevcikova, Smahel, & Cerna, 2012 ), and the emergence of other antisocial behaviours in the real world, including theft ( Hemphill, Kotevski, & Heerde, 2015 ) or aggressions at school ( Baldry, Farrington, & Sorrentino, 2015 ).

In addition to these psychological and behavioural consequences, cyber-aggression may also involve legal consequences. Many activities that define cyber-aggression constitute criminal offenses in most countries. In Spain, the Penal Code classifies the following behaviours as crimes: threats, coercion, slander, deception over the phone or internet of a minor less than 16 years old to obtain pornographic material in which a minor appears, or crimes against privacy (for example, seizing a victim's personal messages without his/her consent, accessing private or intimate data or information and sharing it, or disclosing images of another person against his/her will that were obtained with his/her consent on a personal level). According to the Organic Law 5/2000, January 12, regulating the criminal liability of minors, if a criminal act has been committed by a minor between 14 and 18 years old, corrective measures aimed at rehabilitation (such as providing community service activities, attending day care centres or institutionalisation in re-education centres) should apply instead of adult penalties (such as imprisonment or fines). Offending minors under age 14 may not be held criminally liable, and the Administration, through Social Services, is to promote measures designed to control, re-educate, and protect in cases of familial neglect. The minor's family will assume civil liability that may arise from the committed act. However, the vast majority of cyber-aggression in adolescence may have an educational solution in family and school environments before it becomes a criminal matter ( Contreras, Molina, & Cano, 2011; Cutrín, Gómez-Fraguela, & Luengo, 2015 ).

Thus, it is important to understand this new type of aggression as well as to identify these types of incidents early to minimise their psychological and legal consequences. This process requires appropriate evaluation tools. Because cyber-aggression occurs in a virtual and often private environment, detecting it via direct observation or consultation with teachers or family members can be difficult. Even interviews or focus groups with adolescents can be fruitless because participants lack awareness of the incidents or harbour shame or fear of reporting or acknowledging particular cases of cyber-aggression, particularly those of a serious nature. Therefore, applying anonymous self-reports in which adolescents are asked to what extent they conduct or suffer from cyber-aggression may be more appropriate. Furthermore, the self-reports are more manageable as a screening measure because they can assess many people quickly; they allow coding and analysis of the obtained information more easily than other techniques, such as observation, interviews, or focus groups; and, applied collectively and anonymously, they favour honesty from the respondent.

For self-reports to be useful, their measures should be valid and reliable. In recent years, some self-reports have been published that are designed to evaluate how frequently the respondent is the perpetrator via mobile phone or the Internet and are validated with adolescents. Those validated in Spain are very rare, and the items related to cyber-aggression constitute only one factor. Both the European Cyberbullying Intervention Project Questionnaire (ECIPQ) by Del Rey et al. (2015) and the Cyberbullying Questionnaire (CBQ) by Gámez-Guadix, Villa-George, and Calvete (2014) include two factors (cyber-aggression and cyber-victimization), and the Cyberbullying test ( Garaigordobil, 2013 ) includes three (victimization, aggression, and observation). This coincides with other validated self-reports at the international level: the Cyberbullying Scale (CS) by Menesini, Nocentini, and Calussi (2011) includes two factors (cyber-aggression and cyber-victimization); the Virtual Aggression among Students subscale by Jiménez, Castillo, and Cisternas (2012) includes two factors (“conducted or observed virtual aggressions” and “victimisation through virtual aggression”); the Multidimensional Peer Bullying Scale (MPVS-RB) by Betts, Houston, and Steer (2015) measures a factor consisting of cyber-aggression behaviours and the rest relating to offline aggression; and the cyber-aggression scale of the Revised Cyber Bullying Inventory (RCBI) by Topcu and Erdur-Baker (2010) has a unifactorial structure.

Nevertheless, from a practical point of view, it may be interesting to be able to measure different types of cyber-aggression without needing to provide the score for each of the many observable indicators (items) that these scales evaluate. Additionally, the possibility that adolescents lump together responses on items that represent different types of cyber-aggression also has theoretical interest, as it would offer empirical evidence for the definition and delimitation of the construct. None of the reviewed questionnaires that have been validated in Spain offer a multifactorial structure, but examples do exist beyond Spain. These include the E-Bullying Scale (E-BS) by Lam and Li (2013) , composed of two factors (mild and serious); the cyber-aggression scale of the Cyber Victim and Bullying Scale (CVBS) by Çetin, Yaman, and Peker (2011) , which includes three factors (cyber verbal bullying, hiding identity, and cyber forgery); or the Florence CyberBullying-CyberVictimization Scales (FCBVSs) by Palladino, Nocentini, and Menesini (2015) , in which the cyber-aggression scale is composed of four factors (written-verbal, visual, impersonation, and exclusion). None of these three questionnaires have been validated in Spain.

Therefore, this study attempts to construct a self-report scale to measure cyber-aggression in adolescents from a multifactorial model. Among the different reviewed factorial structures, the classification based on the level of severity ( Lam & Li, 2013 ) could be subjective and debatable because each person could experience the severity of the incident differently. As such, using the type of aggression conducted as the criterion is preferable. In this regard, the proposal by Nocentini et al. (2010) has been taken as reference, supported empirically with validation from the FCBVS scales with Italian adolescents ( Palladino et al., 2015).

Therefore, this study aims to design a new self-report directed toward adolescents to evaluate the extent that the respondent conducts aggressions through a mobile phone or the internet (the Cyber-Aggression Questionnaire for Adolescents ; CYBA) and to validate it in a sample of adolescents from Asturias, Spain. Two subordinate objectives summarise this overall research objective. The first is to identify the factorial structure that best represents the obtained data with the application of the questionnaire. A multifactorial model, derived from the model proposed by Nocentini et al. (2010) , is expected to present a good fit to the data and adequate discriminant validity, and it would likely generate reliable scores in terms of internal consistency of the factors and representativeness of each item regarding the construct in which it is included. The second specific objective is to analyse the criterion validity of the test. The scores on the factors of the CYBA are expected to correlate positively with aggressions at school and impulsivity, and negatively with empathy, because previous research suggests that these three variables correlate this way with cyber-aggression.

Method Participants

The responses of 3,148 CSE students, from 12 to 18 years old ( M = 14.01, SD = 1.39), are analysed. Of the sample, 48.1% are girls. Sex is distributed homogeneously across the different ages of the sample, χ 2(6) = 11.695, p = .069, Cramer's V = .061. Some 95.1% of the evaluated adolescents have their own mobile phones, 84.1% surf the internet in their free time for non-school tasks, 93.6% use instant messaging programs (Messenger, WhatsApp, or others), and 77.9% participate in social networks (Tuenti, Facebook, or others) in their free time.

The participants evaluated attended 19 schools, selected using stratified random sampling from all publicly funded schools in which CSE is taught in Asturias. To select the sample, the population of schools was divided according to their ownership (public or state-subsidised private) and, in each stratum, a number of schools proportional to the population was selected. As a result, 11 public and 8 state-subsidised private schools were selected. The selected schools are in predominantly urban areas where residents are in the middle socioeconomic class. All groups were evaluated in each selected school. A total of 28.2% of the participants are in their 1 st year, 25.3% are in their 2nd year, 24.4% are in their 3rd year, and 22.1% are in their 4 th year of CSE.

Measures

Cyber-Aggression Questionnaire for Adolescents (CYBA). The CYBA questionnaire designed and tested in this study is a self-report composed of 19 items, each of which describes an aggression conducted through a mobile phone or the internet (see Appendix). The evaluated participant should indicate the frequency that he/she has conducted the different actions in the last three months, using a Likert-type scale with four options (1 = never, 2 = rarely, 3 = often, and 4 = always).

Aggression at school . A self-report scale that was developed ad hoc was used; the scale was based on a selection and adaptation of six items from the Aggression Questionnaire by Álvarez-García, Menéndez, González-Castro, and Rodríguez (2012) . This study uses a scale composed of the following items related to the frequency with which the participant says he/she conducts aggressions in the physical school environment: “No he dejado participar en mi grupo a algún compañero, durante alguna actividad de recreo o de educación física” [“I have not let a classmate participate in my group during the break or physical education”], “No he dejado participar en mi grupo a algún compañero en alguna actividad de clase” [“I have not let a classmate participate in my group in a class activity”], “Me he reído y burlado de algún compañero/alguna compañera” [“I have laughed at or teased a classmate”], “He hablado mal de algún compañero/alguna compañera a sus espaldas” [“I have talked badly about a classmate behind his/her back”], “He insultado a la cara a algún compañero/alguna compañera” [“I have insulted a classmate to his/her face”] and “He pegado a algún alumno/alguna alumna del centro, dentro o a la salida del recinto escolar” [“I have hit a student of the school, within or outside the school grounds”]. The response is a Likert-type list with four options (1 = never, 2 = rarely, 3 = often, 4 = always ). The internal consistency of the scores obtained with this scale in this study sample is high (Raykov's rho = .917).

Impulsivity . The self-report scale is developed ad hoc from some of the criteria for impulsivity proposed by the DSM-5 for the diagnosis of ADHD ( American Psychiatric Association, 2013 ). The scale used is composed of five items, referring to the extent that the respondent considers it true that he/she conducts certain behaviours of an impulsive nature: “En clase o en juegos, a menudo me cuesta esperar turno, por lo que me cuelo o interrumpo” [“In class or when playing games, it is often difficult for me to wait my turn, so I jump in or interrupt”], “A menudo contesto antes de que se haya completado la pregunta” [“I often answer before the question has finished”], “A menudo digo lo que me viene a la cabeza, sin pensar primero sus consecuencias o si es oportuno para la conversación” [“I often say what comes to mind without thinking first of the consequences or whether it is appropriate for the conversation”], “A menudo hago cosas sin pensar en las consecuencias” [“I often do things without thinking of the consequences”] and “Habitualmente me resulta difícil esperar turno, por lo que me adelanto a hablar cuando no me corresponde o interrumpo a quien está hablando” [“Usually, I find it difficult to wait my turn, so I jump in to speak when it is not my turn or I interrupt the person talking”]. The response is a Likert-type list with four options (1 = totally false, 2 = somewhat false, 3 = somewhat true, 4 = totally true ). The internal consistency of the scores obtained with the scale in this study sample is high (Raykov's rho = .865).

Empathy . The self-report scale was developed ad hoc and composed of five items referring to the extent that the respondent considers it to be true that he/she is capable of identifying with others and sharing their feelings: “Siento las desgracias de los demás” [“I feel the misfortunes of others”], “Si se burlan de un compañero, me siento mal pensando en lo mal que lo está pasando” [“If a classmate is teased, I feel bad thinking about what is happening to him/her”], “Soy paciente con las personas que hacen las cosas peor que yo” [“I am patient with people who do things worse than I do”], “Cuando veo que un/a amigo/a está triste, yo también me entristezco” [“When I see that a friend is sad, I also become sad”] and “Me alegro cuando le pasa algo bueno a un conocido” [“I am happy when something good happens to someone I know”]. The response is a Likert-type list with four options (1 = totally false, 2 = somewhat false, 3 = somewhat true, 4 = totally true ). The internal consistency of the scores obtained with the scale in this study sample is adequate (Raykov's rho = .755).

Procedure

First, the CYBA questionnaire was designed. The wording of the items came from a theoretical model of six factors or types of cyber-aggression ( Table 1 ), based on the model of four factors (written-verbal, visual, impersonation, and exclusion) proposed by Nocentini et al. (2010) . The authors produced the written items. Once the questionnaire was designed and the schools selected, permission was requested from the schools’ respective leadership teams to implement the questionnaire. Each leadership team was informed of the objectives and procedures of the study, its voluntary and anonymous nature, and the confidential treatment of the results. Once the school agreed to participate, informed consent was requested from the parents or guardians of the students, given their status as minors. The questionnaire was applied in all the schools during the second and third quarter of the 2014-2015 school year. Before answering the questionnaire, students were also informed of the anonymous, confidential, and voluntary nature of their participation. In general, students had 20 minutes to answer the questionnaires, although the timing was flexible depending on the age and characteristics of the respondents. The research team administered the test to all groups from each school during school hours.

Table 1

A priori Models Proposed for the Analysis of the Dimensionality of the Cyber-Aggression Questionnaire for Adolescents (CYBA).

Model Factors Items
Second-order  First-order 
1FM  Cyber-aggression  All (1-19) 
6FM - Impersonation  1, 12 and 18 
Visual - Sexual  2, 9 and 14 
Visual - Teasing/Happy Slapping  3, 6, 10 and 15 
Verbal - Written  8, 11 and 19 
Verbal - Oral  5, 7 and 17 
Online exclusion  4, 13 and 16 
6FM2 Cyber-aggression Impersonation  1, 12 and 18 
Visual - Sexual  2, 9 and 14 
Visual - Teasing/Happy slapping  3, 6, 10 and 15 
Verbal - Written  8, 11 and 19 
Verbal - Oral  5, 7 and 17 
Online exclusion  4, 13 and 16 

Note. 1FM = one-factor model, 6FM = six-factor model, 6FM2 = hierarchical model with six first-order factors and one second-order factor.

Data analysis

First, in the data matrix, the presence of values that were missing or outside the scale was analysed. All items were answered in an acceptable way by more than 90% of the respondents, so none were removed from the final analysis of the test on that basis. Of the 3,233 students who responded to the CYBA, 85 responded in an acceptable way to less than 90% of the items, so they were removed from the data matrix. Once they were removed, the number of missing values in the CYBA matrix was reduced to 122 (0.2% of the total).

Second, the statistical program EQS 6.2 ( Bentler, 2014 ) was used to analyse the normality of the data and the dimensionality of the test. Normality was assessed by analysing the multivariate kurtosis. Dimensionality was examined using confirmatory factor analysis. The multivariate kurtosis pointed to the non-normality of the obtained data (Mardia coefficient = 2571.16). Consequently, Arbitrary Generalized Least Squares (AGLS) was used as estimation method. Given the ordinal nature of the scale, the analyses were completed based on the matrix of polychoric correlations ( Lila, Oliver, Catalá-Miñana, Galiana, & Gracia, 2014 ). Missing values were treated by computing the matrix of covariances using the pairwise method because AGLS requires large sample sizes and to avoid removing more subjects and be able to use all available data.

To determine the degree of fit of the tested models, the chi-square ( χ 2)/degrees of freedom (df ) ratio, the Comparative Fit Index (CFI), the Bentler-Bonett Non-normed Fit Index (NNFI), the Standardized Root Mean Square Residual (SRMR), the Root Mean Square Error of Approximation (RMSEA), and the Akaike Information Criterion (AIC) were utilized. Usually, the fit is considered good when CFI ≥ .95, NNFI ≥ .95, SRMR ≤.08, and RMSEA ≤ .06 ( Hu & Bentler, 1999), and χ 2/df < 3 ( Ruiz, Pardo, & San Martín, 2010 ). The AIC allows models to be compared, with those having lower values being preferable.

The discriminant validity of the model with best fit was studied, analysing the correlation between its factors. Very high correlations ( r ≥ .85) warn of possible collinearity or redundancy between factors and, therefore, poor discriminant validity ( Brown, 2015 ). Given that a very high correlation was indeed found between some factors, post hoc re-specifications in the model were completed.

The indices of fit and the correlations between factors of the two re-specified models were analysed. Once the model with the most appropriate discriminant validity and fit to the data was identified from those tested, the factor weights and standard errors of each item for that model were found. Usually, factor weights greater than .30 are considered acceptable ( Izquierdo, Olea, & Abad, 2014 ).

Subsequently, the reliability of the scores obtained in the test was analysed. The reliability of each subscale was analysed in terms of internal consistency, finding its respective Raykov's rho from the matrix of polychoric correlations. This coefficient is considered as an alternative to the weaknesses of the Cronbach's alpha coefficient, highlighted by authors such as Sijtsma (2009) . In general, the Raykov's rho coefficient is considered acceptable if it has a value above .70, high if it is greater than .80, and very high if it is greater than .90. Additionally, Squared Multiple Correlations of the scores for each item were found, which was an indicator of the proportion of variance on the item explained by the latent variable and, therefore, of the item's reliability to measure that variable ( Bollen, 1989).

Finally, the statistical software SPSS 21 ( IBM Corp., 2012 ) was used to analyse the criterion validity. For this, the Spearman correlation coefficient was calculated between the score on each of the CYBA's three factors and three external criteria about which previous evidence demonstrates their association with cyber-aggression: aggression at school, impulsivity, and empathy.

Results Analysis of the Initial Theoretical Model

Factorial validity . At first, the indices of fit of the initial theoretical model, composed of significantly interrelated six factors or types of cyber-aggression (6FM), were obtained and compared with those from two additional plausible models from a theoretical perspective ( Table 1 ). On the one hand, they were compared with a unifactorial model (1FM), whereas all the items from the test could measure only one construct (cyber-aggression). On the other hand, they were compared with a hierarchical model of six first-order factors–the same as 6FM–and one second-order factor–cyber-aggression–(6FM2), whereas each item could be explained by a first-order factor and each first-order factor by the common construct cyber-aggression. As presented in Table 2 , the 6FM presents good indices of fit to the obtained data, the best from among the three models initially tested.

Table 2

Goodness-of-fit Indexes of the Three a priori Tested Models for the CYBA, with the Total Sample ( N = 3148).

Model  χ2  df  p  χ 2 /df  CFI  NNFI  SRMR  RMSEA [CI 90%]  AIC 
1FM  956.45  152  <.001  6.29  .956  .950  .117  .041 [.039, .043]  652.45 
6FM  395.49  137  <.001  2.89  .986  .982  .081  .024 [.022, .027]  121.49 
6FM2  604.16  146  <.001  4.14  .975  .971  .084  .032 [.029, .034]  312.16 

Note. 1FM = one-factor model, 6FM = six-factor model, 6FM2 = hierarchical model with six first-order factors and one second-order factor.

χ2 = chi-square, df = degrees of freedom, p = probability level, CFI = Comparative Fit Index, NNFI = Non-Normed Fit Index, SRMR = Standardized Root Mean Square Residual, RMSEA = Root Mean Square Error of Approximation, CI = Confidence interval, AIC = Akaike Information Criterion.

Discriminant validity . All the 6FM factors have positive and statistically significant correlations among themselves ( Table 3 ). Some of these correlations are very high ( r > .85): the factor Visual Cyber-Aggression Teasing/Happy Slapping is very highly correlated with Visual-Sexual Cyber-Aggression, Impersonation, and Online Exclusion; and the factor Online Exclusion is very highly correlated with Verbal-Written Cyber-Aggression and Verbal-Oral Cyber-Aggression, in addition to its relationship with Visual Cyber-Aggression Teasing/Happy Slapping referenced above.

Table 3

Correlation Matrix between the Components of the Six-factor Model ( N = 3148).

  Impersonation  Visual-Sexual  Visual - Teasing/Happy slapping  Verbal-Written  Verbal-Oral  Online exclusion 
Impersonation  1.00           
Visual-Sexual  .80  1.00         
Visual - Teasing/Happy Slapping  .90  .97  1.00       
Verbal-Written  .77  .78  .79  1.00     
Verbal-Oral  .70  .77  .77  .82  1.00   
Online Exclusion  .72  .83  .86  .95  .90  1.00 
Post-hoc Modifications

Due to the high correlation between some factors, two modified models of the initial 6FM were designed and tested: a model with five factors (5FM) and another with three (3FM). In both, items referring to Visual Cyber-Aggression Teasing/Happy Slapping were removed from the model and are now considered additional indicators included in the questionnaire but outside the model. Thus, the 5FM is composed of the same factors as the 6FM, with the exception of the factor Visual Cyber-Aggression Teasing/Happy Slapping. Furthermore, in the 3FM, the items initially included in the factors Verbal-Written Cyber-Aggression, Verbal-Oral Cyber-Aggression, and Online Exclusion now constitute only one factor, called Verbal Cyber-Aggression and Exclusion ( Figure 1). As indicated in Table 4 , both models present good indices of fit. The 5FM presents the best fit of both. Nonetheless, the correlation between the factor Online Exclusion and the two factors referring to Verbal Cyber-Aggression (Oral and Written), remain very high ( r > .85). In the case of the 3FM, no correlations exist between factors greater than .85 ( Figure 1 ). That is why this model (3FM) is considered the one that best represents the obtained data among those tested.

Figure 1

Factorial Structure of the CYBA.

Note. I = impersonation, V-S = visual-sexual cyber-aggression, V-E = verbal cyber-aggression and exclusion.

Table 4

Goodness of Fit Indexes of the Two post hoc Models, with the Total Sample (N = 3148).

Model  χ2  df  p  χ 2 /df  CFI  NNFI  SRMR  RMSEA [CI 90%]  AIC 
3FM  323.09  87  < .001  3.71  .967  .960  .097  .029 [.026, .033]  149.09 
5FM  210.87  80  < .001  2.64  .982  .976  .070  .023 [.019, .027]  50.87 

Note. 3FM = three-factor model, 5FM = five-factor model.

χ2 = chi-square, df = degrees of freedom, p = probability level, CFI = Comparative Fit Index, NNFI = Non-Normed Fit Index, SRMR = Standardized Root Mean Square Residual, RMSEA = Root Mean Square Error of Approximation, CI = Confidence interval, AIC = Akaike Information Criterion.

Analysis of the Three-factor Model

Factor weights . In the 3FM, the factor weights of each item in its factor are high ( Figure 1 ), with standardised regression coefficients equal to or greater than .70 on all items except 13 (λ = .69).

Reliability . The reliability of scores in each subscale, evaluated in terms of internal consistency, could be considered high, and for each item the reliability scores could be considered moderate or high ( Table 5 ). The reliability coefficient Raykov's rho of each factor is greater than .80. The proportion of variance of the item explained by the latent variable ( R 2 ) is between 47% and 82%.

Table 5

Reliability of Each Item and Factor of the CYBA ( N = 3148).

Factor  Item  ρ  R 2 
Impersonation   .866   
  .772 
12    .819 
18    .757 
Visual-Sexual cyber-aggression   .809   
  .790 
  .497 
14    .760 
Verbal cyber-aggression and exclusion   .932   
  .483 
  .504 
  .711 
  .712 
11    .615 
13    .471 
16    .626 
17    .730 
19    .635 

Note. ρ = Raykov's rho coefficient, R 2 = squared multiple correlation.

Criterion validity . The scores of the three factors of the CYBA have a statistically significant correlation with the three external criteria analysed. The correlation is positive with the frequency of aggression conducted in the physical school environment and impulsive behaviours, reported by the evaluated adolescents themselves; the correlation is negative with empathy ( Table 6).

Table 6

Spearman's Rank Correlation Coefficients between the Score on Each Factor of the CYBA and the Scores on the Aggression at School, Impulsivity, and Empathy Scales ( N = 3148).

  Aggression at school  Impulsivity  Empathy 
Impersonation  .220 *   .140 *   -.117 *  
Visual-Sexual Cyber-aggression  .215 *   .136 *   -.089 *  
Verbal Cyber-aggression and Exclusion  .553 *   .362 *   -.182 *  
*

p < .01.

Discussion

This study aimed to design a new self-report directed toward adolescents to evaluate the extent that the respondent conducts aggressions through a mobile phone or the internet (the Cyber-Aggression Questionnaire for Adolescents - CYBA) and to validate it in a sample of adolescents from Asturias (Spain). The results obtained indicate that the CYBA presents adequate statistical assurance for its use for its designed purpose.

The first specific objective of this study was to identify the factorial structure that best represents the data obtained through implementation of the questionnaire. As was initially hypothesised, the model that best fits the data is multifactorial. However, the initially proposed six-factor model presented a very high correlation between the scores of some of its factors. The tested model, which best represents the obtained data with adequate discriminant validity, is composed of three factors (Impersonation, Visual-Sexual Cybervictimization, and Verbal Cybervictimization and Exclusion), in which the four items of Visual Cybervictimization Teasing/Happy Slapping are extracted from the model and remain complementary indicators. The previously validated questionnaires in Spain had factor structures in which cyber-aggression constituted only one factor ( Del Rey et al., 2015; Gámez-Guadix et al., 2014; Garaigordobil, 2013 ). Therefore, an overall score on cyber-aggression could be obtained using these, as well as a specific score on each indicator (item) separately. However, the results obtained in this study offer empirical support for the idea that the cyber-aggression construct consists of significantly interrelated but distinct types, and they justify the collection of scores for each of those types as well as for each observable indicator (item). Other questionnaires published in the international sphere also involve multifactorial structures (CVBS, Çetin et al., 2011; E-BS, Lam & Li, 2013; FCBVSs, Palladino et al., 2015 ). However, the CYBA differs from them in the types of cyber-aggression that it evaluates and in the observable indicators that define them. Similar to the CYBA, these three questionnaires include items related to verbal cyber-aggression and two of them also include items related to impersonation. However, only the CYBA offers a specific score on visual cyber-aggression of a sexual nature, differentiated from other indicators of visual cyber-aggression linked to teasing and physical aggression. Furthermore, only the CYBA and the scale on cyber-aggression by Palladino et al. (2015) include items related to online exclusion. The results obtained in this study, unlike those obtained by Palladino et al. (2015) , do not justify the consideration of online exclusion as a type of cyber-aggression distinct from verbal cyber-aggression.

The reliability analyses support the relevance of the three-factor model considered later. The indices of internal consistency of the scores obtained for each factor or type of cyber-aggression suggest that the items or observable indicators that shape them measure the same latent variable without being redundant. Furthermore, the squared multiple correlation of each item demonstrates that all indicators utilized are relevant for measuring the type of cyber-aggression included.

The second specific objective of this study has been to analyse the criterion validity of the test. As had been hypothesised, the scores obtained with the CYBA have a statistically significant correlation with three external variables, which, according to previous evidence, correlate with cyber-aggression in adolescence. The score on the CYBA correlates positively with the frequency with which the adolescent reports having attacked his/her classmates at their school and with the frequency with which he/she admits conducting impulsive behaviours, and it correlates negatively with empathy. These results are consistent with those presented in the recent review paper by Baldry et al. (2015) , which, similar to this study, concludes that the variable most associated with cyber-aggression in adolescence is being a perpetrator in the physical school context.

This study has various theoretical and practical implications. From the theoretical point of view, it provides empirical evidence that contributes to the conceptual delimitation of cyber-aggression in adolescence, the distinction of types, and the identification of representative observable indicators of each. The results obtained support the hypothesis of the multifactorial nature of cyber-aggression in adolescence and the suitability of the observable indicators included in the CYBA. From a practical standpoint, it offers professionals in the field of psychology a tool to evaluate cyber-aggression that is concise, simple to implement, code, and analyse, economical in terms of time and cost, compared to other evaluation methods, and with adequate metric properties. In the school context, CYBA offers a screening measure to detect cyber-aggression among adolescents. Early identification, accompanied by educational treatment, can prevent it from becoming a criminal problem. In the context of juvenile justice, the CYBA helps analyse the evolution of cyber-aggression (if it persists or stops in adulthood, if it evolves into another type of criminal activity) and its major risk factors to design preventive interventions, both at the school and community levels, and to guide the treatment of perpetrated cases. The relationship observed in this study and in previous studies ( Baldry et al., 2015 ) between cyber-aggression and the three analysed external criteria suggests that including the development of empathy and behavioural self-control as well as the prevention of aggression among students in the educational context benefits the prevention and treatment of cyber-aggression in adolescence.

Therefore, this study proposes a contribution to the study of cyber-aggression in adolescence. However, it also has some limitations, three of which must be highlighted. First, because the CYBA is a self-reporting tool, the results could be affected by response biases, such as falsification or social desirability ( Jiménez, Sánchez, & Tobón, 2009 ). In the future, CYBA could include a scale for truthfulness. Second, aggression at school, impulsivity, and empathy have been evaluated using self reports developed ad hoc . Although its metric properties have been adequate in the sample analysed in this study, it would have been preferable to use measures whose scores had already been validated in the Spanish adolescent population ( Torregrosa, Inglés, Estévez-López, Musitu, & García-Fernández, 2011 ). Finally, the CYBA has been tested with a large, randomly selected sample, but it was taken from a particular population and limited to certain ages and geographical areas. Therefore, any generalisation of this study's findings to other ages or regions should be done with caution ( Muñiz, Elosua, & Hambleton, 2013 ). Looking ahead, it would be of interest to validate the test in other age groups and cultural contexts.

Conflict of Interest

The authors of this article declare no conflict of interest.

References
Álvarez-García et al., 2015
D. Álvarez-García
A. Dobarro
J.C. Núñez
Validez y fiabilidad del Cuestionario de cibervictimización en estudiantes de Secundaria [Validity and reliability of the Cybervictimization questionnaire in secondary education students]
Aula Abierta
43
1
2015
32-8
Álvarez-García et al., 2012
D. Álvarez-García
S. Menéndez
P. González-Castro
C. Rodríguez
Hiperactividad-impulsividad y déficit de atención como predictores de participación en situaciones de violencia escolar [Hyperactivity-impulsivity and attention deficit as predictors of involvement in school violence situations]
International Journal of Psychology and Psychological Therapy
12
2012
185-202
American Psychiatric Association, 2013
American Psychiatric Association
Diagnostic and Statistical Manual of Mental Disorders (DSM-5)
2013
Arce et al., 2014
R. Arce
J. Velasco
M. Novo
F. Fariña
Elaboración y validación de una escala para la evaluación del acoso escolar [Development and validation of a scale to assess bullying]
Revista Iberoamericana de Psicología y Salud
5
1
2014
71-104
Baldry et al., 2015
A.C. Baldry
D.P. Farrington
A. Sorrentino
Am I at risk of cyberbullying? A narrative review and conceptual framework for research on risk of cyberbullying and cybervictimization: The risk and needs assessment approach
Aggression and Violent Behavior
23
2015
36-51
Bentler, 2014
P.M. Bentler
EQS 6. 2 for Windows (Build 107) - Structural Equation Modeling Software
2014
Betts et al., 2015
L.R. Betts
J.E. Houston
O.L. Steer
Development of the Multidimensional Peer Victimization Scale–Revised (MPVS-R) and the Multidimensional Peer Bullying Scale (MPVS-RB)
The Journal of Genetic Psychology
176
2015
93-109
Bollen, 1989
K.A. Bollen
Structural Equations with Latent Variables
1989
Brewer and Kerslake, 2015
G. Brewer
J. Kerslake
Cyberbullying, self-esteem, empathy and loneliness
Computers in Human Behavior
48
2015
255-60
Brown, 2015
T.A. Brown
Confirmatory Factor Analysis for Applied Research
(2nd Edition)
2015
Çetin et al., 2011
B. Çetin
E. Yaman
A. Peker
Cyber victim and bullying scale: A study of validity and reliability
Computers & Education
57
2011
2261-71
Contreras et al., 2011
L. Contreras
V. Molina
M.C. Cano
In search of psychosocial variables linked to the recidivism in young offenders
The European Journal of Psychology Applied to Legal Context
3
2011
77-88
Corcoran et al., 2015
L. Corcoran
C. Mc Guckin
G. Prentice
Cyberbullying or Cyber Aggression? A Review of Existing Definitions of Cyber-Based Peer-to-Peer Aggression
Societies
5
2015
245-55
Cutrín et al., 2015
O. Cutrín
J.A. Gómez-Fraguela
M.A. Luengo
Peer-group mediation in the relationship between family and juvenile antisocial behavior
The European Journal of Psychology Applied to Legal Context
7
2015
59-65
Del Rey et al., 2015
Del Rey, R., Casas, J. A., Ortega-Ruiz, R., Schultze-Krumbholz, A., Scheithauer, H., Smith, P., … Plichta, P. (2015). Structural validation and cross-cultural robustness of the European Cyberbullying Intervention Project Questionnaire. Computers in Human Behavior, 50 , 141-147. doi: 10.1016/j.chb.2015.03.065.
Díaz-Aguado et al., 2013
M.J. Díaz-Aguado
R. Martínez
J. Martín
El acoso entre adolescentes en España. Prevalencia, papeles adoptados por todo el grupo y características a las que atribuyen la victimización [Bullying among adolescents in Spain. Prevalence, participants¿ roles and characteristics attributable to victimization by victims and aggressors]
Revista de Educación
362
2013
348-79
Findahl, 2014
Findahl, O. (2014). Svenskarna och internet 2014 [The Swedes and the internet 2014]. Stockholm: Stiftelsen för internetinfrastruktur. Retrieved from https://www.iis.se/docs/SOI2014.pdf .
Gámez-Guadix et al., 2014
M. Gámez-Guadix
F. Villa-George
E. Calvete
Psychometric properties of the Cyberbullying Questionnaire (CBQ) among Mexican adolescents
Violence and Victims
29
2014
232-47
Garaigordobil, 2013
Garaigordobil, M. (2013). Cyberbullying. Screening de acoso entre iguales [Cyberbullying: Screening of peer harassment]. Madrid: TEA Ediciones.
Gini and Espelage, 2014
G. Gini
D.L. Espelage
Peer Victimization, Cyberbullying, and Suicide Risk in Children and Adolescents
JAMA
312
2014
545-6
Hemphill et al., 2015
S.A. Hemphill
A. Kotevski
J.A. Heerde
Longitudinal associations between cyber-bullying perpetration and victimization and problem behavior and mental health problems in young Australians
International Journal of Public Health
60
2015
227-37
Hu and Bentler, 1999
L. Hu
P.M. Bentler
Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives
Structural Equation Modeling: A Multidisciplinary Journal
6
1999
1-55
IBM, 2012
IBM Corp. (2012). SPSS Statistics for Windows , Version 21.0. Armonk, NY: IBM Corp.
Instituto Nacional de Estadística, 2014
Instituto Nacional de Estadística (2014). Encuesta sobre Equipamiento y Uso de Tecnologías de Información y Comunicación en los hogares 2014 [Survey on the Equipment and Use of Information and Communication Technologies in Households. Year 2014]. Retrived from h t t p & # 5 8 ; & # 4 7 ; & # 4 7 ; w w w & # 4 6 ; i n e & # 4 6 ; e s & # 4 7 ; j a x i & # 4 7 ; m e n u & # 4 6 ; d o & # 6 3 ; t y p e & # 6 1 ; p c a x i s & # 3 8 ; path=/t25/p450/base_2011/a2014/&file=pcaxis .
Izquierdo et al., 2014
I. Izquierdo
J. Olea
F.J. Abad
Exploratory factor analysis in validation studies: Uses and recommendations
Psicothema
26
2014
395-400
Jiménez et al., 2012
A.E. Jiménez
V.D. Castillo
L.C. Cisternas
Validación de la escala de agresión entre pares, y subescala de agresión virtual en escolares chilenos [Validation of the Aggression Among Peers Scale and Virtual Aggression Sub-scale with Chilean students]
Revista Latinoamericana de Ciencias Sociales, Niñez y Juventud
10
2012
825-40
Jiménez et al., 2009
F. Jiménez
G. Sánchez
C. Tobón
A social desirability scale for the MMPI-2. Which of the two: Wiggins (WSD) or Edwards (ESD)?
The European Journal of Psychology Applied to Legal Context
1
2009
147-63
Lam and Li, 2013
L.T. Lam
Y. Li
The validation of the E-Victimisation Scale (E-VS) and the E-Bullying Scale (E-BS) for adolescents
Computers in Human Behavior
29
2013
3-7
Li et al., 2012
Q. Li
D. Cross
P.K. Smith
Cyberbullying in the Global Playground: Research from International Perspectives
2012
Lila et al., 2014
M. Lila
A. Oliver
A. Catalá-Miñana
L. Galiana
E. Gracia
The Intimate Partner Violence Responsibility Attribution Scale (IPVRAS)
The European Journal of Psychology Applied to Legal Context
6
2014
29-36
Menesini et al., 2011
E. Menesini
A. Nocentini
P. Calussi
The measurement of cyberbullying: dimensional structure and relative item severity and discrimination
Cyberpsychology, Behavior, and Social Networking
14
2011
267-74
Muñiz et al., 2013
J. Muñiz
P. Elosua
R.K. Hambleton
International Test Commission Guidelines for test translation and adaptation: Second edition
Psicothema
25
2013
151-7
Nocentini et al., 2010
A. Nocentini
J. Calmaestra
A. Schultze-Krumbholz
H. Scheithauer
R. Ortega
E. Menesini
Cyberbullying: Labels, behaviours and definition in three European countries
Australian Journal of Guidance & Counselling
20
2010
129-42
O’Neill and Dinh, 2015
B. O’Neill
T. Dinh
Net Children Go Mobile: Full findings from Ireland
2015
Palladino et al., 2015
B.E. Palladino
A. Nocentini
E. Menesini
Psychometric Properties of the Florence CyberBullying-CyberVictimization Scales
Cyberpsychology, Behavior, and Social Networking
18
2015
112-9
Pouria and Brousse, 2011
Pouria, S., & Brousse, M. (2011). Enfants et Internet. Barométre 2011 del’ opération nationale de sensibilisation: Génération numérique [Children and Internet. Barometer 2011 of the national sensitization campaign: Digital Generation]. Montreuil: CALYSTO & La Voix de L’enfant. Retrieved from h t t p & # 5 8 ; & # 4 7 ; & # 4 7 ; w w w & # 4 6 ; g e n e r a t i o n n u m e r i q u e & # 4 6 ; c o m & # 4 7 ; w p - c o n t e n t & # 4 7 ; u p l oads/2012/02/Barometre-Calysto-F%C3%A9vrier-2012-BD.pdf .
Ruiz et al., 2010
M.A. Ruiz
A. Pardo
R. San Martín
Modelos de ecuaciones estructurales [Structural Equation Models]
Papeles del Psicólogo
31
2010
34-45
Sijtsma, 2009
K. Sijtsma
On the use, misuse, and the very limited usefulness of Cronbach's Alpha
Psychometrika
74
2009
107-20
Topcu and Erdur-Baker, 2010
Ç. Topcu
Ö. Erdur-Baker
The Revised Cyber Bullying Inventory (RCBI): validity and reliability studies
Procedia Social and Behavioral Sciences
5
2010
660-4
Torregrosa et al., 2011
M.S. Torregrosa
C.J. Inglés
E. Estévez-López
G. Musitu
J.M. García-Fernández
Evaluación de la conducta violenta en la adolescencia: Revisión de cuestionarios, inventarios y escalas en población española [Assessment of violent behaviour in adolescence: Review of questionnaires, inventories, and scales in Spanish population]
Aula Abierta
39
2011
37-50
Vazsonyi et al., 2012
A.T. Vazsonyi
H. Machackova
A. Sevcikova
D. Smahel
A. Cerna
Cyberbullying in context: Direct and indirect effects by low self-control across 25 European countries
European Journal of Developmental Psychology
9
2012
210-27

Copyright © 2019. Colegio Oficial de Psicólogos de Madrid

© Copyright 2019. Colegio Oficial de Psicólogos de Madrid Privacy PolicyCookies Policy

We use our own and third­party cookies. The data we compile is analysed to improve the website and to offer more personalized services. By continuing to browse, you are agreeing to our use of cookies. For more information, see our cookies policy

Aceptar