David Álvarez-García, José Carlos Núñez, Trinidad García and Alejandra Barreiro-Collazo
Department of Psychology, University of Oviedo, SpainReceived 14 de September de 2017, Accepted 15 de February de 2018
The objective of this article is to the predictive capacity of some individual, family, and community variables concerning the likelihood that a teenager will engage in aggressive behavior toward others using a mobile phone or the Internet, occasionally or intensely, controlling for the effect of potential confounding variables. To that end, the Cyber-Aggression Questionnaire for Adolescents (CYBA) as well as previously validated scales for the evaluation of the potential indicators d were applied to 3,059 adolescents 12 to 18 years of age (M = 14.01, SD = 1.39). The aforementioned scales sociodemographic variables (age and sex) and variables referring to the use of the Internet (social networks, instant messaging programs, and the Internet for non-school tasks), parental control (behavioral control, rules for the use of the Internet, Internet use monitoring, and affection and communication), personality (impulsivity and empathy), antisocial behavior (frequency of aggression toward others at school, antisocial behavior outside the school, and antisocial friendships), and frequency of cyber-victimization. Multivariate regression analyses highlight the predictive capacity of impulsivity, aggression at school, and cyber-victimization as risk factors of cyber-aggression. They also suggest the existence of indirect or even spurious relationships between some of the variables d and cyber-aggression. We discuss the practical implications of these results.
El objetivo de este artículo es analizar la capacidad predictiva de algunas variables individuales, familiares y comunitarias sobre la probabilidad de que un adolescente se involucre en comportamientos agresivos hacia otros mediante el empleo del teléfono móvil o Internet, de forma ocasional o severa, controlando el efecto de potenciales variables de confusión. Para ello, se aplicó el Cuestionario de Ciberagresión para Adolescentes (CYBA) a 3,059 adolescentes de 12 a 18 años de edad (M = 14.01, DT = 1.39), así como escalas previamente validadas para la evaluación de los potenciales indicadores analizados. Estas escalas incluyen variables sociodemográficas (edad y sexo) y variables referentes al uso de Internet (redes sociales, programas de mensajería instantánea e Internet para tareas no escolares), el control parental (control del comportamiento, reglas para el uso de Internet, la supervisión del uso de esta y afecto y comunicación), la personalidad (impulsividad y empatía), el comportamiento antisocial (frecuencia de agresión hacia otros en la escuela, comportamiento antisocial fuera de la escuela y amistades antisociales) y la frecuencia de la cibervictimización. Los análisis de regresión multivariante destacan la capacidad predictiva de la impulsividad, la agresión en la escuela y la cibervictimización como factores de riesgo de ciberagresión. También sugieren la existencia de relaciones indirectas o incluso espurias entre algunas de las variables analizadas y la ciberagresión. Se analizan las implicaciones prácticas de estos resultados.
Cyber-aggression, Predictors, Risk factors, Protective factors, Adolescence
Ciberagresión, Predictores, Factores de riesgo, Factores de protección, Adolescencia
Mobile phones and the Internet can be very enriching tools for adolescent development. However, if used to cause harm, they also have the potential to be very dangerous. The term cyber-aggression is typically used to refer to acts that are intentionally hurtful, offensive, or harmful to people or institutions through electronic communication devices (Corcoran, McGuckin, & Prentice, 2015). These acts may take several forms, including verbal aggressions (oral or written), the use of mocking or compromising images, impersonation, or online exclusion (Nocentini et al., 2010).
Cyber-aggression among adolescents is a matter that warrants attention due to its prevalence and effects. Preventing cyber-aggression among adolescents is of paramount importance. To that end, identifying its main predictors, including risk and protection factors, is essential. To date, the research published on the subject is limited. Nonetheless, it helps appreciate the relationship between cyber-aggression and certain individual, family, and community variables. Among the individual variables, sociodemographic variables, particularly age and sex, have been those most extensively studied. Regarding age, although studies differ in the sample age range, higher perpetration risks have been consistently found among older students (Ciucci & Baroncelli, 2014; Del Rey, Lazuras et al., 2016; Festl, Scharkow, & Quandt, 2015; Mishna, Khoury-Kassabri, Gadalla, & Daciuk, 2012). With regard to sex, the results are less clear. Although most studies conclude that boys tend to be more cyber-aggressive than girls (Álvarez-García, Barreiro-Collazo, & Núñez, 2017), a significant number of studies conclude that there are no differences and a few indicate that girls tend to engage in cyber-aggression more frequently than boys (Navarro, 2016).
Other individual variables analyzed as potential indicators of cyber-aggression are those related to Internet use. In this sense, it has been found that teenagers who use the Internet for more hours a day are more likely to be cyber-aggressors (Festl et al., 2015; Mishna et al., 2012; Rice et al., 2015) and, specifically, that the frequency of online communication and use of social networks is a risk factor for cyber-aggression (Lee & Shin, 2017; Sticca, Ruggieri, Alsaker, & Perren, 2013). Having been a victim of cyber-aggression increases the probability of becoming a cyber-aggressor as well (Beran, Mishna, McInroy, & Shariff, 2015; Festl et al., 2015; Kowalski, Giumetti, Schroeder, & Lattanner, 2014).
Other individual variables analyzed as potential predictors of cyber-aggression are personality variables, such as impulsivity and empathy. The results obtained by previously published studies suggest that impulsivity is a risk factor (Vazsonyi, Machackova, Sevcikova, Smahel, & Cerna, 2012) and empathy, a protective factor (Brewer & Kerslake, 2015; Del Rey, Lazuras et al., 2016) from becoming cyber-aggressors. In relation to these variables, the predictive capacities of exercising aggressive behaviors toward peers at school or other types of antisocial behavior outside school and becoming a cyber-aggressor have been analyzed. Both cross-sectional and longitudinal studies suggest that committing aggressive acts against peers in the educational center is an important risk factor for becoming an online aggressor (Festl et al., 2015; Fletcher et al., 2014; Sticca et al., 2013; Vazsonyi et al., 2012). Being a cyber-aggressor also positively correlates with other types of antisocial behaviors that are contrary to the social norm, such as consuming tobacco, alcohol, or illegal drugs, hurting animals, theft, or damaging other people’s property (Buelga, Iranzo, Cava, & Torralba, 2015; Chan & La Greca, 2016; Sticca et al., 2013).
Along the same line, it comes as no surprise that a community variable such as having antisocial friendships would also be a risk factor for being a cyber-aggressor. There is evidence that having these types of friends increases the likelihood of performing antisocial behaviors (Criss et al., 2016; Cutrín, Gómez-Fraguela, & Luengo, 2015) and that having friendships that encourage cyber-aggression increases the chances of engaging in cyber-aggressive behavior (Bastiaensens et al., 2016). However, to the best of our knowledge, there is no published research that specifically focuses on the predictive capacity of antisocial friendships for becoming a cyber-aggressor.
Among potentially predictive family variables of cyber-aggression, previous research suggests that family affection and communication constitute protective factors. Adolescents who perceive greater empathy, closeness, emotional warmth, affection, and support on the part of their parents are at lower risk of becoming cyber-aggressors (Floros, Siomos, Fisoun, Dafouli, & Geroukalis, 2013; Martins, Veiga, Freire, Caetano, & Matos, 2016). Analyzing the relationship between behavioral control and being a cyber-aggressor gives less clear results. Most research has found that teens with fewer family rules and monitoring have an increased risk of being cyber-aggressors (Kokkinos, Antoniadou, Asdre, & Voulgaridou, 2016; Martins et al., 2016). Nonetheless, some studies have found that high levels of overprotection are associated with a higher likelihood of being a cyber-aggressor (Floros et al., 2013). The studies that have specifically analyzed parental control of Internet use have also yielded mixed results. Some studies highlight its protective role. For example, Aoyama, Utsumi, and Hasegawa (2012) highlight the indirect influence of parental control and monitoring on cyber-aggression through time spent on the Internet. By contrast, other studies do not consider it to be a significant indicator (Floros et al., 2013; Law, Shapka, & Olson, 2010).
Despite the growing social interest in cyber-aggression among teenagers, few studies have focused on identifying their main predictors. The previously published studies represent a breakthrough in the knowledge about the subject. However, as stated above, the predictive role of some of the variables analyzed is not yet entirely clear. One method of illuminating this matter is to test a predictive model that includes all these variables, thus making it possible to identify potential confounding variables and to obtain estimated indicators that describe as faithfully as possible the true predictive capacity of each variable. No previous study has tested a predictive model that includes all of the variables noted above. The overall aim of this paper is to analyze the predictive capacity of individual, family, and community variables noted on the likelihood of engaging in cyber-aggression among adolescents, occasionally or intensely, controlling for the effect of potential confounding variables. This overall objective can be broken down into a series of specific objectives which are presented below, along with their corresponding starting hypotheses:
O1. Analyzing the predictive capacity of the adolescent’s age concerning the probability that they commit cyber-aggressive acts towards other adolescents.
H1. Older students are expected to demonstrate a higher risk of being cyber-aggressors.
O2. Analyzing the predictive capacity of an adolescent’s sex concerning the probability that they commit cyber-aggressive acts towards other adolescents.
H2. Boys are expected to demonstrate a higher probability of being cyber-aggressors than girls.
O3. Analyzing the predictive capacity of an adolescent’s use of social networks concerning the probability that they commit cyber-aggressive acts towards other adolescents.
H3. Adolescents who use social networks are expected to be more likely to be cyber-aggressors.
O4. Analyzing the predictive capacity of an adolescent’s use of instant messaging programs concerning the probability that they commit cyber-aggressive acts towards other adolescents.
H4. Adolescents who use instant messaging programs are expected to be more likely cyber-aggressors.
O5. Analyzing the predictive capacity of an adolescent’s frequency of non-school-related Internet use concerning the probability that they commit cyber-aggressive acts towards other adolescents.
H5. Adolescents who spend more time on the Internet for non-school-related activities are expected to be more likely to be cyber-aggressors.
O6. Analyzing the predictive capacity of being a victim of cyber-aggression concerning the probability of performing cyber-agressive acts.
H6. Adolescent victims of cyber-aggression are expected to exhibit a higher probability of being cyber-aggressors.
O7. Analyzing the predictive capacity of an adolescent’s level of impulsivity concerning the probability that they commit cyber-aggressive acts.
H7. Impulsivity is expected to be a risk factor for committing cyber-aggressive acts.
O8. Analyzing the predictive capacity of an adolescent’s empathy concerning the probability that they commit cyber-aggressive acts towards other adolescents.
H8. Empathy is expected to be a protective factor.
O9. Analyzing the predictive capacity of committing aggressive acts off-line in a school environment concerning the probability of committing cyber-aggressive acts.
H9. Adolescents who commit more aggressive acts in school environments are expected to also commit more cyber-aggressive acts.
O10. Analyzing the predictive capacity of other types of antisocial behavior concerning the probability of committing cyber-aggressive acts.
H10. Adolescents who engage in other types of antisocial behavior are expected to also commit more acts of cyber-aggression.
O11. Analyzing the predictive capacity of having antisocial friends concerning the probability of committing cyber-aggressive acts towards other adolescents.
H11. Having antisocial friends is expected to be a risk factor for being a cyber-aggressor.
O12. Analyzing the predictive capacity of parents’ affection and communication concerning the probability of their children committing cyber-aggressive acts.
H12. Adolescents who perceive more affection and better communication with their parents are expected to be less likely to commit cyber-aggressive acts.
O13. Analyzing the predictive capacity of parental behavioral control concerning the probability that their children commit cyber-aggressive acts.
H13. Parental behavioral control is expected to be a protective factor in relation to their children committing cyber-aggressive acts.
O14. Analyzing the predictive capacity of parental control of Internet use concerning the probability that their children commit cyber-aggressive acts.
H14. Parental control of Internet use is expected to be a protective factor in relation to their children committing cyber-aggressive acts.Method
A total of 3,059 adolescents (51.5% boys) 12 to 18 years of age (M = 14.01, SD = 1.39) from Asturias (Spain) were assessed. In total, 95% had their own mobile phone, 93.5% used instant messaging programs in their free time, and 77.9% participated in social networking sites in their free time. The sample was selected through stratified random and cluster sampling from the total number of students in Compulsory Secondary Education (CSE) in Asturias. To that end, the population of CSE schools supported by public funds (97.6% of the total) in Asturias was divided into seven geographical areas. In each area, a number of schools proportional to the total number of schools in the area in the population was randomly selected. As a result, 19 CSE schools were eventually selected. The questionnaires were given to all of the groups in years 1 to 4 of CSE in each center selected. The selected centers were found in both rural and urban areas and were attended by students of heterogeneous socioeconomic levels.
In the 19 schools selected, there were 3,697 students in Compulsory Secondary Education at the time of the study. Of those, 3,233 (87.5%) completed the questionnaires. The questionnaires were not given to the remaining 464 students (12.5% of the total) due either to them being absent on that day or to not having obtained the consent of their parents or legal guardians. Of the 3,233 students who did complete the questionnaires, 174 (5.4%) were eliminated from analysis due to errors or omissions in a significant proportion of their responses in one or more of the questionnaires. The final sample described above was therefore made up of 3,059 students.
Demographic data and management of communication technologies. An ad hoc questionnaire was used to collect information regarding age, sex, and the grade year of respondents as well as information referring to the availability and frequency of use of communication technologies. Age and school grade were evaluated through two open-ended questions. The other variables were evaluated by means of dichotomous Yes/No response items (“I own a mobile phone”, “In my free time, I participate in social networks —Tuenti, Facebook, or other”—, and “In my free time, I use instant messaging programs —WhatsApp, Messenger, or other”—), or polytomous items (“In general, from Monday to Friday, how many hours per day do you spend using the Internet for non-school tasks?”, and “In general, during the weekend, how many hours per day do you spend using the Internet for non-school tasks?”) with five options (none, less than an hour, between one and two hours, between two and three hours, and more than three hours).
Behavioral control. To measure parental control of activities as perceived by adolescents, the “behavioral control” factor from the Dimensions of Parenting Style Questionnaire by Álvarez-García, García, Barreiro-Collazo, Dobarro, and Antúnez (2016) was used. It consists of four items (“My parents try to know where I am going when I leave home”, “If I return home late, my parents ask me why and who I was with”, “My parents set limits on the time that I should return home”, and “My parents ask me how I spend money”), in which respondents are asked to assess the extent to which each statement is true (from 0, completely false, to 3, completely true). The total score for each respondent in this factor corresponds to the sum of the scores on each item (minimum 0, maximum 12). High scores indicate high levels of behavioral control. The internal consistency of this scale with this study sample is adequate (α = .75).
Rules for Internet use. Parents establishing limits on the Internet use, as perceived by adolescents, was evaluated by using the “rules for Internet use” factor from the Parental Control on Internet Usage Questionnaire by Álvarez-García, García, Cueli, and Núñez (2017). It consists of three items (“My parents or legal guardian set some rules regarding what I can or cannot do on the Internet”, “My parents or legal guardian limit the content I can access on the Internet through computer filters”, and “My parents or legal guardian limit the hours I use the Internet, either by word or by configuring the computer”, with four response options (from 0, completely false, to 3, completely true). The total score for each respondent in this factor corresponds to the sum of the scores on each item (minimum 0, maximum 9). High scores indicate greater establishment of standards by families. The internal consistency of this scale with the sample for this study is adequate (α = .71).
Supervision of Internet use. The opinions of adolescents regarding the extent to which their parents supervise how they use the Internet were evaluated with the “monitoring Internet use” factor from the Parental Control on Internet Use Questionnaire by Álvarez-García, García et al. (2017). It consists of four items (“When I access the Internet in my free time, my parents monitor and take a look at the screen”, “My parents know the passwords to access my email accounts, social networking sites, and messaging programs”, “My parents monitor my profiles on social networks, e.g., Facebook, Tuenti, Twitter, Instagram”, and “My parents know my contact lists”), with four response options (from 0, completely false, to 3, completely true). The total score for each respondent in this factor corresponds to the sum of the scores on each item (minimum 0, maximum 12). High scores indicate high levels of supervision. The internal consistency of this scale with the sample for this study is high (α = .80).
Affection and communication. The perceptions of adolescents regarding their parents’ support, emotional closeness, and willingness to talk were assessed by the “affection and communication” factor from the Dimensions of Parenting Style Questionnaire by Álvarez-García, García et al. (2016). It consists of four items (“When I talk to my parents, they show interest and pay attention”, “My parents encourage me to tell them my problems and concerns”, “If I have a problem, I can count on my parents’ help”, and “My parents show interest in me when I am sad and angry”), with four response options (from 0, completely false, to 3, completely true). The total score for each respondent in this factor corresponds to the sum of the scores on each item (minimum 0, maximum 12). High scores indicate high levels of affection and communication. The internal consistency of this scale in this study sample is high (α = .87).
Impulsivity. The degree of the respondents’ impulsivity was assessed using a self-reported scale previously used by the research team (Álvarez-García, Barreiro-Collazo, Núñez, & Dobarro, 2016). The scale was created using part of the impulsivity criteria proposed by the DSM-5 for the diagnosis of Attention Deficit and Hyperactivity Disorder (American Psychiatric Association, 2013). It consists of five items: “In class or when playing games, I often find it difficult to wait, so I jump in or interrupt others”, “I often blurt out an answer before a question has been completed”, “I often say what comes to mind without thinking of the consequences first or whether it is appropriate for the conversation”, “I often do things without thinking of the consequences”, and “I often find waiting difficult, so I do not wait for my turn to speak or I interrupt the person who is speaking”. The response is a Likert-type scale with four options (from 0, completely false, to 3, completely true). The total score for each respondent in this factor corresponds to the sum of the scores on each item (minimum 0, maximum 15). High scores indicate high levels of impulsivity. The internal consistency of the scores obtained with the scale in this study sample is adequate (α = .75).
Empathy. The degree of empathy in the adolescents evaluated was measured by a self-reported scale previously used by the research team (Álvarez-García, Barreiro-Collazo et al., 2016). It is composed of five items that refer to the extent to which a respondent believes that he/she is capable of identifying with others and sharing his/her feelings: “I feel the misfortunes of others”, “If a classmate is teased, I feel bad thinking about what is happening to him/her”, “I am patient with people who do things worse than I do”, “When I see that a friend is sad, I also become sad”, and “I am happy when something good happens to someone I know.” The response is a Likert-type scale with four options (from 0, completely false, to 3, completely true). The total score for each respondent in this factor corresponds to the sum of the scores on each item (minimum 0, maximum 15). High scores indicate high levels of empathy. The internal consistency of the scores obtained with the scale in this study sample is adequate (α = .70).
Aggression at school. To assess the extent to which the respondents have engaged in aggressive behavior on school premises over the past three months, a self-reported scale previously designed and used by the research team was used (Álvarez-García, García et al., 2016). It is composed of six items: “I have not let a classmate participate in my group during the break or physical education”, “I have not let a classmate participate in my group in a class activity”, “I have laughed at or teased a classmate”, “I have talked badly about a classmate behind his/her back”, “I have insulted a classmate to his/her face”, and “I have hit a student from the school, on or off school grounds”. The response is a Likert-type list scale four options (from 0, never to 3, always). The total score for each respondent in this factor corresponds to the sum of the scores on each item (minimum 0, maximum 18). High scores indicate high levels of aggression. The internal consistency of this scale with this study sample is adequate (α = .75).
Antisocial behavior. To assess the extent to which respondents recognize engaging in different types of antisocial behavior, a self-reported scale previously designed and used by the research team was used (Álvarez-García, García et al., 2016). It consists of six items: “I consciously soiled, damaged, or destroyed public furniture, e.g., a wall, a trashcan, a lamppost, seats on the bus”, “I stole something from a shop, school, or a private home”, “I trespassed on private property”, “I have hit or fought with a stranger to the point of harming him/her”, “I used illegal drugs”, and “I have gotten drunk.” The response requested from respondents is dichotomous (1 = true, 0 = false), indicating whether they have performed these activities at least once in the last year. The total score for each respondent in this factor corresponds to the sum of the scores on each item (minimum 0, maximum 6). High scores indicate high levels of antisocial behavior. The internal consistency of the scale in this sample is adequate (KR20 = .73).
Antisocial friendships. To assess the extent to which respondents relate to antisocial friendships, a scale previously used by the research team was used (Álvarez-García, García et al., 2016). It comprises four items, in which the respondents indicate whether the situation described has occurred during the past year: “One or some of my best friends have soiled, damaged, or destroyed public furniture, e.g., a wall, a trashcan, a lamppost, seats on the bus”, “One or some of my best friends have stolen something from a shop, school, or a private home”, “One or some of my best friends have had a real physical fight with another young person”, and “One or some of my best friends have consumed illegal drugs”. The response requested from the respondent is dichotomous (1 = true, 0 = false). The total score for each respondent in this factor corresponds to the sum of the scores on each item (minimum 0, maximum 4). High scores indicate high levels of antisocial friendships. The internal consistency of the scale in this sample is adequate (KR20 = .72).
Cyber-victimization. The frequency with which the respondents have been victims of aggression via mobile phone or the Internet during the last three months was assessed using the Cyber Victimization Questionnaire for Adolescents (CYVIC; Álvarez-García, Núñez, Barreiro-Collazo, & García, 2017). It consists of 19 items that refer to five types of cyber-victimization: verbal cyber-victimization (e.g., “I have been made fun of using offensive or insulting comments on social networks”); online exclusion (e.g., “A group of people has agreed to ignore me on social networks”); visual-sexual cyber-victimization (e.g., “I have been pressured to do things that I did not want to, regardless of whether I agreed to do them or not, under the threat of having intimate conversations or images of me disseminated”); visual cyber-victimization – teasing/happy slapping (e.g., “I have been forced to do something humiliating, and this has been recorded and then disseminated to make fun of me”); and impersonation (e.g., “I have been impersonated on the Internet, and comments have been posted in my name, as though coming from me”). A Likert-type response format is used (from 0 = never to 3 = always). In this study, the total score on cyber-victimization for each respondent was obtained by adding the scores from the 19 items (minimum 0, theoretical maximum 57). High scores indicate high levels of cyber-victimization. The internal consistency of the scale in this study sample is adequate (α = .79).
Cyber-aggression. The frequency with which the respondents admit having committed aggressions via mobile phone or the Internet during the last three months was evaluated through the Cyber-Aggression Questionnaire for Adolescents (CYBA; Álvarez-García, Barreiro-Collazo et al., 2016). This self-reported scale has the same indicators and response format as the CYVIC, but this time referring to aggression. The total score for each respondent regarding cyber-aggression responds to the sum of the scores on the 19 items (minimum 0, theoretical maximum 57). High scores indicate high levels of cyber-agression. The internal consistency of the scale in the sample of this study is high (α = .82).
Permission to administer the questionnaires was requested from the administration in each center selected. Each teaching center acquired family consent for the participation of the students in the study because they were underage. The questionnaires were completed by the students at the educational center during school hours. At the time of the application of the questionnaires, participants were informed of the voluntary and anonymous nature of the test as well as the confidential treatment of the data obtained.
First, the two polytomous variables used (“In general, from Monday to Friday, how many hours per day do you spend using the Internet for non-school tasks?” and “In general, during the weekend, how many hours per day do you spend using the Internet for non-school tasks?”) were recoded as dichotomous variables (three hours or fewer/more than three hours).
Then, the sample was divided into three subgroups, depending on the degree of cyber-aggression exerted. The “no cyber-aggression” group is composed of those who answered never to all of the statements of the CYBA questionnaire. The “severe cyber-aggression” group is composed of those who obtained a score equal to or greater than the 99th percentile in the CYBA (direct score ≥ 14.40). To determine this cut-off point, the data on severe cyber-aggression in adolescence – which is approximately 1% – offered by recent national studies published in Spain were taken as a reference (Calmaestra et al., 2016; Díaz-Aguado, 2010). The “occasional cyber-aggression” group comprises those who acknowledged having engaged in aggressive behavior at least once but whose total score in the CYBA is below the 99th percentile.
Once the database was closed, analyses were conducted to address the study objectives. To that end, first, a preliminary analysis and, then, a multinomial logistic regression analysis were conducted. The preliminary analysis consisted of an initial descriptive analysis of the sample set and potential predictors analyzed, in terms of means and standard deviations or frequencies and percentages, depending on the categorical or continuous nature of the variable analyzed. Second, the means or percentages of non-cyber-aggressors, occasional cyber-aggressors, and severe cyber-aggressors were compared, focusing on the potential predictors analyzed. To that end, one-way analyses of variance (ANOVA) or chi-square tests were conducted, depending on the nature of each predictor variable; the effect size was calculated using the eta squared or Cramer’s V, respectively. Next, the simple correlation between cyber-aggression and each of the potential predictors analyzed was studied, in addition to the degree of association between each of the predictors. To that end, the Pearson’s correlation coefficient, the point-biserial correlation coefficient, or the phi coefficient was used, depending on the nature of the variables involved in the association analyzed.
Subsequently, multinomial logistic regression analyses were conducted to identify the predictive capacity of the variables analyzed regarding being an occasional or a severe aggressor, in both cases taking the non-cyber-aggressor group as reference. For both occasional and severe cyber-aggression, first, crude odds ratios were calculated using univariate regression analysis. Subsequently, multivariate regression analyses were conducted, aiming to provide adjusted estimators that approximate as closely as possible the real relationship between potential predictors and being a cyber-aggressor. To that end, each group of predictive variables was progressively included, in blocks. As a result, there were six predictive models. Crude and adjusted odds ratio were compared, in addition to those of each fitted model and those of the next. When the inclusion of a new indicator variable in the model significantly affected the odds ratio of the rest, the degree of association between them was analyzed to determine the reason. In this manner, we were able to identify not only the variables with higher predictive capacity but also potentially confounding variables, indirect relationships, and even spurious relationships between the potential predictors analyzed and cyber-aggression. Finally, to analyze the validity of the six models studied and, therefore, the explanatory contribution of each block of variables introduced in each step, the Nagelkerke’s pseudo R2 and the percentage of correct classifications were calculated.Results
Description of the sample on the basis of potential predictive variables.Table 1 provides an overview of the sample with respect to the potential predictors analyzed. There are statistically significant differences, depending on the degree of cyber-aggression exercised in all the potential predictors analyzed, except for sex. The greater the severity, the greater the age, Internet use, impulsiveness, antisocial behavior, and cyber-victimization, and the lesser the parental control and empathy.
Association between the variables analyzed. As shown in Table 2, the correlation between cyber-aggression and each of the potential predictors analyzed is statistically significant in all cases. It is positive in most of them, except for empathy and the four variables relating to parental control, in which case cyber-aggression correlates negatively. The magnitude of these correlations is small, except for impulsivity, cyber-victimization, and the three variables relating to antisocial behavior, with which cyber-aggression correlates moderately.
The relationship between potential predictors is statistically significant in 112 out of 120 pairs analyzed (Table 2). The magnitude of the association between indicators is small, except in 18 of the pairs, in which it is moderate.
Multinomial Logistic Regression
Predictors of occasional cyber-aggression. The unadjusted (univariate) regression analysis suggests that 15 of the16 predictive variables analyzed (all except sex) are statistically significantly related to becoming an occasional cyber-aggressor. These 15 variables would constitute risk factors, except for empathy and the variables concerning parental control. These two would, in turn, act as protective factors (Table 3). However, the multivariate regression analysis, which included all variables analyzed (Model 6), shows that only seven of them had a statistically significant relationship with being an occasional cyber-aggressor. These seven variables include participating in social networks, using instant messaging programs, degree of impulsivity, frequency with which they attack other students in the educational center, antisocial behavior outside the center, antisocial friendships, and degree of cyber-victimization endured. All of these are risk factors. Teenagers who participate in social networks and those who use instant messaging programs are 1.6 and 1.7 times more likely, respectively, to be occasional cyber-aggressors than those who do not use them. The probability of being an occasional cyber-aggressor is 1.1 times higher in those who score one point higher in impulsivity, antisocial behavior, or antisocial friendships and 1.4 times higher in those who score one point higher in exercising school aggression or in suffering cyber-victimization.
Table 3 shows how the inclusion of each new block of variables affects the estimators of the previous model. Changes in the magnitude of the estimators after the inclusion of each new block of variables are consistent with the association found between predictors, which is provided in Table 2.
Predictors of severe cyber-aggression. The univariate regression analysis suggests that 15 of the 16 predictive variables analyzed (all except the use of instant messaging programs) are statistically significantly related to being a severe cyber-aggressor. The 15 variables would constitute risk factors, with the exception of empathy and the variables relating to parental control, which would constitute protective factors (Table 4). However, in the multivariate regression analysis that included all variables analyzed (Model 6), only four of them showed statistically significant correlations with becoming a severe cyber-aggressor. These four variables are degree of impulsivity, frequency with which attacks on other students are committed in the educational center, and degree of cyber-victimization suffered, as risk factors, and degree of empathy, as a protective factor. The probability of being a severe cyber-aggressor is 1.2, 1.6, or 1.9 times higher in those who score one point higher in impulsivity, cyber-victimization endured, or school aggression exerted, respectively, and 1.3 times lower in those who score one point higher in empathy.
Table 4 shows how the inclusion of each new block of variables affects the estimators of the previous model. Changes in the magnitude of estimators after the inclusion of each new block of variables are consistent with the association found between predictors, shown in Table 2.
Analysis of model validity. The variables included in the final model (Model 6) together explain 45% of the variance in cyber-aggression scores (Table 5). The percentage of cases that this final model is able to correctly predict is 76.6%. In both cases, these values are the best obtained in all models analyzed. The largest increase in R2 and in the percentage of correct classifications occurs when variables related to antisocial behavior (Model 5) are included. The smallest increase occurs with the inclusion of variables related to parental control (Model 3).
The overall objective of this study was to analyze the predictive capacity of some individual, family, and community variables on the probability of committing occasional or severe cyber-aggression among adolescents, controlling for confounding variables. The results obtained suggest a complex relationship between these variables. Although there is a relationship between the potential predictors analyzed and cyber-aggression, if the effect of other variables is statistically controlled, only a few still maintain a statistically significant relationship. In other words, given that the mean differences based on the degree of cyber-aggression, the simple correlations between each predictor and cyber-aggression, and the univariate estimators in the regression analysis are statistically significant (except for a few cases that were noted above and will be recalled), it could be inferred that the potential predictors analyzed have a direct causal relationship with cyber-aggression. However, the results obtained with the multivariate regression analysis and the correlations found between predictors cast doubt on this inference. Only the level of adolescent impulsivity, the frequency with which they engage in aggressive behavior at school, and the frequency with which they suffer cyber-aggression show a significant predictive relationship in all analyses performed, in all three cases as risk factors. Therefore, the results obtained suggest the existence of not only certain direct relationships but also some indirect or even spurious relationships.
With regard to sociodemographic variables, the results suggest that age is positively associated with both occasional and severe cyber-aggression (the older the adolescent, the greater cyber-aggression), which supports the starting hypothesis. However, results suggest that this relationship is indirect. Age is related to other variables with greater predictive capacity. Age determines the possibility of using social networking and instant messaging programs (in Spain, these tools cannot be used without parental authorization before the age of 14 years) and, with it, both to cause and to suffer aggressions through them. In addition, older students (who have failed a year) are more impulsive and less empathetic, and they also engage in more aggressive acts at school and in antisocial behavior. Of the variables analyzed, sex has the least correlation with the overall score of cyber-aggression used. The starting hypothesis in this study was that boys would be more likely to commit acts of cyber aggression than girls. However, the inclusion of impulsivity and empathy in the regression model prevents sex from constituting a statistically significant predictor. As in previous research, in the present study, boys tend to be more impulsive (Chapple, Vaske, & Hope, 2010) and less empathetic (Llorca-Mestre, Malonda-Vidal, & Samper-García, 2017). Therefore, results show that also the relationship between sex and cyber-aggression is indirect.
With regard to variables related to Internet use, those adolescents who use social networks, instant messaging, and the Internet for non-school-related activities were expected to be more likely to be cyber-aggressors. Results, however, do not suggest such a clear, direct relationship. These variables exhibit the greatest overestimation in their relationship with cyber-aggression in univariate analysis. That is, they have the greatest difference between crude odds ratios and adjusted odds ratios. The use of the Internet relates to the other indicators. An overestimation occurs in both occasional and severe cyber-aggression, but above all in the latter. For occasional cyber-aggression, the use of both social networking and instant messaging programs remain statistically significant predictors of cyber-aggression, after the inclusion of all potential predictors in the model.
Mean differences, simple correlations, and the univariate regression analysis suggest that the four family variables studied (behavioral control, rules for Internet use, monitoring Internet use, and affection and communication) are protective factors against cyber-aggression, as hypothesized at the beginning. However, when sociodemographic, personality, and Internet use variables are introduced in the regression model, the four family variables cease to be statistically significant predictors of severe cyber-aggression. If the three variables related to antisocial behavior (school aggression, antisocial behavior, and antisocial friendships) are also included, family variables cease to be statistically significant predictors of occasional cyber-aggression. It suggests the existence of indirect and even spurious relationships between the family variables analyzed and cyber-aggression. Thus, it is possible that parental control affects Internet use, which, in turn, affects the probability of committing cyber-aggression. It is also possible that parenting style influences adolescent impulsivity and empathy, which, in turn, as indicated previously, may affect the probability of exerting school aggression, antisocial behavior, and cyber-aggression. It is also likely that there is a spurious relationship between parental control and the degree of cyber-aggression committed, explained by variables such as age, sex, or frequency of Internet use. These variables may have a significant effect on both degree of parental control and frequency of cyber-aggression. In this regard, more research is needed. In this study, family variables have generated the smallest increase in the explanatory capacity of the model. This finding suggests that other variables included in the model represent more significant predictors of cyber-aggression than the family variables analyzed.
With regard to the personality variables analyzed, the starting hypothesis in this study was that impulsivity would be a risk factor and empathy would be a protective factor for committing cyber-aggressive acts. Indeed, the results obtained suggest that impulsivity is a risk factor of cyber-aggression. Impulsivity is one of the three predictors, along with exerting school aggression and suffering cyber-victimization, that consistently show a statistically significant predictive capacity with regard to cyber-aggression in the various analyses conducted. Their relationship seems to be not only direct but also indirect through their relationship with other relevant variables, such as use of the Internet or school aggression. The other personality variable analyzed, empathy, is a protective factor, especially for severe cyber-aggression.
Results support the hypothesis of the importance of variables related to antisocial behavior as a risk factor for cyber-agression. The largest increase in the predictive capacity of the model occurs when these variables are included. In particular, all analyses conducted in this study suggest that school aggression is one of the main predictors of cyber-aggression among adolescents. In fact, some authors have proposed that, in adolescence, school aggression and cyber-aggression are two manifestations of the same phenomenon (Olweus, 2012). In this study, the only community variable analyzed, antisocial friendships, is found to be a risk factor for occasional cyber-aggression but not for severe cyber-aggression. This result suggests that severe aggression depends more on individual variables, such as impulsivity and lack of empathy, than on peer pressure or encouragement by the group of friends. Conversely, having antisocial friends may cause adolescents who have a lower tendency to break the rules or be aggressive to be encouraged by peer pressure to do so occasionally. Aggression can be a means of improving self-image and gaining status within the group (Chan & La Greca, 2016).
Finally, all analyses conducted in this study suggest that, as hypothesized, cyber-victimization is a risk factor for cyber-aggression. There are two main explanations for this result. Firstly, some research suggests that adolescents who have been victims of cyber-bullying (severe victims) may become bullies to get revenge or to protect themselves (improve their social position or image) (Haltigan & Vaillancourt, 2014). Secondly, although the attacks through electronic means can be planned ahead – proactive aggression – (Calvete, Orue, Estévez, Villardón, & Padilla, 2010), the fact that cyber-victimization and impulsivity are significant risk factors for cyber-aggression suggests that cyber-aggression may also be the result of an impulsive reaction to a previously received aggression – reactive aggression.
In short, although differences in means, simple correlations, and the univariate regression analysis suggest that virtually all of the variables analyzed are predictive of cyber-aggression among adolescents, the statistical control using multivariate regression analysis shows that only impulsivity, empathy, school aggression, and cyber-victimization are significant predictors of severe cyber-aggression. Empathy is a protective factor, and the others are risk factors. The use of social networking and instant messaging programs, impulsivity, school aggression, antisocial behavior, antisocial friendships, and cyber-victimization are significant predictors of occasional cyber-aggression. In all cases, they constitute risk factors.
This study represents a contribution to the research field, adding to the limited work published to date on predictors of cyber-aggression among adolescents. Regarding research, this study warns of the danger of drawing conclusions on the basis of mean differences, simple correlations, and univariate regression analyses. There is a need for a statistical control of the effect of other relevant variables to obtain estimators of predictive capacity that are as precise and realistic as possible. In practice, the results of this work contribute to clarifying the risk and protective factors of cyber-aggression among adolescents to design intervention programs that prevent the problem in the most effective manner possible. In this regard, similar to previous studies (Del Rey, Casas, & Ortega, 2016), the results obtained suggest that good practices to prevent antisocial behavior and school aggression will also have a positive effect in the prevention of cyber-aggression. In particular, it is important to prevent impulsivity, to develop empathy, and to teach to handle peer pressure. Therefore, the development of socio-emotional skills is of the utmost importance for the prevention and treatment of the problem (Palladino, Nocentini, & Menesini, 2016). It is also important to work on digital competence, particularly with respect to the use of social networking and instant messaging programs. In fact, these media favor immediate and impulsive responses (also as a response to an aggression), in addition to a lack of empathy (by not allowing the consequences of the perpetrator’s actions on the victim to be witnessed). The role of family is also very important for the prevention of cyber-aggression among adolescents. However, the results of this study suggest that the most effective strategy is not direct parental control but a parenting style that promotes reflective and empathic behavior on the part of the adolescent and that discourages attacks and attitudes that favor aggression.
Therefore, this study is a contribution to research on cyber-aggression among adolescents. However, it is not exempt from some limitations. Four of them are noted here. First, data are self-reported; thus, results may be biased due to distorted responses or social desirability (Navarro-González, Lorenzo-Seva, & Vigil-Colet, 2016). In the future, these results may be complemented with other types of assessment techniques. Second, the study was conducted with a wide and randomly selected sample but a sample that is limited to some ages and a specific geographical area. Therefore, any generalization of the conclusions of this study to other age groups or regions should be performed with caution (Muñiz, Elosua, & Hambleton, 2013). In the future, it would be interesting to replicate this study in other age groups and regions. Third, the present study was a cross-sectional and correlational research; thus, it cannot establish causal relationships between variables with certainty. The conclusions of this study are hypotheses to test in future studies and with other methodologies. In this sense, it would be important to expand the number of longitudinal studies. Fourth, and finally, this study included a large number of potentially predictive variables, drawn from the most important variables according to the previous evidence available. The set of variables included explain a significant percentage of the variance of the scores in cyber-aggression. However, the variables analyzed here do not exhaust all possible predictive or confounding variables. Further work should refine the regression model to have the greatest predictive capacity, without becoming unmanageable.
Álvarez-García, D., Barreiro-Collazo, A, & Núñez, J. C. (2017). Cyberaggression among adolescents: Prevalence and gender differences. Comunicar, 50, 89-97. https://doi.org/10.3916/C50-2017-08
Álvarez-García, D., Barreiro-Collazo, A., Núñez, J. C., & Dobarro, A. (2016). Validity and reliability of the Cyber-aggression Questionnaire for Adolescents (CYBA). The European Journal of Psychology Applied to Legal Context, 8, 69-77. https://doi.org/10.1016/j.ejpal.2016.02.003
Álvarez-García, D., García, T., Barreiro-Collazo, A., Dobarro, A., & Antúnez, A. (2016). Parenting style dimensions as predictors of adolescent antisocial behavior. Frontiers in Psychology, 7, 1383. https://doi.org/10.3389/fpsyg.2016.01383
Álvarez-García, D., García, T., Cueli, M., & Núñez, J. C. (2017). Control parental del uso de Internet durante la adolescencia: evolución y diferencias de género [Parental control of Internet use during adolescence: Evolution and gender differences]. Manuscript submitted for publication.
Álvarez-García, D., Núñez, J. C., Barreiro-Collazo, A., & García, T. (2017). Validation of the Cybervictimization Questionnaire (CYVIC) for adolescents. Computers in Human Behavior, 70, 270-281. https://doi.org/10.1016/j.chb.2017.01.007
Aoyama, I., Utsumi, S., & Hasegawa, M. (2012). Cyberbullying in Japan: Cases, government reports, adolescent relational aggression, and parental monitoring roles. In Q. Li, D. Cross, & P. K. Smith (Eds.), Cyberbullying in the global playground. Research from international perspectives (pp. 183-201). Malden, MA: Wiley-Blackwell.
Bastiaensens, S., Pabian, S., Vandebosch, H., Poels, K., Van Cleemput, K., DeSmet, A., & De Bourdeaudhuij, I. (2016). From normative influence to social pressure: How relevant others affect whether bystanders join in cyberbullying. Social Development, 25, 193-211. https://doi.org/10.1111/sode.12134
Beran, T., Mishna, F., McInroy, L. B., & Shariff, S. (2015). Children’s experiences of cyberbullying: A Canadian national study. Children & Schools, 37, 207-214. https://doi.org/10.1093/cs/cdv024
Brewer, G., & Kerslake, J. (2015). Cyberbullying, self-esteem, empathy and loneliness. Computers in Human Behavior, 48, 255-260. https://doi.org/10.1016/j.chb.2015.01.073
Buelga, S., Iranzo, B., Cava, M. J., & Torralba, E. (2015). Psychological profile of adolescent cyberbullying aggressors. Revista de Psicología Social, 30, 382-406. https://doi.org/10.1080/21711976.2015.1016754
Calmaestra, J., Escorial, A., García, P., Del Moral, C., Perazzo, C., & Ubrich, T. (2016). Yo a eso no juego. Bullying y cyberbullying en la infancia [I don’t play that. Bullying and cyberbullying in childhood]. Madrid, España: Save the Children.
Calvete, E., Orue, I., Estévez, A., Villardón, L., & Padilla, P. (2010). Cyberbullying in adolescents: Modalities and aggressors’ profile. Computers in Human Behavior, 26, 1128-1135. https://doi.org/10.1016/j.chb.2010.03.017
Chan, S. F., & La Greca, A. M. (2016). Cyber victimization and aggression: Are they linked with adolescent smoking and drinking? Child & Youth Care Forum, 45, 47-63. https://doi.org/10.1007/s10566-015-9318-x
Chapple, C. L., Vaske, J., & Hope, T. L. (2010). Sex differences in the causes of self-control: An examination of mediation, moderation, and gendered etiologies. Journal of Criminal Justice, 38, 1122-1131. https://doi.org/10.1016/j.jcrimjus.2010.08.004
Ciucci, E., & Baroncelli, A. (2014). Emotion-related personality traits and peer social standing: Unique and interactive effects in cyberbullying behaviors. Cyberpsychology, Behavior, and Social Networking, 17, 584-490. https://doi.org/10.1089/cyber.2014.0020
Corcoran, L., Mc Guckin, C., & Prentice, G. (2015). Cyberbullying or cyber aggression? A review of existing definitions of cyber-based peer-to-peer aggression. Societies, 5, 245-255. https://doi.org/10.3390/soc5020245
Criss, M. M., Houltbergb, B., J., Cui, L., Bosler, C. D., Morris, A. S., & Silk, J. S. (2016). Direct and indirect links between peer factors and adolescent adjustment difficulties. Journal of Applied Developmental Psychology, 43, 83-90.https://doi.org/10.1016/j.appdev.2016.01.002
Cutrín, O., Gómez-Fraguela, J. A., & Luengo, M. A. (2015). Peer-group mediation in the relationship between family and juvenile antisocial behavior. The European Journal of Psychology Applied to Legal Context, 7, 59-65. https://doi.org/10.1016/j.ejpal.2014.11.005
Del Rey, R., Casas, J. A., & Ortega, R. (2016). Impact of the ConRed Program on different cyberbulling roles. Aggressive Behavior, 42, 123-135. https://doi.org/10.1002/ab.21608
Del Rey, R., Lazuras, L., Casas, J. A., Barkoukis, V., Ortega, R., & Tsorbatzoudis, H. (2016). Does empathy predict (cyber) bullying perpetration, and how do age, gender and nationality affect this relationship? Learning and Individual Differences, 45, 275-281. https://doi.org/10.1016/j.lindif.2015.11.021
Díaz-Aguado, M. J. (2010). Estudio estatal sobre la convivencia escolar en la Educación Secundaria Obligatoria [National survey on relations at school in Compulsory Secondary Education]. Madrid, España: Ministerio de Educación.
Festl, R., Scharkow, M., & Quandt, T. (2015). The individual or the group: A multilevel analysis of cyberbullying in school classes. Human Communication Research, 41, 535-556. https://doi.org/10.1111/hcre.12056
Floros, G. D., Siomos, K. E., Fisoun, V., Dafouli, E., & Geroukalis, D. (2013). Adolescent online cyberbullying in Greece: The impact of parental online security practices, bonding, and online impulsiveness. Journal of School Health, 83, 445-453. https://doi.org/10.1111/josh.12049
Fletcher, A., Fitzgerald-Yau, N., Jones, R., Allen, E., Viner. R. M., & Bonell, Ch. (2014). Brief report: Cyberbullying perpetration and its associations with socio-demographics, aggressive behaviour at school, and mental health outcomes. Journal of Adolescence, 37, 1393-1398. https://doi.org/10.1016/j.adolescence.2014.10.005
Haltigan, J. D., & Vaillancourt, T. (2014). Joint trajectories of bullying and peer victimization across Elementary and Middle School and associations with symptoms of psychopathology. Developmental Psychology, 50, 2426-2436. https://doi.org/10.1017/S0954579417000979
Kokkinos, C. M., Antoniadou, N., Asdre, A., & Voulgaridou, K. (2016). Parenting and Internet behavior predictors of cyber-bullying and cyber-victimization among preadolescents. Deviant Behavior, 37, 439-455. https://doi.org/10.1080/01639625.2015.1060087
Kowalski, R. M., Giumetti, G. W., Schroeder, A. N., & Lattanner, M. R. (2014). Bullying in the digital age: A critical review and meta-analysis of cyberbullying research among youth. Psychological Bulletin, 140, 1073-1137. https://doi.org/10.1037/a0035618
Law, D. M., Shapka, J. D., & Olson, B. F. (2010). To control or not to control? Parenting behaviours and adolescent online aggression. Computers in Human Behavior, 26, 1651-1656. https://doi.org/10.1016/j.chb.2010.06.013
Lee, C., & Shin, N. (2017). Prevalence of cyberbullying and predictors of cyberbullying perpetration among Korean adolescents. Computers in Human Behavior, 68, 352-358. https://doi.org/10.1016/j.chb.2016.11.047
Llorca-Mestre, A., Malonda-Vidal, E., & Samper-García, P. (2017). Prosocial reasoning and emotions in young offenders and non-offenders. The European Journal of Psychology Applied to Legal Context, 9, 65-73. https://doi.org/10.1016/j.ejpal.2017.01.001
Martins, M. J. D., Veiga, A. M., Freire, I., Caetano, A. P., & Matos, A. (2016). Cyber-victimization and cyber-aggression among Portuguese adolescents: The relation to family support and family rules. International Journal of Cyber Behavior, Psychology and Learning, 6(3), 65-78. https://doi.org/10.4018/IJCBPL.2016070105
Mishna, F., Khoury-Kassabri, M., Gadalla, T., & Daciuk, J. (2012). Risk factors for involvement in cyber bullying: Victims, bullies and bully–victims. Children and Youth Services Review, 34, 63-70. https://doi.org/10.1016/j.childyouth.2011.08.032
Muñiz, J., Elosua, P., & Hambleton, R. K. (2013). International Test Commission guidelines for test translation and adaptation: Second edition. Psicothema, 25, 151-157.
Navarro, R. (2016). Gender issues and cyberbullying in children and adolescents: From gender differences to gender identity measures. In R. Navarro, S. Yubero, & E. Larrañaga (Eds.). Cyberbullying across the globe: Gender, family, and mental health (pp. 35-61). Cham, Switzerland: Springer.
Navarro-González, D., Lorenzo-Seva, U., & Vigil-Colet, A. (2016). How response bias affects the factorial structure of personality self-reports. Psicothema, 28, 465-470. https://doi.org/10.7334/psicothema2016.113
Nocentini, A., Calmaestra, J., Schultze-Krumbholz, A., Scheithauer, H., Ortega, R., & Menesini, E. (2010). Cyberbullying: Labels, behaviours and definition in three European countries. Australian Journal of Guidance & Counselling, 20(2), 129-142
Olweus, D. (2012). Cyberbullying: An overrated phenomenon? European Journal of Developmental Psychology, 9, 520-538. https://doi.org/10.1080/17405629.2012.682358
Palladino, B. E., Nocentini, A., & Menesini, E. (2016). Evidence-based intervention against bullying and cyberbullying: Evaluation of the NoTrap! program in two independent trials. Aggressive Behavior, 42, 194-206. https://doi.org/10.1002/ab.21636
Rice, E., Petering, R., Rhoades, H., Winetrobe, H., Goldbach, J., Plant, A., ... Kordic, T. (2015). Cyberbullying perpetration and victimization among Middle-School students. American Journal of Public Health, 105(3), e66-e72. https://doi.org/10.2105/AJPH.2014.302393
Sticca, F., Ruggieri, S., Alsaker, F., & Perren, S. (2013). Longitudinal risk factors for cyberbullying in adoles- cence. Journal of Community and Applied Social Psychology, 23, 52-67.
Cite this article as: Álvarez-García, D., Núñez, J. C., García, T., & Barreiro-Collazo, A. (2018). Individual, family, and community predictors of cyber-aggression among adolescents. The European Journal of Psychology Applied to Legal Context, 10, 79-88. https://doi.org/10.5093/ejpalc2018a8
Correspondence: email@example.com (D. Álvarez-García).
Copyright © 2019. Colegio Oficial de Psicólogos de Madrid