Preliminary Validation and Gender Invariance of the Technology Anxiety Scale in Older Spanish Adults

Introduction

Interest in affective concepts related to the use of technology and information systems has grown enormously in recent decades (Davidson, 1998; Salazar-Concha et al., 2021), especially concerning its “dark side,” that is, negative emotions (Salanova et al., 2014; Tarafdar et al., 2019). However, there are many conceptual and methodological gaps in the investigation of this topic (Agogo & Hess, 2018).

Three concepts are close to that of technology anxiety. In chronological order of appearance, they are “computer anxiety” (Masters, 1967), “technophobia” (Paschen & Gresser, 1974), and “technostress” (Brod, 1982). Although used as synonyms, probably because they have been investigated from areas outside Psychology, such as information technologies or human resources, the difference can be seen in their names: compared to phobia, anxiety is an emotional response that is not necessarily abnormal; regarding stress, anxiety is a specific emotion like anger or sadness (Agogo & Hess, 2018; Khasawneh, 2018; Tacy, 2016). In this sense, “technology anxiety” is a generalization to other technologies of the original “computer anxiety”.

Older people are especially vulnerable to technology anxiety. For example, two studies developed in Italy showed that high levels of computer anxiety in older people were associated with disuse, lack of autonomy, and the need for help in the management of technology, linked to high levels of technophobia (di Giacomo et al., 2020; di Giacomo et al., 2020).

The risks of technology anxiety can transcend playful or social aspects, causing harm in other areas. In the professional context, Hou et al. (2017) experimentally demonstrated that the combination of advanced age and technophobia generated great reading difficulties on screens. Tams (2017), concerning the increased technology anxiety of older workers, proposed an explanation that is based on differences in inhibitory control compared with younger adults.

As for health, technology anxiety linked to internet use is associated with lower well-being and quality of life (Nimrod, 2018a). It can even interfere with the acceptability of the use of surveillance devices in older people with heart disease (Tsai et al., 2020) or tumor monitoring systems (Özdemir-Güngör & Camgöz-Akdag, 2018). Technology anxiety is considered a risk factor for older people’s health, reaching critical levels during the COVID-19 pandemic (Nimrod, 2021, 2022). If age is linked to low levels of education, the problem is aggravated, to the point of requiring psychoeducational intervention programs to address it.

Psychometric measures of “technostress” (Nimrod, 2018b; Penado Abilleira et al., 2020), “technophobia” (Martínez-Córcoles et al., 2017), and “computer anxiety” (Raub, 1981) have been developed. Concerning technology anxiety, Meuter et al. (2003) developed the first measure by adapting that of Raub (1981) to any information and communication technology (ICT) in a consumer study on self-service technologies. This produced a unifactorial instrument, with nine items, reliability of = .90, predicting the use of self-service technology well above other variables such as age, gender, income level, or educational level. Niemelä-Nyrhinen (2007) tested the psychometric properties of this scale with a sample of 620 Finnish baby boomers, finding a bifactorial structure. The factor they called “fear” was composed of the first six items, which are drafted in the sense of anxiety, and they obtained a reliability of = .89. The factor they called “confidence” was composed of the last three items, drafted in the opposite direction to anxiety but they did not report the reliability, although they stated that it was significantly lower, probably due to the smaller number of items.

López-Bonilla and López-Bonilla (2012) adapted a scale composed of the items of that of Meuter et al. (2003) and another scale of “computer anxiety” (Loyd & Gressard, 1984) to Spanish, using a sample of 819 university students. To our knowledge, there is no psychometric measure of technology anxiety in Spanish validated in the older population.

Considering the above, the aims of this work were, firstly, to adapt the Technology Anxiety Scale to Spanish in the older population and, secondly, to differentially determine its fit to a unifactorial (Meuter et al., 2003) and bifactorial structure (Niemelä-Nyrhinen, 2007).

Method

Participants

The sample included 355 Spaniards between the ages of 63 and 97, with a mean age of 78.63 (SD = 6.77). Of them, 122 were men and 233 were women, of whom 115 (32.4%) indicate not having any studies, 148 (41.7%) have primary education, 49 have secondary education (13.8%), and 43 indicate having completed university studies (12.1%). Regarding marital status, most were married (47%; n = 167) or widowed (48.5%; n = 172), although 13 were divorced people (3.7%), 2 with a common-law partner (0.6%), and 1 single person (0.3%) also participated.

Instrument

Technology Anxiety Scale (Meuter et al., 2003)

As indicated in the introduction, this scale consists of nine items, generated from those that made up the Computer Anxiety Scale (Raub, 1981), which are rated on a seven-point Likert scale, ranging from strongly disagree to strongly agree. Higher scores indicate higher levels of anxiety.

Items from other scales were used to assess the concurrent validity. Concretely, the four items of personal growth (e.g., “For me, life has been a continuous process of study, change, and growth”), the five items of positive relationships with others (e.g., “I know I can trust my friends and they know they can trust me”), and the four items of self-acceptance (e.g., “For the most part, I am proud of who I am and the life I lead”) from the Ryff Scales of Psychological Well-being (Ryff, 1989) in its Spanish version (Villar et al., 2010). The response scale of them ranged from 1 = strongly disagree to 5 = strongly agree. Higher scores indicate higher levels of psychological well-being. To assess perceived quality of life and health satisfaction, the items “How would you rate your quality of life?” and “How satisfied are you with your health?” were used. Both are scored on a 5-point Likert scale and are included in the “WhoQol Bref” Quality of Life Scale, developed by the World Health Organization and validated in Spain with older people by Lucas-Carrasco et al. (2011).

Procedure

As part of a research project on ICTs and intergenerational relationships that included older people, and because of the limitations in the application of the scale in person due to COVID-19, discretionary sampling was carried out through the snowball technique by telematic means. A Google Forms link was sent to university students through social networks and/or email to spread among their friends and peers so that they could send it to their relatives over 60 years old who interacted with ICTs and were willing to collaborate in the study.

The form requested the older person to collaborate anonymously and required informed consent before showing the scale, which did not request any data that would allow personal identification. The data processing was carried out following the European Data Protection Regulation (GDPR) of 2016.

The adaptation of this scale to Spanish was carried out following the guidelines of the International Test Commission (ITC) for the intercultural adaptation of tests (Muñiz et al., 2013), and the list for the quality control of the translation-adaptation of the items of Hambleton and Zenisky (2011).

Two native Spanish speakers with a C1 English level separately translated the items into Spanish. There was an initial agreement of 89%, with only one discrepancy in item 4, the word “apprehensive”, that was resolved by consensus: two native English speakers, an American and a New Zealander, with a C1 Spanish level; then it was translated back into English, with no discrepancies. The results of the back translations were identical.

Regarding the response options, although their range is directly related to the sensitivity of the scale (Bisquerra & Pérez-Escoda, 2015), we preferred to use a five-point scale compared to the original of seven points because the cognitive impairment that may be found in older people (Mora Simón et al., 2012) could affect their discriminative capacity in responses to scales with high ranges. In fact, recent studies show that the adjusted prevalence rates for mild cognitive impairment range from 5.2% in the population aged 65 to 69 years to 45.3% in populations aged 85 years or older (Vega Alonso et al., 2018). Not controlling this source of variation at older ages would have posed a substantial threat to the internal validity of the results.

Data Analysis

Descriptive data were obtained for the items that make up the instrument: means, standard deviations, skewness, and kurtosis. Given the multivariate non-normality and the ordinal nature of the items, weighted least squares with mean and variance adjusted (WLSMV) estimation was used (Finney & DiStefano, 2013) to perform two confirmatory factor analyses. First, a one-dimensional model (M1) was tested (Meuter et al., 2003) and then another model of two correlated factors (M2): the “fear factor” and the “confidence factor” (Niemelä-Nyrhinen, 2007).

To check the fit of the models, the following fit indices were considered: the comparative fit index (CFI), the Tucker-Lewis index (TLI), and the standard root mean square residual (SRMR). Kenny et al. (2015) and Taasoobshirazi and Wang (2016) argue that the root mean square error of approximation (RMSEA) does not perform well with few degrees of freedom and small sample sizes. For this reason, this index is not considered in the interpretation of results.

The reference values were a minimum value of .90 for the CFI and TLI, and a maximum cut-off of .08 for the SRMR to consider as indicative of a good fit (Hu & Bentler, 1995; Marsh et al., 1996). The factor measurement reliability (Kline, 2016) was evaluated with the composite reliability index (CRI) (Werts et al., 1974), which is identical to the ω coefficient (McDonald, 1999) but using the standardized factor loadings (Raykov & Marcoulides, 2015). Then, the average variance extracted (AVE) (Fornell & Larcker, 1981) was estimated to determine factor measurement validity (Raykov, 1997). The alpha values are also given for the two subscales and for the total scale. For the CRI and alpha, values higher than .70 are considered acceptable (Raykov, 2004), whereas for AVE values above .50 are necessary (Hair, 2010).

To test the concurrent validity of the Technology Anxiety Scale, bivariate correlations with measures of psychological well-being (personal growth, positive relationships with others and self-acceptance), quality of life, and health satisfaction were calculated.

We assessed the invariance of the factorial structure by gender with the factorial structure that presented the best fit. Three nested models with increasing degree of restriction were tested: the base model assessed configural invariance and allowed free estimation of all the parameters for each group. The metric (weak) invariance model, nested in the configural model, added the restriction of invariant factor loadings among groups. Finally, the scalar (strong) invariance model, nested in the second model, added the thresholds constraint of the invariant items among the comparison groups. When sample size is small, sample sizes are unequal, and the pattern of noninvariance is uniform, the following cutoff criteria are suggested: for testing loading invariance, a change of ≤ -.005 in CFI, supplemented by a change of ≥ .025 in SRMR would indicate noninvariance; for testing intercept or residual invariance, a change of ≤ -.005 in CFI, supplemented by a change of ≥ .005 in SRMR would indicate noninvariance (Chen, 2007).

All these analyses were carried out using the program Mplus 8.7 (Muthén & Muthén, 2017). To obtain descriptions of the sociodemographic variables and the items of the Technology Anxiety Scale and for concurrent validity measures, the IBM SPSS 26 statistical package was used.

Results

The process of translating the items was described in the Procedure section. The resulting scale in Spanish is shown in Appendix.

The descriptive data of the items are shown in Table 1. As can be seen, the items with a higher mean were “technological terminology sounds like confusing jargon to me” and “I’m sure I can learn technology-related skills”, whereas the item “I feel apprehensive about the use of technology” is the one that presented a lower mean. The total score of the scale in this sample has a mean of 25.82 with 9 being the minimum score and 45 the maximum score. Therefore, this sample presents moderately high values of technology anxiety.

As for the factor analyses, as can be seen in Table 2, Model 1 (one-dimensional) showed acceptable fit indices, although the two-factor model clearly presented better values.

All factor loadings were higher than .600, as shown in Figure 1. In addition, the Confidence factor and the Fear factor had a negative and high correlation. CRI values were good both for the Fear (CR = .924) and the Confidence (CR = .773) factors, as occurred with the AVE values (Fear = .714, Confidence = .609). Both the overall Cronbach’s alpha (α = .899) and for each subscale (Fear α = .896, Confidence α = .717) were good.

Regarding the results of concurrent validity, the fear factor showed negative and significant correlations with Personal Relationships with Others (r = -.286; p < .001), Personal Growth (r = -.209, p < .001), Quality of Life (r = -.153, p < .001), and Health Satisfaction (r = -.188, p < .001), and no significant correlation with Self-acceptance (r = -.041, p = .438). On the other hand, the Confidence factor showed positive and significant correlations with Personal Relationships with Others (r = .293, p < .001), Personal Growth (r = .248, p < .001), Quality of Life (r = .253, p < .001), and Health Satisfaction (r = .270, p < .001) and did not show significant correlation with Self-acceptance (r = .079, p = .134).

Using the two-factor model, which showed better fit, gender invariance analysis was performed. Table 3 shows that this model fit men’s group well and women’s group excellently. After this confirmation, we verified that the criteria for the scalar invariance of the instrument were met and, therefore, the latent means between genders could be compared. The results showed that, by setting the reference group (men) to 0, there were no group differences either in the Fear factor (b = 0.080, z = 0.092, p = .359) or the Confidence factor (b = 0.057, z = 0.620, p = .535).

Discussion

Our objective was to validate the Technology Anxiety Scale (Meuter et al., 2003) in the Spanish-speaking older population.

After the process of cross-cultural adaptation of the items, which was carried out without problems, both the unifactorial structure proposed by Meuter et al. (2003) and that of two correlated factors found by Niemelä-Nyrhinen (2007) showed adequate fit. Therefore, we can conclude, preliminarily, that this instrument is suitable for measuring technology anxiety in older Spaniards. Of course, our results are in line with those of Niemelä-Nyrhinen (2007), and therefore support the use of the scale to measure two inversely correlated factors, which correspond to the direction of the drafting of the items: the factor called “fear”, which integrates the first six items, reflecting negative emotionality towards technology, and the factor called “confidence”, which integrates the last three items and reflects trust, that is, a positive experience of the relationship with technology.

We cannot compare our fit indices with those of the two studies that serve as a reference because neither of them performed confirmatory factor analyses, but only exploratory factor analyses. As a reference, our factor loadings, ranging between .63 and .90, were even higher than those obtained by Meuter et al. (2003), between .58 and .89, and very similar to those obtained by Niemelä-Nyrhinen (2007), between .61 and .91. Nor can we take as a reference the scale validated in Spanish university students by López-Bonilla and López-Bonilla (2012), not only because they merged the technology anxiety scale of Meuter et al. (2003) with another “computer anxiety” scale (Loyd & Gressard, 1984), but also because they did not report the fit indices of the confirmatory factor analysis in their work.

Also, although the two previous studies were conducted with people of both genders, 53% and 59% of women respectively, the validity of the instrument for both genders had not actually been established. Our study confirms the factorial invariance of the scale by gender and therefore endorses the use of this scale in men and women.

Regarding concurrent validity, it was confirmed that older adults with higher technology anxiety had lower scores on well-being, quality of life, and health satisfaction, in line with the results obtained by Nimrod (2018a) and Pfaffinger et al. (2021).

The study presents some limitations. First, due to the COVID-19 pandemic situation, the questionnaire was applied precisely using technology, through Google Forms, also distributed through technology such as email, instant messaging, or social networks. This could have produced a sample bias favoring participants with lower levels of technology anxiety. In any case, the fact that it was their relatives who urged the older people to carry out the survey could mitigate this bias. Secondly, and concerning the previous limitation, although the sample was sufficient to reach the desired statistical power, it would have been desirable to have a higher number of participants, a fact that perhaps was also conditioned by the moment of application, during the COVID-19 pandemic. Finally, as argued in the introduction, to facilitate the application to older participants and even to those with potential cognitive impairment (Mora Simón et al., 2012; Vega Alonso et al., 2018), we decided to simplify the response format, going from seven to five Likert-type alternatives. In fact, the sample was finally made up of people with an average age of more than 78 years, and more than 70% had no studies or had only primary studies. However, despite being the desired effect on our part, the reduction of the response alternatives may have influenced the good factorial fit we found. In any case, these limitations imply that this validation should be considered preliminary and that studies with larger and more representative samples are required.

Evidence indicates that technology anxiety is one of the main barriers to older people’s adaptation to digital life (di Giacomo et al., 2019, 2020; Nimrod, 2018b; Tsai et al., 2020). Interventions are therefore required to counter it, which, in turn, need appropriate screening tools for this purpose. In this sense, the main practical implication of our study has been to demonstrate the preliminary validity of the Technology Anxiety Scale to evaluate technology anxiety in older Spaniards.