Controlled Trial of a Short-term Intensive Parent Training Program within the Context of Routine Services for Autistic Children in China

Introduction

Autism is characterized by impairments in reciprocal social-communicative interaction and restricted and repetitive behaviors or interests (American Psychiatric Association, 2013). Although the national prevalence of autism is yet to be estimated in China, a large regional study of children aged six to 12 years published in 2020 revealed a local rate of 0.70%, similar to the global estimate of 0.76% (Baxter et al., 2015; Zhou et al., 2020).

Autistic children experience more co-occurring behavioral problems (such as oppositional defiant behaviors and conduct problems) than their non-autistic cohorts (Simonoff et al., 2008; Totsika et al., 2011), which create further limitations in children’s daily activities and impose more challenges to parenting (Hastings & Brown, 2002). Short-term parent training is widely used to support families of autistic children in low-resource settings. Due to the increasing number of diagnoses of autism, limited local service provision, and substantial economic cost of the autism-associated consequences at both family- and state-levels, such programs have been increasingly advocated globally as a cost-effective alternative to more conventional intensive interventions delivered for consecutive years (such as the Early Intensive Behavioral Intervention; Anan et al., 2008; Bearss et al., 2015; De Vries, 2016; Liao et al., 2022; Morris et al., 2011).

Short-term parent training programs may be a more scalable approach in Chinese settings. As with some other countries, access to autism treatments in China is prohibited by the widening service gap, absence of educational opportunities, high cost of private therapies, and insufficient financial support (Chang & Zaroff, 2017; Liao et al., 2020; Liao et al., 2022; P. Liu & Liu, 2018; McCabe, 2012). In a survey involving more than 3,800 Chinese parents of children with autism, over 80% of the respondents reported a lack of support and a strong need for parent training programs to improve parental efficacy and overcome a range of autism-induced challenges (Guo et al., 2014). Further, a qualitative study investigating the perspectives of caregivers and service providers with regard to a parent training program for autistic children in China, also emphasized the importance of delivering comprehensive program topics within a short period and providing substantial opportunities for practice and feedback, especially given the severe lack of parental knowledge about autism and a sense of urgency to intervene while their child was still young (Fang, Lachman, et al., 2022).

Five systematic reviews have been conducted to assess the evidence-base for parent training programs for autism (Fang, Barlow, et al., 2022; Nevill et al., 2018; Oono et al., 2013; Postorino et al., 2017; Strauss et al., 2013). The results showed that they can reduce child emotional and behavioral problems (which are not necessarily unique to autistic children) (Fang, Barlow, et al., 2022; Postorino et al., 2017), decrease autism symptoms (Nevill et al., 2018; Oono et al., 2013), promote child social communication and language development (Nevill et al., 2018; Strauss et al., 2013), and improve parent-child interaction (Fang, Barlow, et al., 2022; Oono et al., 2013). The review conducted by Fang, Barlow, et al. (2022) focused on programs delivered in China, by employing comprehensive searches in the English international and Chinese regional databases. Eleven short-term parent training programs for families of autistic children were identified, and it highlighted brevity as a common characteristic, despite the fact that families of autistic children in low-resource settings often experience accumulated adversities and need substantial support, as well as an absence of rigorous evaluations of such programs being delivered in real-world settings (Fang, Barlow, et al., 2022).

This paper reports a controlled trial of a short-term intensive parent training program that has been supporting thousands of Chinese families of autistic children aged three to six years since the 1990s. The program is delivered by a long-established local NGO, known as Beijing Stars and Rain Education Institute for Autism (SREIA). Despite its long history and wide coverage, the program was only assessed using a single-group pre-post study (Xiong et al., 2010) and lacked evidence of effectiveness generated from a more robust experimental design. This report adheres to the Journal Article Reporting Standards (Appelbaum et al., 2018). Results of the evaluation might be used to inform service provision in other low- and middle-income countries (LMICs) where caregivers of autistic children face similar challenges.

Method

Study Design

We conducted a real-world quasi-experimental study to assess the effectiveness of the SREIA program in reducing child externalizing behaviors within the context of routine service provision. Participants were drawn from a permanent SREIA waiting list. Enrolment to the program was on a first-come, first-serve basis. Families attending between September 2020 and November 2020 comprised the treatment group. Those who signed up for the delivery between November 2020 and January 2021 comprised the waitlist control group. Due to COVID-19, three families on the waiting list decided not to attend the program and were thus also allocated into the control group. Outcomes were measured at baseline and immediate post-intervention. A concurrent process evaluation was also conducted to investigate program acceptability, participant involvement, and implementation.

Participants

Recruitment of parent-child dyads was based on the standard criteria used by the SREIA program, including a) children between three to six years of age with an official diagnosis of autism issued by hospitals, and b) primary caregivers undertaking the responsibility for the daily childcare for at least five days a week. No restrictions are imposed on the severity of diagnosis or the level of child abilities, but as part of routine service, the program administers the Autism Treatment Evaluation Checklist (ATEC; Rimland & Edelson, 1999) at baseline and post-intervention time points to track child changes. The ATEC comprises four subscales, including Speech/Language/Communication, Sociability, Sensory/Cognitive Awareness, and Health/Physical/Behavior. It has a total score ranging from 0 to 179, with lower scores indicating fewer autism symptoms. However, it should be noted that the ATEC is mainly designed to capture treatment benefits and monitor child progress, rather than give a diagnosis or rate the severity. The decision not to administer other standardized measures in this research was based on concerns about participant burden and overall data quality. Recruitment was conducted by the implementing organization inviting caregivers on the waiting list to the study. Each family had one parent-child dyad participating in the program. Financial reimbursements were provided to participants who completed data collection at both time points. Eight program practitioners were also recruited to the study.

Table 1 summarizes the demographic characteristics of the participants. More than 80% of the children involved in this program were male, in line with the fact that autism is about four times more common in boys than girls (Maenner et al., 2020). The mean age was 4.70 (SD = 1.12). The proportion of children from ethnic minority groups (7.21%, n = 8) was roughly in accordance with that reported in the Sixth National Population Census of China (8.49%) (National Bureau of Statistics of China, 2011). The ATEC total scores at baseline varied substantially, ranging from 16 to 118, with a mean score of 67.13 (SD = 21.29), speech/language/communication: range 0-26, M (SD) = 15.67 (6.49); sociability: 3-32, 15.99 (6.17); sensory/cognitive awareness: 5-31, 18.69 (5.79); health/physical/behavior: 2-41, 16.77 (8.15). Participating caregivers were predominantly female (92.79%, n = 103) aged 34.75 (SD = 5.8) on average. More than half were full-time caregivers (63.06%, n = 70). One tenth attended middle school or lower (17.12%, n = 19), 40% received high school-level education (41.44%, n = 46), and 40% attended college or more (41.44%, n = 46). About one third (28.83%, n = 32) had an annual income below the national average salary. Half of the families (50.45%, n = 56) had two or more children and slightly fewer have a rural household registration (42.34%, n = 47).

Ethical Considerations and Preregistration

Ethical approval was granted by the University of Oxford (R67619/RE001) and Beijing Normal University (SSDPP-HSC2020001). Verbal informed consent was collected from all participants by trained research staff. The study was preregistered (NCT04257331).

Intervention

The SREIA parent training program has a long history of providing services to families of autistic children in China. It was developed in the 1990s based on applied behavior analysis (ABA), which is grounded in operant conditioning (Cooper et al., 2007; Skinner, 1950), and has evolved to address issues identified during delivery of the program since then. It is aimed at reducing child behavioral problems and promoting child development by increasing parental knowledge, improving parenting skills, promoting parental mental health, and creating social support. The program combines ABA-based protocols (such as discrete trial teaching) with social learning theory-based parenting techniques (such as instruction giving, ignore, and redirect) (Bandura, 1971) to improve child behavior and foster child skill development. Developmental perspectives are integrated by teaching parents to understand child developmental stages and domains and engage children in active parent-child communication and interaction, with the purpose of facilitating affective exchange and creating a foundation for learning other skills (Schreibman et al., 2015). Cognitive behavioral principles are also incorporated to increase parental knowledge and alter parental attitudes towards their child and themselves, as well as their situation and prospects, so that parental emotions and behaviors can be further improved. Additionally, a brief introduction to TEACCH [Treatment and Education of Autistic and related Communication-handicapped Children] strategies is provided to inform parents about the utilization of child visual strengths and the creation of structured work routines (Mesibov & Shea, 2010). Based on those underpinning theories, the program offers a wide range of techniques and is delivered five days a week for 11 weeks (Table 2).

Fourteen large didactic presentations on the core themes are offered across the 11 weeks and attended by all participating caregivers, with childcare provided by volunteers when there is no COVID-19 pandemic or restrictions on mass gatherings. Tailored training and practice opportunities are then arranged in smaller groups comprising ten parent-child dyads to improve the use of skills. The program is primarily delivered in-person, while the fourteen presentations were also streamed online to reduce mass gathering during COVID-19. Individual pre-consultations are offered with the aim of enabling program practitioners to better understand family strengths and concerns. An individualized treatment plan is created collaboratively and followed for each dyad, covering targeted child behaviors, caregiver areas for improvement, and other priorities identified by caregivers. Materials and handouts are provided. Practitioners are from various backgrounds (without healthcare-related education) but all trained and certified by the implementing organization. To promote participant engagement and learning, the program integrated a variety of delivery techniques and substantial opportunities for practice (Table 2).

The program has two delivery formats, which are implemented concurrently and share the same program content, with the purpose of exploring ways to reduce cost while improving outcomes. Both versions divide participants to several groups based on children’s ATEC assessments. Version A has four groups of dyads who attend didactic sessions together in a larger group but practice sessions separately in smaller groups, whereas Version B has three groups of dyads and delivers all (didactic and practice) sessions separately for each group. Participants are offered the option of selecting either version when signing up for the waiting list.

Power Calculation

As a real-world evaluation conducted in a routine service context, the sample size was determined by the service capacity of the implementing organization. The minimal detectable standardized mean difference from two-sample, two-tailed t-test was -0.57 on the primary outcome measure. This was based on a final sample size of 111 with a significance level of .05 and statistical power of .80.

Outcomes and Measures

The outcomes of interest were identified by the program practitioners and developers in discussion with the research team. Child behavioral problems was considered to be the primary outcome reflecting the wider range of behavioral challenges experienced by parents on a daily basis and that the program aimed to address. It was assessed using the Externalizing scale of the Child Behavior Checklist (CBCL) for ages 1.5-5 (25 items) (Achenbach & Rescorla, 2001), as the majority of children were under six and similar levels of internal consistency of CBCL 1.5-5 was shown in autistic children aged five and those aged six (Basten et al., 2014). The Externalising scale contains two subscales: Attention Problems and Aggressive Behaviours. The Attention Problems subscale consists of five items (e.g., “can’t concentrate”, “quickly shifts”, and “wanders away”), and the Aggressive Behaviours subscale has 19 items (e.g., “can’t stand waiting” and “demands met”). The scale has been applied in children with a range of developmental disabilities, including autism, in high-income countries, such as the Incredible Years-Autism Spectrum and Language Delays Programme in the UK (McIntyre, 2008; Williams et al., 2020) and the Parents’ Plus Programme for children with developmental disabilities in Ireland (Quinn et al., 2007). It has demonstrated good cross-cultural factorial validity in China and adequate internal consistency in autistic children (J. Liu et al., 2011; Pandolfi et al., 2009).

Impact on several secondary outcomes was examined. “Autism symptoms” was assessed using the Chinese version of the ATEC (Rimland & Edelson, 1999). “Parenting style” was measured using the Over-Reactivity subscale of the Arnold-O’Leary Parenting Scale (Arnold et al., 1993) and the Supporting Positive Behavior subscale of the Parenting Young Children Scale (McEachern et al., 2012). “Parental mental health” was assessed using the Chinese version of the Depression Anxiety Stress Scale (Lovibond & Lovibond, 1995). “Family function” was assessed using the Chinese version of the Family APGAR scale (Smilkstein et al., 1982). All scales have been validated in China, except the Over-Reactivity subscale, which has been validated in families with high risks for child behavioral problems in the U.S. “Parental knowledge” was measured using a questionnaire designed for the SREIA evaluation to assess parental understanding of autism and child behaviors. For questionnaires that do not have a Chinese version, the translation followed the WHO guideline of the Process of Translation and Adaptation of Instruments (World Health Organization, 2020). All outcome measures were reported by participating parents.

“Program acceptability” was assessed only at post-intervention using a satisfaction questionnaire tailored to the SREIA program and the Chinese context (Incredible Years, 2020). The level of “participant involvement” was captured by calculating rates of attendance, engagement, and program dropout. The attendance and engagement rates were recorded using an attendance and engagement registry, reported by practitioners each day. The level of caregiver engagement was judged based on four criteria: responsiveness to practitioners, interactions with other caregivers, participation in group discussion, and completion of homework activities. Program dropout rate was calculated based on families who missed at least ten days. “Implementation” was measured through fidelity and quality of delivery. Fidelity was assessed using practitioner checklists. Practitioners reported whether they delivered the core components prescribed in the manual each week. The fidelity score comprised the overall ratio of delivered components to prescribed components. Quality of delivery was assessed by research staff observing sessions and filling out a quality of delivery survey. The survey contains 24 items, assessing the general skills of facilitating parent groups (such as knowledge of content, teaching skills, sensitivity to participants’ feelings and experiences, ability to handle tension, etc.) and skills of modelling parenting techniques and facilitating practice sessions (such as appropriately setting up the space and providing adequate support during participant practice). A total score is summed. Two sessions for each practitioner were randomly selected for research staff to rate the quality of delivery. The program process was also investigated using qualitative in-depth interviews and focus group discussions, the results of which are reported separately (Fang, Lachman, et al., 2022).

Data Collection and Analysis

Data collection took place between August to September 2020 for the baseline assessment, and November to December 2020 for the immediate post-test. Trained researchers administered the parent questionnaires to the treatment and control groups, by reading out the questions to caregivers. All data collectors had a master’s degree in social work and were trained in a range of relevant topics. Caregivers were requested to keep their allocation condition private from outcome data collectors and the data collectors were asked whether they knew to which group the participant belonged after each interview. The process evaluation data, including satisfaction survey, were collected by a separate group of research staff. No violation of blinding was detected.

Data analyses were conducted in R v4.0.3 and followed the principles of intent-to-treat by including all participant recruited at baseline. Baseline group differences were explored using two independent samples t-tests and chi-square tests. To reduce selection bias and improve baseline balance for the between-group comparison, the treatment effects were estimated using a difference-in-differences (DiD) design with propensity score weighting (PSW) (Stuart et al., 2014). Propensity scores were calculated using logistic regression. The key covariates were selected based on the relevant literature and included the baseline outcome measures and child gender. The treatment and control groups were weighted using the overlap weighting scheme to avoid extreme weights and ad hoc trimming (Li & Thomas, 2019). Group balance was checked using the absolute standardized mean difference, with a rule of thumb that the absolute standardized mean difference should be controlled under 0.1 (Rubin, 2001).

Weighted DiD models were then fitted, with accounting for the key covariates, to further adjust for unobserved variables and secular time trends (Wing et al., 2018). Dependency of outcomes for participants within each version and each caregiver group were checked using intra-cluster correlation coefficients and, when necessary, accounted for in linear mixed models with caregiver group or delivery version being treated as the random effect. Exploratory analyses were also conducted by replacing the binary treatment variable with a three-level variable in the regression models to assess the differences in treatment effects between the two versions of delivery. Robustness checks were conducted using DiD with inverse probability weighting and kernel matching (Austin & Stuart, 2015; Caliendo & Kopeinig, 2008). The ignorability assumption of propensity score weighting and parallel trend assumption of DiD were checked by examining the effect of group assignment on the baseline primary outcome measure and treating financial health (a placebo outcome associated with child externalizing behaviors (Slopen et al., 2010) but not affected by group assignment) as the outcome variable in the weighted DiD (Imbens, 2004; Wing et al., 2018). Beta coefficients were converted to Cohen’s d for reporting using the method suggested by Brysbaert and Stevens (2018).

Multivariate models were created to examine potential factors associated with participant involvement, with accounting for confounders drawn from existing evidence, including quality of delivery, baseline child externalizing problems, parental mental health, autism symptoms, parental education, adult age, adult gender, and family income. The possible associations between implementation factors and the treatment effect were investigated by first using univariate models and then adjusting for the key covariates. The missing data comprised less than 5% of the entire dataset and were compensated using REML (Hartung et al., 2008). No data were missing in variables that were included in the DiD models. Residuals in all regression models were checked for normality and outliers.

Transparency and openness. We reported how we determined our sample size, all manipulations, and all measures in the study, and we followed the JARS guidelines (Appelbaum et al., 2018). We excluded data from three participants, due to misdiagnosis and data withdrawal. The anonymized individual participant data, analytic code, unpublished full study protocol, and other research materials will be available upon request from researchers who provide a methodologically sound proposal. To gain access, requestors will need to sign a data sharing agreement.

Community involvement. The implementation organization took part in the planning of the overall research project, by contributing to the development of the research design, research questions, and evaluation tools. The recruitment of participants and implementation of the program were undertaken by the organization using its existing resources. The organization did not participate in data collection or analysis but has been actively involved in the interpretation of results and taking the lead in conveying the research findings to a wider autism community in China.

Results

Figure 1 shows the flow of participants through the study. The final sample involved 63 caregiver-child dyads in the treatment group—37 in Version A and 26 in Version B—and 48 in the control group. Table 3 presents the absolute standardized mean differences for each baseline measure. Table 4 presents the baseline and post-intervention scores for all outcome measures. While the t-tests and chi-square tests detected no significant group difference in demographic characteristics (Table 1) and outcome measure at baseline, several key covariates demonstrated substantial absolute standardized mean differences. The overlap weighting successfully reduced the absolute standardized mean differences across all covariates and achieved excellent group balance.

Estimation of Treatment Effects

The estimations of DiD are summarized in Table 5. Analyses found that the program was linked to reduced child externalizing problems (b =-2.71, 95% CI [-5.23, -0.18], Cohen’s d = -2.80), compared to the comparison group. The results of robustness checks (inverse probability weighting: b = -2.76, 95% CI [-5.08, -0.44]; kernel matching: b = -3.28, 95% CI [-4.59, -1.97]) were in line with the estimation and did not detect violation of underlying assumptions.

In terms of secondary outcomes, analyses found that the program was associated with increased parental knowledge of autism and child development (b = 2.08, 95% CI [2.07, 2.17], Cohen’s d = 2.91), improved parental mental health (b = -5.96, 95% CI [-11.74, -0.17], Cohen’s d = 2.89), and reduced over-reactive parenting (b = -0.63, 95% CI [-0.98, -0.27], Cohen’s d = -5.02), in comparison to control condition. No significant associations were detected for autism symptoms (b = -1.53, 95% CI [-6.44, 3.38], Cohen’s d = -0.86), family function (b = -0.08, 95% CI [-0.88, 0.71] , Cohen’s d = -0.28), or positive parenting practices (b = 0.16, 95% CI [-1.89, 2.21] , Cohen’s d = 0.22).

The exploratory analysis on version difference indicated that, compared to the comparison group, Version B was linked to a reduction in externalizing problems (b = -3.77, 95% CI [-6.26, -1.28], Cohen’s d = -4.26) and an improvement in parental mental health (b = -12.52, 95% CI [-19.87, -5.18] , Cohen’s d = -4.79), whereas Version A did not differ from the control group (externalizing problems: b = -2.08, 95% CI [-4.56, 0.41] , Cohen’s d = -2.35; mental health: b = -1.02, 95% CI [-8.35, 6.31] , Cohen’s d = -0.39). Both versions were associated with an increase in parental knowledge (VA: b = 1.49, 95% CI [0.28, 2.69] , Cohen’s d = 3.47; VB: b = 3.03, 95% CI [1.82, 4.24], Cohen’s d = 7.05) and a reduction in parental over-reactivity (VA: b = -0.72, 95% CI [-1.15, -0.28] , Cohen’s d = -4.68; VB: b = -0.44, 95% CI [-0.88, -0.01] , Cohen’s = -2.86). However, due to the small subgroup sample sizes, the analyses were underpowered.

Implementation Evaluation

Table 6 shows that the level of participant involvement was high, with most caregivers attending 98% of the time and the average engagement score being 5.58 (SD = 0.32). The main reasons for absence were child illness and family issues. On average, 95% of pre-specified topics and activities were implemented across the seven caregiver groups based on the fidelity checklists, with a mean quality of delivery score of 51.86 (SD = 1.13). The overall satisfaction achieved 78.19 (SD = 6.94).

The multivariate models showed that more parental mental health problems at baseline were related to lower engagement levels (b = -0.003, 95% CI [-0.006, -0.001]) but not attendance rate. No other factors examined were significantly associated with participant involvement (Table 7).

Further analysis conducted in the treatment group found that higher attendance rates were linked to fewer externalizing problems (b = -66.60, 95% CI [-125.30, -7.80]). While this relationship was not present after accounting for key covariates, higher satisfaction and engagement levels were both associated with better treatment effects in the treatment group (satisfaction: b = -0.24, 95% CI [-0.44, -0.03]; engagement: b = -6.81, 95% CI [-12.19, -1.44]) (Table 8).

Discussion

To our knowledge, this study is the first to evaluate a short-term intensive parent training program with a long history of supporting families of autistic children in China using a controlled trial. The evaluation was conducted in a routine service context with substantial effort to reduce sources of bias using a difference-in-differences design with propensity score weighting. Findings of this study might inform service provision in other LMICs where there is also a severe lack of resources for families of autistic children.

Program Effects

The results showed that the program was associated with a reduction in child externalizing behaviors, a change that equates to a Cohen’s d of -2.80, which is generally consistent with the finding (Hedges’ g = -1.47) of a previous systematic review of such programs in China (Fang, Barlow, et al., 2022). Although the systematic review reported Hedges’ g rather than Cohen’s d, research indicates that they produce extremely similar estimates when the sample size is larger than 20 (Lin & Aloe, 2021). However, our effect size was larger than other existing studies synthesizing trials conducted primarily in high-income countries and yielding effect sizes ranging from SMD = -0.39 to -0.59 (Postorino et al., 2017; Skotarczak & Lee, 2015; Tellegen & Sanders, 2013). There may be multiple reasons for this. First, the program provided substantial opportunities for participants to observe practitioner modelling, practice in sessions, and receive in vivo feedback, which promoted caregiver acquisition of skills. In addition, the wider evidence shows that parent training programs typically have a greater impact for families living in low-resource settings and who have an urgent need to address child behavior problems (Gardner et al., 2015; Leijten et al., 2013). The Chinese context is characterized by a severe lack of autism-related resources, and caregivers were in desperate need of evidence-based techniques with which to tackle some of the problems being experienced by their children, which could have increased the effect sizes. Autistic children demonstrate more behavior problems than children without autism or with other developmental disabilities (Totsika et al., 2011), and the wider research suggests that programs targeting families of children with existing behavior problems have stronger effects than universal or selective programs (Leijten et al., 2019). While the severity of the problems in children involved in this study may also be reflected by caregivers’ determination to participate in the program amidst all the challenges caused by COVID19, this might also have introduced a potential selection bias, in terms of making the sample less representative.

Analyses of secondary outcomes indicated that the program was also linked to better parental mental health, and a reduction in over-reactive parenting. This result differs from that of a previous review that found no group difference in parental stress in a parent-mediated early intervention delivered to caregivers of autistic children aged 17-72 months (Oono et al., 2013). This discrepancy may be because caregivers in the current study experienced an inclusive environment for the first time, which empowered and guided them towards greater acceptance of their children and themselves. Also, the improved parental self-efficacy and their perceived progress in child development might have created a sense of hope and increased feelings of control.

While we found no evidence of benefit in terms of reducing autism symptoms, which again is not consistent with the findings of the earlier reviews (Nevill et al., 2018; Oono et al., 2013), this may reflect the challenges of introducing a large number of techniques within a short timeframe, the absence of a long-term follow-up period, and the fact that it takes time for the techniques to become established (van Aar et al., 2017). The lack of association with parenting practices could be attributed to the services and information previously obtained by the participants, which predominantly emphasized the use of positive reinforcements. In addition, in order to reduce participant burden and increase the overall quality of data, a short scale was used to assess positive parenting practice, leading to a trade-off with regard to scale sensitivity. The absence of an impact on family functioning may be due to the same reason. Further, consistent with earlier research, the data suggests that conflicts occurred when non-participating family members did not understand the diagnosis or new parenting techniques (Mockford & Barlow, 2004).

The exploration of version differences indicated that better practitioner-dyads ratio was related to fewer child externalizing behaviors and better parental mental health, possibly as a result of the more individualized feedback and reduced practitioner workload. However, the results should be interpreted with caution due to the small samples available for subgroup analysis.

Implementation Evaluation

Although previous evidence shows that engaging parents can be a challenge for providers of parenting interventions (Axford et al., 2012), this program was delivered with a high attendance and engagement rate, in spite of its high intensity of five days a week over 11 consecutive weeks. The reasons for this may be three-fold. First, the substantial service gap and prohibitive cost of prolonged treatments might have led to a degree of urgency in terms of participant needs for parent training and social support. Second, the participants may have been highly motivated because the period from two to six years of age was considered by them to be the most ‘critical’ and the only chance for intervention, especially given the absence of local services for older autistic children (Chang & Zaroff, 2017). Third, a variety of delivery techniques were used to promote engagement, such as ongoing and flexible communication, reinforcement by practitioners, and peer support. In terms of quality of delivery, the program was rated as medium, which might be due to the heavy workload and lack of organizational support for practitioners (Fang, Lachman, et al., 2022).

The exploratory analyses to examine factors affecting participant involvement showed that a lower engagement rate was related to more baseline mental health symptoms, which is consistent with findings from previous research (Carr et al., 2016). While the literature tends to suggest that families from disadvantaged backgrounds may have lower attendance and engagement rates (Chacko et al., 2016), the current study found no such relationship, possibly reflecting the dearth of services and the need for support across the different socio-economic groups in LIMC contexts.

The analyses showed that a higher level of program satisfaction and engagement appeared to be associated with improved child behavioral adjustment, whereas the quality of delivery, attendance rate, and fidelity score made little difference. However, the results should be interpreted with caution, as the estimation of dose effect was limited by the small number of parent groups and the similarly high attendance and fidelity rates across groups. Future research should explore further the relationships between dosage and outcomes using methods such as complier average causal effect.

Strengths and Limitations

This study has several limitations. First, as a real-world evaluation of a program that is embedded within routine services, it was not possible to conduct randomization, which could have resulted in unidentified confounders. Also, some families withdrew due to COVID-19, potentially increasing the risk of bias. Second, it was not possible to conduct multiple assessments prior to the program to test the underlying parallel trends assumption of DiD due to COVID-19, again pointing to the risk of unobserved confounders. To mitigate the impact of bias, key covariates identified from the literature were included. The robustness checks also showed that unobserved factors were time-invariant and did not affect the estimation of outcomes (Wing et al., 2018). In addition, although this study involved more participants than other existing evaluations in China (Fang, Barlow, et al., 2022), the sample size was still small, limiting the power and precision to identify treatment effects. Further, the outcome measures relied on parent-report data, which may be subject to social desirability bias, and we were not able to triangulate the data using practitioner-report or observational measures, due to resource constraints. Efforts were, however, made to reduce this source of bias, by using measurement tools with adequate reliability and blinding data collectors to group allocation. The measurement of participant engagement relied on practitioner report. A consultation on criteria for the level of engagement was conducted with the practitioners to complement findings from the literature. However, the reliability of the results could have been further improved by rating the accuracy of parents’ practice with children and using standardized questionnaires. The study had limited male caregivers, undermining the generalizability of the results. Last, the data collection time point was immediately post-intervention. Longer follow-up periods are necessary to examine what proportion of techniques become firmly established and the extent to which the effects are maintained over time.

Despite the limitations, the study has a number of strengths. It is one of the first real-world evaluations of a parent training program for autistic children conducted in a routine service context, thereby being more likely to provide valuable information about the range of participants in clinical practice. Local stakeholders being actively engaged in the research also ensured that the research was locally relevant and increased the possibility of evidence uptake by community advocates and policymakers. Moreover, while many autism treatments have low levels of diversity in terms of the participants, being delivered either to families with high or low socio-economic status, this program was attended by participants from diverse backgrounds (e.g., 59% without a college degree and 42% with a rural household registration) and children with different levels of autism-related challenges, which implies that the program may be well applicable to a wide range of families. In addition, previous reviews of parent training for autism commonly highlight the inadequate reporting of their implementation and the potential impact of this. The concurrent process evaluation within this study provided important information with regard to program implementation while also allowing us to integrate the process data with outcome assessment to examine the associations between treatment effects and implementation variability (Oakley et al., 2006). Also, the DiD method was combined with propensity score weighting to compensate for the lack of randomization in the context of routine service and remove secular trend effects.

Implication and Conclusion

The findings of this study provide tentative support for the use of short-term intensive and practice-based parent training programs for families of young autistic children in LMICs to promote child behavioral adjustment and reduce wider health disparities. Although treatment models for autism can entail a variety of techniques and formats, researchers commonly advocate for intensive interventions that begin at an early age (National Research Council, 2001). Early intensive behavioral interventions have been found to be effective in promoting child development (Reichow et al., 2018), but can often be inaccessible due to the high cost and requirement for qualified professionals. As such, short-term parent training and parent-directed treatments have been developed as an alternative for families of young autistic children. The high participant commitment and satisfaction in the current study possibly reflects the substantial need for such programs in areas where there is a severe lack of services. Program providers were from a range of disciplines and trained by the implementing agency, thereby demonstrating the possibility of having non-specialists deliver the programs following training. Furthermore, as caregivers often play a key role as the first advocate and support for autistic children, such parent training programs would not only increase access to services, but also enable caregivers of autistic children in LMICs to be empowered, and collaborative partnerships between families and the wider autism community to become established.

Although LMIC contexts pose greater challenges to the well-being and development of families of autistic children, there is limited evidence for caregiver support in these conditions. The promising results in this study indicate the value of research in LMICs using more robust designs, such as randomized controlled trials, to further build the evidence base of family supports for autism outside Western countries, with the ultimate goal of eliminating disparities in population health. Such research should involve larger sample sizes and further examine the treatment effects of programs that are led by trained non-specialists and whether there are differential effects based on levels of autism severity. Future studies should also investigate the moderators and mediators of such programs in order to better identify what works, for whom, under which circumstances. A number of other experimental designs could be used to establish causal links between critical components and outcomes, such as micro-trials (Howe et al., 2010) and factorial experiments within the MOST framework (Collins et al., 2014). Future studies should also document costs and cost-effectiveness to ensure the best use of limited funds. Observational measures and extended follow-up periods are recommended to reduce the source of bias and assess the long-term impact of such programs.