Time Spent on Homework and Academic Achievement: A Meta-analysis Study Related to Results of TIMSS

Introduction

Homework is a common part of most students’ school lives (Cooper et al., 2006; Cooper & Valentine, 2001; Epstein & Van Voorhis, 2001; Fernández-Alonso et al., 2019; Kumar, 2006). However, there have been times when it is opposed as much as it is a supported instructional tool because of technological, economic, and cultural events of the related time (Cooper, 2007). These shifts have not reduced the amount of time, effort, and energy that is spent on homework by not only students but also parents, teachers, policymakers, and researchers yet (Cooper et al., 2006; Fan et al., 2017; Fernández-Alonso et al. 2019; Zhu & Leung, 2012). The attention given to homework by the educational stakeholders and researchers thus derives from its importance as an education and teaching tool (Epstein & Van Voorhis, 2001).

Homework is generally considered to facilitate various forms of student development, but researchers have debated its impact on students’ academic achievement for more than four decades (Cooper, 1989; Cooper & Valentine, 2001; Maltese et al. 2012; Scheerens et al., 2013; Trautwein, 2007; Trautwein & Köller, 2003). Not only have researchers addressed the homework-achievement relation through individual studies, but also they have tried to present an understanding about it by synthesizing them. However, it could be asserted that there has still been a gap in homework research owing to limitations of previous studies and inconsistent results. Most of these studies examined homework-achievement relationships in general (without considering subject differences in homework), and few of them dealt with science courses (Cooper et al., 2006; Fan et al., 2017). Also, achievement was measured through the results of national and non-standard tests, findings of individual studies, or an international standard test that belonged to only one period. Additionally, their sampling may not have been representative, and the majority of studies did not address the moderating role of culture. Finally, some studies revealed the positive and significant effect of homework on achievement (Cheema & Sheridan, 2015; Cool & Keith, 1991; Cooper, 1989; Cooper et al. 2006; Fan et al., 2017; Fernández-Alonso et al., 2015; Fernández-Alonso et al., 2019; Gustafsson, 2013; Keith & Cool, 1992), though the others indicated negative or no relations between these two concepts (De Jong et al., 2000; Kitsantas et al., 2011; Trautwein, 2007). Thus, this meta-analysis research is intended to make a significant contribution to the homework-achievement research deriving data from a periodic internal exam that provides more representative and diverse data on both sampling and potential moderators. The article first reviews literature about homework. Next, studies with their wide-ranging implication were drawn from to understand the influence of homework on achievement. Finally, we present the findings of our meta-analysis and discussion of these findings in relation to other studies, bringing a new perspective to this topic.

Literature about Homework

Homework can be defined as “tasks assigned to students by school teachers to be carried out during non-school hours” (Cooper, 1989, p. 7). It can be distinguished from other educational activities with the help of its characteristics: (i) it is performed in the absence of the teacher (Hong & Milgram, 2000), (ii) it is a purely academic activities, and (iii) its contents and the parameters of the instructional activities are determined by teachers (Cooper, 1989; Cooper & Valentine, 2001; Trautwein & Köller, 2003). Given these properties, homework requires extra time, energy, and effort by teachers, students, and parents (Trautwein et al., 2006; Trautwein et al., 2009). Whether the students receive a worthwhile return for these investments is a crucial issue (Cooper, 1989; Cooper et al., 2006; Maltese et al., 2012).

Conflicts among educational stakeholders and researchers about the outcomes of students’ homework have been going on for a long time (Cooper, 1989, 1991; Cooper et al., 1998; Cooper et al., 2006; Zhu & Leung, 2012). On the one hand, engaging in instructional activities outside of school time limits the time available to students for leisure activities (Alanne & Macgregor, 2007; Cooper, 1989; Fleischer & Ohel, 1974). For students, it results in boredom, fatigue, negative feelings such as tension, anxiety, and negative attitude towards school (Alanne & Macgregor, 2007; Cooper, 1989; Fleischer & Ohel, 1974). On the other hand, the learning process is assumed to continue as long as they interact with teaching materials (Walberg & Paschal, 1995). As their interaction with homework increases, their understanding, thinking skills, and retention of knowledge will improve (Cooper, 2007). Additionally, by doing homework, students can gain self-direction, self-discipline, time management skills (Brewster & Fager, 2000; Cooper, 1989, 2007; Corno, 2000; Epstein & Van Voorhis, 2001; Hetherington, 2005), problem-solving skills, and inquisitiveness (Cooper, 2007).

Concerning its academic outcomes of homework, it has long been unclear whether more time spent on homework equates to increased achievement for students. There is, therefore, a continuing interest in homework research. Individual studies related to homework-achievement research have provided valuable contributions despite their contradictory results. One possible explanation of these contradictory results could be variations in the type of homework studied, its frequency, and amount of effort spent on it. Variations in achievement indicators used, such as standardized and non-standardized test scores, could affect the results (Trautwein, 2007). In addition, national characteristics that influence the view of homework and its practice could cause differences in results (Cooper, 1989), as could socio-economic changes that affect educational needs and activities (Cooper, 2007). Based on these factors and related inconsistencies, the research of Cooper (1989), Cooper et al. (2006), and Fan et al. (2017) synthesized the individual studies in the literature to understand contradictory results.

Cooper (1989) reviewed 50 correlation studies on the relationship between time spent on homework and achievement. Forty-three of them revealed that students spending more time on homework were more successful than peers or vice versa. The researcher found the overall effect was to d = 0.21, despite the different amount of the relation among students at different grade levels. Similarly, Cooper et al. (2006) summarized the studies on this topic from 1987 to 2003 in the USA. The researches grouped the studies by taking into consideration their research designs. All research designs showed a relationship between homework and achievement, and 50 out of 69 correlations were in positive direction. Additionally, the meta-analysis of Fan et al. (2017) discussed the relationship between time on homework and achievement through several homework indicators in addition to time spent on it as distinct from the studies of Cooper (1989) and Cooper et al. (2006). They revealed that all homework indicators, including time on homework, affected achievement.

All three studies revealed time spent on homework is positively related to achievement, though they reported different levels of relation. These differences included student grades, nationalities, and subject contents. For example, Cooper (1989) concluded that the effect increased with grade level (.15 for the 4-6th grade, .31 for the 7-9th grade). Moreover, the amount of relations has varied across countries. Fan et al. (2017) concluded that its influence on Asian students was weaker than on US students (.283 for US students, .075 for Asian students). Finally, Cooper et al. (2006) concluded that a small effect size difference was observed between reading and mathematics as Fan et al. (2017) reached similar results when comparing the effect sizes between mathematics and science (.209 and .233). However, they advised caution in interpreting these findings, due to insufficient data across different subjects.

These studies have made a valuable contribution to homework literature and have alerted education stakeholders and researchers to its importance. However, the effect of time spent on homework on achievement, and moderators playing a role in this effect have not been completely clarified (Dettmers et al., 2009). There are some possible moderators such as culture that have not been considered yet. Additionally, earlier studies used limited data related to different subjects, especially science (Cooper et al., 2006; Fan et al., 2017). Moreover, as achievement indicators, these studies used findings of individual studies or limited data related to achievement that were only standard achievement test results from one country or a single standard achievement test results from different countries. A comprehensive understanding of this issue is needed, rather than more small-scale studies, or syntheses of these studies from the literature. This need will be addressed in the current study designed by using the results of a periodic international standardized exam performed over a long time. Analysis of TIMSS results provides us with more representative sampling and diverse potential moderators. Furthermore, TIMSS’ validity and reliability (Joncas & Foy, 2011) contributes to the present research in terms of these aspects. As a result, the determination of the amount and direction of the possible relationship and its significant moderators might encourage students, teachers, parents, and education policymakers to review their understanding and practice about homework.

Purpose of the Study

The current study examined the effect of the amount of time spent on homework on TIMSS achievements of students. The aim of this study was twofold: (a) to determine the overall effect size of the amount of time spent on homework on students’ achievements and (b) to examine if culture, grade level, subject matter, and time played significant moderator roles in this effect with an internationally perspective.

To expand and extend studies on this topic concerning data and moderator diversity, it is beneficial to use data obtained from the internationally representative sample at different times. In this study, data including five achievement test results (TIMSS) and demographic questions about the amount of time students spend on homework were analyzed. For this purpose, the following hypotheses were developed:

• H1: The amount of time spent on homework affected students’ academic achievement.
• H2: Culture was a moderator in the effect of the amount of time spent on homework on achievement.
• H3: Grade was a moderator in the effect of the amount of the time spent on homework on achievement.
• H4: Subject matter was a moderator of the effect of amount of time spent on homework on achievement.
• H5: Year was a moderator in the effect of the amount of time spent on homework on achievement.

Method

Meta-analysis aims to summarize results from several individual studies to evaluate differences in the results among studies, to overcome limitations of small sample sizes of individual studies, to increase precision in estimating effects, to interpret the effects in subsets of patients, and to determine if new studies are needed further examination of a topic (Hernandez, 2009).

This study aimed to examine the effect of time spent on homework on academic achievement comprehensively; therefore, all TIMSS data from 1999 to 2015 needed to be combined for the analysis process. It has been performed seven times because of its four-year period. There were too many independent studies that included large samples. So, the meta-analysis was seen as more appropriate to analyze this aggregated data than student-level data analysis.

Study’s Sample and Selection Criteria

The sample of this study included students who participated in TIMSS exams from 1999 to 2015 years. TIMSS has been performed for 4^th and 8^th grade students by the International Association for the Evaluation of Educational Achievement (IEA) in four year cycles. It has evaluated achievement in mathematics and science courses at an international context. Additionally, it has asked demographic questions, such as how much time they spent on doing homework. TIMSS has used a two-stage stratified cluster as a sample design, that is, firstly, schools are determined, then one or two classrooms from 4^th and 8^th grades in these schools are included the sample.

The researcher accessed the website of the International Association for the Evaluation of Educational Achievement in May 2020. As a result, the researcher gathered data from 488 independent results from the eight surveys of TIMSS (1995, 1999, 2003, 2007, 2011, and 2015). But data of 1995 were excluded because no results were given for the students who were in the least homework time group. Finally, a sample group of 429,970 students was obtained for this study; 225,430 of them were fourth-grade students and 204,540 were eight grade students.

Procedure

In planning and conducting the process, the five steps of Pigott (2012) were applied. These steps include (1) determining the information taken from a study included in the meta-analysis, (2) choosing the models for a meta-analysis, (3) identifying possible confounding of moderators in the analyses, (4) performing the analyses, (5) interpreting the results. For the first step, a coding form was prepared for collection and analysis of the necessary information from individual studies. Next, the appropiate meta-analysis model was chosen, that is, random or fixed models based on the aim of the research and the properties of data. Thirdly the possible moderators were determined based on the context of the topic and results of previous studies. Fourthly, the meta-analysis was conducted through the Comprehensive Meta-Analysis Program. Finally, the results of the analysis were presented through a table that enables holistically evaluate findings.

Coding Process

The coding process is crucial part in meta-analysis. Dincer (2014) points out the accuracy of the analysis and interpretation process is based on how coding process is performed. Therefore, the researcher should spend much time on coding process of meta-analysis studies because this kind of studies, even small ones, include complex data needed to interpret. Depending on research questions, the information extracted from the studies is determined in the coding process (Pigot, 2012). It was considered that preparing a coding form was beneficial in this process in regards to the hypotheses of this research, and all studies were reviewed and coded through this coding form. The components of the coding form included:

• Sample information [year of study, country, subject matter, class.].
• Quantitative values [sample size, mean, standard deviation, etc.].

Inclusion and Exclusion Criteria

In meta-analysis studies, it is necessary to determine the primary studies that have been included before analyzing the data. In accordance with the characteristics of the data, three criteria for inclusion and exclusion of the studies in the analysis were defined as follows:

• The primary study must include information about sample size and mean in both less time spent on homework group and medium time spent on homework group. If the primary study presented no information about sample size and mean of either of the two groups, it would be excluded. For instance, in 2003, the primary study of the Russian Federation did not state the data at 8th grade on the mean for less time spent on homework group, and for this reason, it was excluded in the study list.
• The primary study must consist of data about mathematics and science achievement. If primary studies presented data related to sub-branches of mathematics and science, it would not be included owing to the possible derivation effect on the results. For instance, in 2015, the primary study of Kazakhstan was related to achievement in biology, chemistry, physics, and earth sciences and, for this reason, it was excluded from the study list.
• Primary studies must indicate data concerning the moderators of the present study, such as country, course, year, and grade level. TIMSS results provided the necessary data about these moderators systemically. Therefore, no primary studies were excluded from the study list.

As a result, 603 primary studies were determined at the beginning of the coding process. After applying the first inclusion criteria, 27 primary studies were excluded, and 576 primary studies remained. Then, the rest of the primary studies were evaluated in terms of second criteria, and then 488 out of 576 primary studies were included in the study list. Finally, it was observed that all the remained primary studies were appropriate to the third criteria, and the meta-analysis study was conducted with 488 primary studies.

Effect Size Analysis

The term named as effect size has been used in social science meta-analyses. It refers to the index representing the amount and direction of the relationship between variables or a difference between two groups (Borenstein et al., 2009, p. 17).

In this study, the standardized mean difference (based on Cohen’s, 1969 d) was used due to the aim of the study, which was a comparison of independent groups (Hedges & Olkin, 1985). Cohen’s d coefficient has enabled to compare the results of the studies in which different questionnaires and scales have been used, especially in educational sciences (Borenstein et al., 2009). Finally, the model used in combining the studies in the meta-analysis process was determined as a random-effects model rather than fixed effect model that has allowed the evaluation of the same ρ (or δ) value underlies all studies in the meta-analysis (Hunter & Schmidt, 2014). The properties of the studies were convenient to the preconditions of random-effects model (Borenstein et al., 2009; Hedges & Olkin, 1985; Littel et al., 2008). This model has permitted to evaluate the possibility that population parameters (ρ or δ values) differ from study to study (Hunter & Schmidt, 2014). The analysis was conducted through the Comprehensive Meta-analysis program.

Moderator Analysis

Moderator analysis enables us to understand the association of differences between subgroups, or between variables (moderators) with the effect size (Cooper, 2017). Littel et al. (2008) explained the term as it “…explores variations in effect size (ES) for different groups created by methodological features and PICO (populations, interventions, comparisons, and outcomes) variables.” (p. 111). Furthermore, Q statistic method developed by Hedges and Olkin (1985) was used to determine the statistical significance of moderator variables. There are two types of Q value as Qbetween[Qb] and Qwithin[Qw]. On the one hand, Qb is used to test whether the average effects from the two groupings are homogenous (Cooper, 2017, p. 239). On the other hand, Qw is used to test whether the average effect of a moderator is homogenous in itself (Kulinskaya et al., 2008). In this study, Qw is used to determine homogeneity of the average effects of the amount of time spent on homework on academic achievement, while Qb is used to determine homogeneity of the average effects of four moderator variables as culture and year in which the research was conducted, subject matters, as well as the grade level of students.

Variables

Academic achievement. Data related to the academic achievement of the students were obtained from TIMSS [Trends in International Mathematics and Science Study] results. TIMSS exams conducted by the International Association for the Evaluation of Educational Achievement (IEA) internationally include questions to determine the achievement of 4th and 8th-grade students in mathematics and science every four years for twenty-five years. These exams provide representative, reliable, and valid databases due to rigorous school and classroom sampling techniques (Joncas & Foy, 2011).

Homework. The correlation between homework and achievement has been discussed in the literature from different aspects. Frequency of homework, effort spent on homework and the time spent on homework have been variables used in studies on homework-achievement relation. In this study, in line with the learning process continuing as long as the student interacts with teaching materials, time spent on homework was handled during the investigation of the relationship between homework and academic achievement. Time spent on homework is a part of the information which TIMSS database covers, such as background knowledge about students, teachers, and administrators. TIMSS presents an index of the amount of time students spent on homework, constructs three categories (high, medium, and low) through its frequency, and amounts their teachers assigned each week. In this study, the two categories (low and medium) were used, because the number of students in high categories was limited, especially at 4th-grade results. It was thought that using the data related to the high category may have caused publication bias, so this category was disregarded.

Moderator Variables

When the studies in the literature were examined, the impact of time spent on homework on academic achievement was mediated by variables such as culture, grade level, subject matter, and exam year. Detailed information about moderator variables is presented below.

Culture classification. As discussed above, studies about homework suggest that homework practices vary across countries in terms of homework frequency and time spent on homework (Chen & Stevenson, 1989; Tam & Chan, 2009; Zhu, 2015; Zhu & Leung, 2012). Fan et al. (2017) has stated that its effect on academic achievement differs across geographical regions. One possible explanation may be that the culture of a country correlates with the effect sizes of homework on achievement, since countries, regions, and cultures are crucial factors in terms of educational practices such as homework (Dettmers et al., 2009; Fan et al., 2017; Fernández-Alonso et al., 2019), owing to the effect of shared elements on the perception of some concepts (Triandis, 1995). Additionally, perception of achievement is related to the social structure of the nation (Palardy et al., 2015; Sirin, 2005). There are several studies about the role of culture in the homework-achievement relation. However, the number of them was very limited to compare them, and their role was not known completely (Cooper et al., 2006; Fan et al., 2017; Fernández-Alonso et al., 2019). For this reason, the moderator role of culture in the effect of homework on achievement needs to be discussed. So, a cultural classification is needed and vertical-collectivism and horizontal-individualism culture classification of Triandis (1995) was based on the forming of the culture moderator. It could be impossible to make static classification for human beings. However, cultural attributes could be beneficial to interpret and to anticipate people’s social behaviors (Singelis et al., 1995). In Triandis’ classification, the researcher grouped cultures according to two concepts as perceiving self and equality. In vertical-collectivism culture, the importance of respecting the society, being a member of a group, and loyalty to society has been imposed on children soon after their birth (Hofstede, 1994). On the other hand, the person in a horizontal-individualism culture perceives the self as an autonomous individual, and all people in this culture have equal status. In other aspects, in countries such as Chile, China, Egypt, or Japan, that are in the vertical-collectivism group, the goals of people coincide with their groups though in countries like Netherlands, England, and Switzerland, that are located in the horizontal-individualism group, people have personal goals regardless of the overlap with their groups (Triandis, 1995).

Grade level. Students’ age can be a factor when the amount, length, and purpose of homework is determined, due to the effect of the developmental level. Moreover, their ages are relevant in studying habits and attendance to stimuli (Zhu & Leung, 2012). Therefore, its effect on academic achievement can vary among students’ ages. Previous studies on this topic indicate that the grade level of students moderated the relationship between homework and achievement (Cooper, 1989; Cooper et al., 2006; Gustafsson, 2013). Therefore, the fact that the relationship between these terms should be tested through more representative data could be beneficial. In this study, the grade level moderator was grouped as 4th and 8th-grade because TIMSS exams are applied to these two grade students.

Subject matter. As stated before, many studies in the literature have not dealt with the linkage between homework and academic achievement according to subject matters. However, Cooper et al. (2006) revealed that subject matters might have an effective role in homework’s effect despite a limited number of research on some subject matters. In light of these findings, the moderator role of subject matters is necessary to investigate through extensive sampling. In this study, the subject matter moderator was formed as science and mathematics, for the achievement in science and mathematics has been measured in TIMSS exams.

The exam year. Perception of the public on homework is inconsistent in years. Cooper et al. (2006) stated that the public viewed homework as a useless educational tool in the 1940s; on the other hand, this attitude changed to more positive aspects in the late 1950s. So, the exam year can be a potential moderator in the effect sizes of homework on achievement.

Results

Publication Bias

One important issue in meta-analysis studies is sample bias. Borenstein et al. (2009) stated that when there is any bias in the studies included in the analysis, this bias reflects in the meta-analysis study. The funnel plot and trim and fill test can be used to evaluate whether there was publication bias of research (Kulinskaya et al., 2008). In this study, the funnel graph of the studies in the meta-analysis is presented in Figure 1. The funnel plot is not asymmetric and does not distribute on one side of the line showing the effect size and it could be asserted that there was no publication bias (Pigott, 2012).

Besides the funnel plot, the trim and fill test was performed to evaluate the amount publication bias and its results was presented in Table 1. According to Table 1, it could be said that there was not any publication bias.

Results of Mean Effect Size and Moderator Variables

Meta-analysis results showing the effect of the time students devote to homework on ‘academic achievement’ are presented in Table 2.

Firstly, it was observed that the findings supported hypothesis H1 that the amount of time spent on homework had an impact on students’ academic achievement (Q = 3181.056, p < .000). The effect value of time spent on homework on success was calculated as d = 0.186, and it was statistically significant. This impact value showed that the amount of time spent on homework has a low and significant impact on students’ academic achievement (see Cohen, 1988). This finding indicated that students who spend moderate time on homework have higher academic achievement than students spending little time on homework.

Secondly, after the moderator analysis, it was observed that hypothesis H2, that the culture of the country (vertical-collective culture and horizontal-individualist culture) in which the research was conducted played a role as a moderator of the effect of homework on students’ academic success, was supported (Q_b = 11.335, p < .001). In the moderator analysis, the effect of time spent on homework on success was statistically significantly higher in vertical-collectivist cultures (d = 0.258) than in horizontal-individualist cultures (d = 0.047).

Thirdly, after the moderator analysis, hypothesis H3, related to the moderator role of the students’ grade level (4^th- 8^th grades) in the time spent on homework- achievement relation (Qb = 26.813, p < .000), was accepted. In the moderator analysis, the effect of the amount of time spent on homework on the students’ achievement is statistically significantly higher at the eighth-grade level (d = 0.256) compared to at the fourth-grade level (d = -0.057).

Fourthy, it was observed that hypothesis H4, that dealt with the moderator role of subject matter (Science-Mathematics) in the effect of the amount of time spent on homework on the students’ academic achievement, was supported (Qb = 76,280, p < .00). The amount of time spent on homework had a lower impact on success in science (d = -0.009) than that in mathematics (d = 0.358).

Finally, it was observed that the H₅ hypothesis that the year (1999, 2001, 2003, 2007, 2011, 2015) played a role as a moderator in the effect of the amount of time spent on homework on academic achievement was accepted (Qb = 84.335, p < .00). In the moderator analysis, the effect of the amount of time spent on homework on success appeared to be statistically significant, showing an increase from 1999 (d = -0.270), 2003 (d = 0.036), 2007 (d = 0.251), 2011 (d = 0.439) to 2015 (d = 0.525).

Summarizing, the current investigation examined whether the amount of time spent on homework affected students’ academic achievement and investigated some variables that may moderate the relationship between homework and achievement through the meta-analysis of TIMSS data. These moderator variables included culture (vertical-collective culture and horizontal-individualist culture), grade level (4th vs. 8th-grade), subject matter (mathematics vs. science), and exam year (1999, 2003, 2007, 2011 vs. 2015). In this context, five hypotheses were formed and tested, and the findings obtained after the analysis process was summarized in this part of the study. The first hypothesis was concerned whether the amount of time spent on homework affected students’ academic achievement, and it was supported, that is, students who spent a medium amount of the time on homework were more successful than students spending less amount of time on homework in TIMSS exams. Moreover, the second hypothesis was concerned whether national culture (vertical-collective culture vs. horizontal-individualist culture) played a moderator role, and it was supported. In other words, the effect of homework time on academic achievement was higher in countries with vertical-collective culture than in those with horizontal-individualist culture. The third hypothesis was related to whether the grade of the student who participated in this exam was a moderator and this too was supported. According to this, the effect of time spent on homework on achievement was higher for 8th-grade students than 4th-grade students. The fourth hypothesis was about whether the type of the course in which achievement measured was a moderator, and it was supported. In other words, the effect of time spent on homework on achievement was higher for mathematics course than science course. Finally, the last hypothesis concerned whether the year in which success measured was a moderator, and it was supported. The effect of time spent on homework on achievement was the highest in 2015 and the least in 1999. All these results are summarized in Table 3.

Discussion

Homework is a universal phenomenon, but all students experience it differently. Not enough attention has been paid to homework in the research literature (Hong & Milgram, 2000). This study aimed to investigate whether the amount of time spent on homework affected the academic achievement of students and to determine the moderators in this probable relationship between them through the meta-analysis of TIMSS data.

Overall, the data of this study revealed that the first hypothesis, which was the amount of time spent on homework that affected the academic achievement of students, was supported. Its effect size was found to be low, but statically significant. This result corresponded to the studies of Cooper (1989), Fan et al. (2017), Fernández-Alonso et al. (2019), Gustafsson (2013), Cheema and Sheridan (2015), Cool and Keith (1991), Keith and Cool (1992), Cooper et al. (2006), and Trautwein and Köller (2003). From this, we infer that academic achievement could be improved by practicing skills and knowledge at non-school hours, and coming to school with prior knowledge obtained apart from school times. Similarly, Trautwein et al. (2002) stated that “time on task” increased students’ academic performance. Eren and Henderson (2008) commented that learning by doing improved students’ achievement as well. Cool and Keith (1991) interpreted this result as the relationship between study habits and students’ success. Researchers stated that successful students were assigned more homework, and homework enabled beneficial influence on their later achievement. But the studies of Falch and Rønning (2012) and Eren and Henderson (2011) revealed there was a modest or large level effect. These different results might derive from the contexts of them because they researched only mathematics achievement. Another possible explanation of the low effect size in this study could be that successful students completed more homework than the others, and its direct effect on their academic achievement was not able to be observed (Cooper, 2007). Additionally, the differences could be dependent on the fact that the amount of time spent on homework affected by many other variables.

Homework is a kind of individual study technique, and it might, therefore, be claimed that its academic effect depends on the extent conditions in which students did homework were conducive to their learning style. “Learning style consists of a unique combination of strengths and weaknesses on elements that reflect various aspects of the environmental, emotional, sociological, and physical conditions under which a person acquires new knowledge and skills.” (Hong & Milgram, 2000, p.7-8). In other words, excessive time spent on homework might indicate that students do homework slowly due to different reasons such as its complexity, its type, lack of resources for completing it and parental help, their prior knowledge required, conditions of the place where they do homework, their concentration and morale levels (Cool & Keith, 1991; Keith & Cool, 1992; Cooper, 1989; Gustafsson, 2013; Jha, 2006; Rønning, 2010; Trautwein, 2007; Trautwein et al., 2002). The weaker or low-ability students might have difficulty in completing homework, and it could take a longer time (Keith, 1982; Trautwein, 2007; Trautwein et al., 2002). Too much time spent on homework might result in a decrease in the motivation of students and might cause exhaustion (Cooper, 2007; Cooper et al., 1998; Trautwein, 2007). On the other hand, some distractive behaviors, such as watching TV and talking on the phone, could cause spending a longer time on homework (Hong & Milgram, 2000). Furthermore, Fernández-Alonso et al (2019) confirmed that teachers’ homework policy played a significant role in the homework-achievement relation. Teachers might use homework to compensate for topics they could not teach in the lessons rather than to reinforce students’ learning or they assigned useless and time consuming homework that does not support learning (Smith, 2003; Trautwein et al., 2002). Homework, which aims to practice the elements of same-day instruction, can require less time than the homework, including new materials related to the next day lessons (Trautwein et al., 2002). Teachers may assign homework not for only instructional purpose but also for non-instructional purposes (Cooper et al., 2006). Additionally, parental help may ease completing homework (Cooper, 2007); thus, this has decreased time spent on homework (Rønning, 2010). Furthermore, home environment conditions, such as space, light, quietness, and materials, can facilitate or hinder doing homework (Cooper, 2007; Jha, 2006). Lastly, the effect of homework on students’ academic achievement would be larger if it is measured through their grades rather than standardized test scores, as the study of Cooper et al. (1998), who concluded that a teacher’s assignment style and grading style might be related to the amount of the homework effect on achievement. This could indicate that the effect of homework is observed more in achievement in nonstandard exams rather than that in standard exams such as TIMSS.

The analysis for the moderator variable of culture revealed that the culture played a moderator role. It was observed that the effect size in horizontal-individualist culture had a significant and positive, but smaller mean effect size, than those in vertical-collective culture. In line with the studies of Dettmers et al. (2009) and Falch and Rønning (2012), the relationship between homework and achievement may differ across countries. Chen and Stevenson (1989) pointed out that the quantity of homework and time spent on homework was varied between China, Japan, and America. Furthermore, Jha (2006) reported that the amount of homework time depended on cultural obligations. A possible explanation was that the students in vertical-collective culture perveived the self as primarily a member of the societal group, so they may have felt an obligation to obey school rules and to do their homework. Additionally, the social capital and socio-economic conditions played a key role in line with the studies of Fernández-Alonso et al. (2019), Palardy et al. (2015), and Sirin (2005). The researchers pointed out the socio-economic structure could be determinative for academic achievement of the students in terms of their educational opportunity, such as home resources and the instructional quality of their schools. Apart from the socioeconomic structure-academic achievement relation, Palardy et al. (2015) concluded that socio-economic structure and racial/ethnic characteristics were associated with distractive learning behaviors. Furthermore, Fernández-Alonso et al. (2019) stated that the effect of time spent on homework is differentiated across countries. In this study, social structure of the countries involved in TIMSS might have an effect on the perception of education, its practice, and academic achievement, and this effect could reflect on the importance that countries gave on homework and achievement. But the findings of the research by Fan et al. (2017), which indicate that the effect of homework on achievement was stronger for US students than Asian students, contradict this. This contradiction could be explained by the fact that data in this study were more representative in terms of cultural diversity.

Concerning grade level, the analysis showed this to be a significant moderator variable, and the effect size in 8th-grade students was larger than in 4th-grade students. In other words, the effect of homework time and achievement was significantly stronger for 8th-grade students who spent time on homework at the medium level than for those in 4th-grade. This finding was in line with previous studies finding that middle school students experienced a more positive effect than elementary school students (Cooper, 2007; Eren & Henderson, 2011). One possible explanation was that younger students were less able to ignore irrelevant stimuli, less developed study habits, controlling their learning by themselves, and paying attention to a task than older students (Cooper, 2007; Xu, 2009). Additionally, the aim of homework for younger students may have been to develop a positive attitude and study habits, whilst for older students the aim was to reinforce their academic knowledge (Cooper, 1989, 2007). It could thus be asserted that skills in managing these factors, findings of cognitive psychology, and purposes of homework affected the amount of time spent on it and its academic gains. Also, the majority of the students in 8th would have been preparing for high school entrance exams, especially in countries having a competitive education system. They would, therefore, have been assigned more homework and spent more time on it compared to the students at 4^th grade. To sum up, the effect on homework time might be related to unobserved characteristics of teachers and students (Eren & Henderson, 2008; Falch & Rønning, 2012).

Concerning subject matter, the impact of homework time on academic achievement was moderated by it. This impact was stronger for achievement in mathematics than that in science. This result was consistent with the findings of Falch & Rønning (2012), which argued the effect of mathematics homework was greater than in other subjects. It might be the case that students spent relatively more time on mathematics homework than other assignments; that is, they allocated their homework time for mathematics assignments, perhaps from one-fifth to two fifths (Kitsantas et al., 2011; Pezdek et al., 2002). However, Cooper et al.’s (2006) stated that the relationship between homework and achievement did not vary across lessons. A possible explanation of this different result could be relatively few studies about homework-science achievement included in the analysis, owing to a limited number of studies on this topic in the literature.

With respect to exam year, the analysis in this study found that average effect sizes of five categories (i.e., 1999, 2003, 2007, 2011, 2015) were significantly different from each other; that from 2015 was the highest and that from 1999 were the lowest. One reason why the effect of homework has varied from time to time could be changing attitudes to homework. Cooper et al. (1998) stated that the attitude towards homework was getting more positive. It could be claimed that this positive aspect may enable to be given importance to homework in terms of teachers, students, and parents. Students and parents might be paying more attention to completing better qualitative homework. Teachers have been getting more interested in giving more beneficial homework improving academic achievement of students.

Finally, the current study, thus, make a valuable contribution to empirical research literature concerning the association between homework and achievement. It might encourage researchers to delve deeper into an area where there have been no or few studies. Its findings and their generalizability are robust, owing to having more representative sampling (data from 74 countries), and moderator diversity than the other meta-analysis studies. As previous studies, it included primary studies conducted only in the USA, or written in English. Moreover, they used limited studies on science courses because they synthesized the research on the literature, and the number of the research on science courses was limited. Finally, the moderator role of culture has not been considered in previous studies. As a result, the present study might be beneficial in providing a comprehensive understanding of the homework-achievement relation, and it could help to maximize the effect of homework on students’ academic development.

Limitations

It was necessary to point out the limitations due to the properties of TIMSS data. Firstly, time spent on homework was classified by TIMSS executives, which therefore, hindered more detailed analyses. Secondly, there were no data related to the gender of the students, other homework indicators such as effort on homework and its types, so these moderators could not be analyzed. Consequently, conducting relevant studies with different research designs, such as multi-level analysis, would provide a better understanding of the relationship between homework and achievement. Thirdly, the academic achievement in mathematics and science has been measured in TIMSS. Therefore, the moderator role of the other subject matters could not be determined. The results of other international exams, such as PIRLS and PISA, could be used for future research. Lastly, qualitative studies addressing the time spent on homework-achievement in different cultures, courses, and in all grade levels in schooling could be highly informative to an understanding of this topic.