Impact on learning outcome by effective use of mathematics textbooks structured for increasing academic learning time: Empirical research in El Salvador

Last registered on July 18, 2021

Pre-Trial

Trial Information

General Information

Title
Impact on learning outcome by effective use of mathematics textbooks structured for increasing academic learning time: Empirical research in El Salvador
RCT ID
AEARCTR-0003169
Initial registration date
August 19, 2018
Last updated
July 18, 2021, 6:34 PM EDT

Locations

Primary Investigator

Affiliation
Hiroshima University

Other Primary Investigator(s)

PI Affiliation
Hitotsubashi University

Additional Trial Information

Status
Completed
Start date
2018-01-01
End date
2019-12-31
Secondary IDs
Abstract
This research investigates whether the effective use of mathematics textbooks structured to increase academic learning time improves learning outcome of students of primary and lower secondary education in El Salvador or not. This study targets for the 2nd and the 7th grades students in El Salvador. Cognitive skills of knowing, application and reasoning on all the domains of mathematics are measured by paper test. The students' attitude for learning of mathematics are measured by questionnaire. This study also evaluates whether mentoring and review of teaching practice enhance the behavioral change of teachers on the effective use of the textbooks. The heterogeneity of impact on learning outcome by intervention on students with lower academic performance at the baseline, and with fewer assets at home will be examined.

The textbook and the related material such as teachers' guidebook and exercise book are developed by the Government of El Salvador with technical cooperation by Japan International Cooperation Agency (JICA). The mathematics textbooks divided subject contents and put them in appropriate sequence to assure small-step learning by students. The textbooks are also designed to change the time-use by increasing students' work of problem solving and pair learning in class and teachers' support for students.

As the textbooks are developed from 1st to 12th grade, it is expected to generate cumulative impact for students who use the textbook continuously in grades. Field survey of this study is planned to be continued for two years in 2018 and 2019 to examine the cumulative impact on students of treatment group, compared to students of the control group who use the textbook only for 1 year in 2019.
External Link(s)

Registration Citation

Citation
Kurosaki, Takashi and Takao Maruyama. 2021. "Impact on learning outcome by effective use of mathematics textbooks structured for increasing academic learning time: Empirical research in El Salvador." AEA RCT Registry. July 18. https://doi.org/10.1257/rct.3169-2.2
Former Citation
Kurosaki, Takashi, Takao Maruyama and Takao Maruyama. 2021. "Impact on learning outcome by effective use of mathematics textbooks structured for increasing academic learning time: Empirical research in El Salvador." AEA RCT Registry. July 18. https://www.socialscienceregistry.org/trials/3169/history/96167
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details

Interventions

Intervention(s)
A package of interventions for enhancing effective use of the textbook structured for increasing academic learning time is composed of i) distribution of textbook, teachers' guidebook and exercise book, ii) in-service teacher training, iii) mentoring of mathematics class of teachers by headmaster, iv) review of teaching practice of mathematics. 2nd and 7th grades are targeted for this evaluation. Regarding the 7th grade, as all the junior secondary schools in El Salvador receive the distribution of textbooks and teachers' guidebook of mathematics newly developed, and all the teachers receive in-service training on the use of textbook, the effectiveness of intervention of distribution of exercise book for increasing learning time at home, and that of mentoring and review of teaching practice for changing teachers' behavior will be identified.

Teachers' guidebooks and exercise books are designed, corresponding to each page and topic of textbook. In-service teacher training is conducted for 2 days, and the contents are presentation of composition of the textbook, and demonstration and practice of mathematics class using the textbook. Trainings for headmaster and representatives of parent association are conducted for half a day, and cover the composition of textbook, the method of mentoring and sensitization of importance of learning at home. Review on teaching method of mathematics is conducted for 3 times a year. Teachers bring the result of test of their students conducted at the end of each trimester, and discuss and analyse for improvement of students' learning.
Intervention Start Date
2018-01-01
Intervention End Date
2018-12-31

Primary Outcomes

Primary Outcomes (end points)
Test score of mathematics of students (total score, score of each domain of mathematics, and score of each cognitive skill), the duration of academic learning time in class, motivation and attitude of students for learning mathematics
Primary Outcomes (explanation)
Academic learning time in class is defined as allocated time by teacher for students on problem solving, group or pair work.

Secondary Outcomes

Secondary Outcomes (end points)
Planning and review of annual teaching plan, preparation of teachers for class, time management of teacher at class, support by teachers for enhancing students' learning, subject content knowledge of teachers, learning time of students at home, mentoring activity by headmaster, support by family for enhancing students' learning
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
250 schools are randomly sampled and assigned to treatment group and control group. Each group is composed of 125 schools. The schools where surveyors have a difficulty on conducting field survey because of security concern or physical access are excluded, consulting with the Ministry of Education and Japan International Cooperation Agency in El Salvador. Treatment group receives intervention from January 2018, and control group receives it from January 2019.
Experimental Design Details
Randomization Method
Randomization is done in office by a computer. The method is stratified cluster (one-level) randomization. Stratification variables are rural/urban dummy, and department dummy.
Randomization Unit
Randomization unit is school. If there are several classes at 2nd grade or 7th grade, one of them is randomly selected at each grade.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
250 schools
Sample size: planned number of observations
Around 4,000 students in 2nd grade and 4,000 students in 7th grade, 250 headmasters and teachers
Sample size (or number of clusters) by treatment arms
125 schools for treatment group, and 125 schools for control group
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Minimum Detectable Effect Size for main outcome (test score) is 0.2 SD. It is anticipated that the attrition rate because of security concern in El Salvador is around 10% at the end-line survey.
IRB

Institutional Review Boards (IRBs)

IRB Name
Hitotsubashi University Research Ethics Examination Committee
IRB Approval Date
2018-08-06
IRB Approval Number
2018C004
Analysis Plan

Analysis Plan Documents

/versions/96165/docs/version/document

MD5:

SHA1:

Uploaded At: October 23, 2018

Pre-analysis plan

MD5: 54fadab261fc8a30014719cc09910e4a

SHA1: 81689de23f11d7bfb7ff3eb7c238a83c91c5a169

Uploaded At: October 23, 2018

Post-Trial

Post Trial Information

Study Withdrawal

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Abstract
School enrollment has rapidly increased since 1990 in developing countries at the primary level but the quality of education has stagnated over the years. In teaching and learning practices, textbooks are an important intermediate that links curriculum, teachers, and students. Since textbooks describe the content and methodology of teaching and learning, they can improve teaching and learning practices, if they are carefully designed. This study evaluates the effectiveness of the package of interventions including the distribution of textbooks that are carefully designed to improve student learning in math through a randomized controlled trial in El Salvador. This experiment tracked same students for two years. The average one-year impact of the package on primary school 2nd grade students’ math learning is estimated around 0.48 standard deviation of test scores. The impact was larger on students with higher baseline scores. The average accumulated impact of the first-year interventions one year after is around 0.12 standard deviation. The package of intervention improved math learning of 2nd grade students, and the impact persisted even after schools of the control group also received the package of interventions in the following year.
Citation
Maruyama, T. & Kurosaki, T. (2021). Developing Textbooks to Improve Student Math Learning: Empirical Evidence from El Salvador. JICA Ogata research institute working paper, No.217.
Abstract
While recent debates on educational development focus on the learning crisis in primary education, the crisis in lower secondary education level is equally profound. Around 58 percent of school-age children worldwide enrolled in lower secondary education are not reaching the minimum proficiency level in mathematics. One of the approaches to improve student learning is a structured pedagogy program that provides schools with teaching and learning materials and other related interventions. The impact of teaching and learning materials on student learning depends upon the support of teachers for students.

This study investigates the impact of additional components in a structured pedagogy program that tried to strengthen support of teachers to improve student math learning at the lower secondary level in El Salvador through a randomized controlled trial. The study tracked the same students for two years. While the average one-year impact of the additional component is estimated at around 0.18 standard deviations of test scores, the impact did not persist when the difference of interventions between the treatment and control groups disappeared in the second year of this research. Furthermore, a causal mediation analysis is conducted to investigate the possible causal path of the additional interventions on student math learning.
Citation
Maruyama, T. (2021). Strengthening Teacher Support for Students to Improve Math Learning: Empirical Evidence on A Structured Pedagogy Program in El Salvador. JICA Ogata research institute working paper, No.222.

Reports & Other Materials