Impact on learning outcome by effective use of mathematics textbooks structured for increasing academic learning time: Empirical research in El Salvador

Last registered on November 02, 2023

Pre-Trial

Trial Information

General Information

Title
Impact on learning outcome by effective use of mathematics textbooks structured for increasing academic learning time: Empirical research in El Salvador
RCT ID
AEARCTR-0003169
Initial registration date
August 19, 2018

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
August 21, 2018, 12:41 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
November 02, 2023, 2:59 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Primary Investigator

Affiliation
Japan International Cooperation Agency

Other Primary Investigator(s)

PI Affiliation
Hitotsubashi University

Additional Trial Information

Status
Completed
Start date
2018-01-01
End date
2019-12-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This research investigates whether the effective use of mathematics textbooks structured to increase academic learning time improves learning outcome of students of primary and lower secondary education in El Salvador or not. This study targets for the 2nd and the 7th grades students in El Salvador. Cognitive skills of knowing, application and reasoning on all the domains of mathematics are measured by paper test. The students' attitude for learning of mathematics are measured by questionnaire. This study also evaluates whether mentoring and review of teaching practice enhance the behavioral change of teachers on the effective use of the textbooks. The heterogeneity of impact on learning outcome by intervention on students with lower academic performance at the baseline, and with fewer assets at home will be examined.

The textbook and the related material such as teachers' guidebook and exercise book are developed by the Government of El Salvador with technical cooperation by Japan International Cooperation Agency (JICA). The mathematics textbooks divided subject contents and put them in appropriate sequence to assure small-step learning by students. The textbooks are also designed to change the time-use by increasing students' work of problem solving and pair learning in class and teachers' support for students.

As the textbooks are developed from 1st to 12th grade, it is expected to generate cumulative impact for students who use the textbook continuously in grades. Field survey of this study is planned to be continued for two years in 2018 and 2019 to examine the cumulative impact on students of treatment group, compared to students of the control group who use the textbook only for 1 year in 2019.
External Link(s)

Registration Citation

Citation
Kurosaki, Takashi and Takao Maruyama. 2023. "Impact on learning outcome by effective use of mathematics textbooks structured for increasing academic learning time: Empirical research in El Salvador." AEA RCT Registry. November 02. https://doi.org/10.1257/rct.3169-2.4
Former Citation
Kurosaki, Takashi and Takao Maruyama. 2023. "Impact on learning outcome by effective use of mathematics textbooks structured for increasing academic learning time: Empirical research in El Salvador." AEA RCT Registry. November 02. https://www.socialscienceregistry.org/trials/3169/history/199505
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
A package of interventions for enhancing effective use of the textbook structured for increasing academic learning time is composed of i) distribution of textbook, teachers' guidebook and exercise book, ii) in-service teacher training, iii) mentoring of mathematics class of teachers by headmaster, iv) review of teaching practice of mathematics. 2nd and 7th grades are targeted for this evaluation. Regarding the 7th grade, as all the junior secondary schools in El Salvador receive the distribution of textbooks and teachers' guidebook of mathematics newly developed, and all the teachers receive in-service training on the use of textbook, the effectiveness of intervention of distribution of exercise book for increasing learning time at home, and that of mentoring and review of teaching practice for changing teachers' behavior will be identified.

Teachers' guidebooks and exercise books are designed, corresponding to each page and topic of textbook. In-service teacher training is conducted for 2 days, and the contents are presentation of composition of the textbook, and demonstration and practice of mathematics class using the textbook. Trainings for headmaster and representatives of parent association are conducted for half a day, and cover the composition of textbook, the method of mentoring and sensitization of importance of learning at home. Review on teaching method of mathematics is conducted for 3 times a year. Teachers bring the result of test of their students conducted at the end of each trimester, and discuss and analyse for improvement of students' learning.
Intervention Start Date
2018-01-01
Intervention End Date
2019-11-01

Primary Outcomes

Primary Outcomes (end points)
Test score of mathematics of students (total score, score of each domain of mathematics, and score of each cognitive skill), the duration of academic learning time in class, motivation and attitude of students for learning mathematics
Primary Outcomes (explanation)
Academic learning time in class is defined as allocated time by teacher for students on problem solving, group or pair work.

Secondary Outcomes

Secondary Outcomes (end points)
Planning and review of annual teaching plan, preparation of teachers for class, time management of teacher at class, support by teachers for enhancing students' learning, subject content knowledge of teachers, learning time of students at home, mentoring activity by headmaster, support by family for enhancing students' learning
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
250 schools are randomly sampled and assigned to treatment group and control group. Each group is composed of 125 schools. The schools where surveyors have a difficulty on conducting field survey because of security concern or physical access are excluded, consulting with the Ministry of Education and Japan International Cooperation Agency in El Salvador. Treatment group receives intervention from January 2018, and control group receives it from January 2019.
Experimental Design Details
Randomization Method
Randomization is done in office by a computer. The method is stratified cluster (one-level) randomization. Stratification variables are rural/urban dummy, and department dummy.
Randomization Unit
Randomization unit is school. If there are several classes at 2nd grade or 7th grade, one of them is randomly selected at each grade.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
250 schools
Sample size: planned number of observations
Around 4,000 students in 2nd grade and 4,000 students in 7th grade, 250 headmasters and teachers
Sample size (or number of clusters) by treatment arms
125 schools for treatment group, and 125 schools for control group
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Minimum Detectable Effect Size for main outcome (test score) is 0.2 SD. It is anticipated that the attrition rate because of security concern in El Salvador is around 10% at the end-line survey.
IRB

Institutional Review Boards (IRBs)

IRB Name
Hitotsubashi University Research Ethics Examination Committee
IRB Approval Date
2018-08-06
IRB Approval Number
2018C004
Analysis Plan

Analysis Plan Documents

Pre-analysis plan

MD5: 54fadab261fc8a30014719cc09910e4a

SHA1: 81689de23f11d7bfb7ff3eb7c238a83c91c5a169

Uploaded At: October 23, 2018

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
November 01, 2019, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
October 31, 2019, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
238 schools.
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
2878 students.
Final Sample Size (or Number of Clusters) by Treatment Arms
117 schools for the treatment group and 121 schools for the control group.
Data Publication

Data Publication

Is public data available?
Yes

Program Files

Program Files
Yes
Reports, Papers & Other Materials

Relevant Paper(s)

Abstract
Although primary school enrollment in developing countries has rapidly increased since 1990, the quality of education has stagnated over the years. In teaching and learning practices, textbooks are an important intermediate that links curriculum, teachers, and students. Through a randomized controlled trial in El Salvador, this study evaluated the impact of a package of interventions including the distribution of textbooks that were carefully designed to improve math learning. This study targeted primary second grade students and tracked them for two years. To capture their progress in learning over the period, this study linked math test scores at the end of the second grade and third grade, using theitem response theory (IRT). The average one-year impact of the package is estimated at 0.49 standard deviations of the IRT scores. Schools in the control group began to receive the package in the second year of research. After two years of exposure to the package—as compared to students in the control group who only received one year of exposure—the impact is estimated at 0.13 standard deviations. Students in the treatment group advanced math learning in the second year of research based on the improved understanding in the previous year.
Citation
Maruyama, T & Kurosaki, T. (Forthcoming). Developing Textbooks to Improve Math Learning in Primary Education: Empirical Evidence from El Salvador. Economic Development and Cultural Change.
Abstract
The learning crisis in lower secondary education is profound. Evidence suggests that a structured pedagogy program that combines the distribution of teaching and learning materials with different interventions is an effective approach. Given a variety of possible combinations of interventions in a program, this study conducted a two-year-long experiment on additional interventions in a program for mathematics in El Salvador. The distribution of mathematics tests and workbooks were included to a program to strengthen support of teachers for students. The average one-year impact on mathematics learning is estimated at 0.17 standard deviations. The impact remained positive but became not statistically significant in the second year of research when the difference of interventions between the treatment and control groups disappeared.
Citation
Maruyama, T. (2022). Strengthening Support of Teachers for Students to Improve Learning Outcomes in Mathematics: Empirical Evidence on a Structured Pedagogy Program in El Salvador. International Journal of Educational Research, Vol. 115: 101977.

Reports & Other Materials