|
Field
Abstract
|
Before
Research on basic education in developing countries has recently been shifting its focus from accessibility to schooling towards the quality of schooling. Inevitably, this will put the performance of teachers at the center of the debate. While recent data from African countries, India and El Salvador document alarmingly low levels of teacher content knowledge, we lack rigorous evidence as to how this problem may be tackled. This field experiment aims at providing novel insights on the effectiveness of subject-based teacher training programs. The evaluated intervention targets primary school math teachers and consists of a five-month in-service training combining (1) computer-assisted self-studying and (2) monthly workshops. The design of the study allows to identify the causal effect of the implemented teacher training program on the content knowledge of teachers and the math skills of their students.
|
After
Research on basic education in developing countries has recently been shifting its focus from accessibility to schooling towards the quality of schooling. Inevitably, this will put the performance of teachers at the center of the debate. While recent data from African countries, India and El Salvador document alarmingly low levels of teacher content knowledge, we lack rigorous evidence as to how this problem may be tackled. This field experiment aims at providing novel insights on the effectiveness of subject-based teacher training programs. The evaluated intervention targets primary school math teachers and consists of a five-month in-service training combining (1) computer-assisted self-studying and (2) monthly workshops. The design of the study allows to identify the causal effect of the implemented teacher training program on the content knowledge of participants and to examine the persistency of the effect over the course of one year.
|
|
Field
Trial End Date
|
Before
October 16, 2020
|
After
June 25, 2021
|
|
Field
Last Published
|
Before
June 28, 2019 01:27 AM
|
After
October 15, 2020 07:04 AM
|
|
Field
Primary Outcomes (End Points)
|
Before
Math skills of primary school teachers measured via pencil and paper assessments covering the local curriculum of grades 2 -- 6. We also assess their students' math ability at the end of the school year 2020 based on grade-specific assessments.
|
After
Math skills of primary school teachers measured via pencil and paper assessments covering the local curriculum of grades 2 -- 6. We measure math ability of teachers in three waves (baseline, endline, and follow-up). The endline assessment is administered to the teachers shortly after the completion of the training in Sept./Oct. 2019, while the follow-up test is conducted one year later in November 2020.
[Originally, we planned to assess students in Sept./Oct. 2020. We cancelled the student assessments due to the COVID-19 pandemic.]
|
|
Field
Experimental Design (Public)
|
Before
Starting point are all primary schools in Morazan, excluding the 50 smallest schools with only one or two teachers covering grades 3--6. The NGO sent out invitations to all eligible teachers in these schools, who were asked to register via telephone, internet or filling out a registration form.
Based on these registrations the applicants were invited to an information meeting in March 2019, where an unannounced baseline math assessment was conducted. Some of the applicants took exactly the same assessment during a representative survey in September 2018 (about 6 months before), and therefore only completed the registration and received additional information about the program.
Based on the baseline assessments, the worst performing applicant of every school was selected; applicants were informed that participation in the March-meeting did not guarantee admission to the program, but the pre-selection criteria were not communicated to them. Following this procedure 175 teachers from 175 different schools across Morazan remained in the sample.
Stratifying on baseline scores and teacher gender, the 175 teachers were randomly assigned to the either the control or treatment group. This gives 87 teachers that participate in the program and 88 teachers constituting the control group.
To assess the impact of the intervention on the students' math abilities, an assessment with the students of the 175 teachers in the sample will be conducted in September / October 2020 (i.e. towards the end of the school year 2020). If a teacher teaches math in more than one class, we randomly sample one class per teacher to conduct the student assessments and collect the monitoring data.
|
After
Starting point are all primary schools in Morazan, excluding the 50 smallest schools with only one or two teachers covering grades 3--6. The NGO sent out invitations to all eligible teachers in these schools, who were asked to register via telephone, internet or filling out a registration form.
Based on these registrations the applicants were invited to an information meeting in March 2019, where an unannounced baseline math assessment was conducted. Some of the applicants took exactly the same assessment during a representative survey in September 2018 (about 6 months before), and therefore only completed the registration and received additional information about the program.
Based on the baseline assessments, the worst performing applicant of every school was selected; applicants were informed that participation in the March-meeting did not guarantee admission to the program, but the pre-selection criteria were not communicated to them. Following this procedure 175 teachers from 175 different schools across Morazan remained in the sample.
Stratifying on baseline scores and teacher gender, the 175 teachers were randomly assigned to the either the control or treatment group. This gives 87 teachers that participate in the program and 88 teachers constituting the control group.
To assess the impact (and its persistency) on the math content knowledge of teachers, an endline assessment and a follow-up assessment will be conducted. The endline assessment is administered to teachers shortly after the end of the program in Sept./Oct. 2019, while the follow-up assessment will take place in November 2020.
|
|
Field
Randomization Unit
|
Before
We randomize on the teacher level stratifying on their gender and baseline score. Note that the pre-selection based on the baseline score leaves only one teacher per school in the sample. Hence, for teachers the randomization of the treatment was not clustered, while for students it is clustered on the teacher level.
|
After
We randomize on the teacher level stratifying on their gender and baseline score. Note that the pre-selection based on the baseline score leaves only one teacher per school in the sample. Hence, for teachers the randomization of the treatment was not clustered.
|
|
Field
Planned Number of Clusters
|
Before
175 teachers (and schools) and 175 classes (one class per teacher).
|
After
175 teachers (and schools).
|
|
Field
Planned Number of Observations
|
Before
88 teachers (and schools) in the control group, 87 teachers (and schools) in the treatment group. The students' of these teachers are also assessed in September / October 2020 (i.e. towards the end of the school year 2020); while the class sizes may change at the start of the new school year in January 2020, we expect on average about 18 students per teacher.
|
After
88 teachers (and schools) in the control group, 87 teachers (and schools) in the treatment group.
|
|
Field
Sample size (or number of clusters) by treatment arms
|
Before
88 teachers (and schools) in the control group, 87 teachers (and schools) in the treatment group. About 1580 students in the control and 1580 students in the treatment group (this is an estimate as class sizes may change at the start of the new school year in January 2020).
|
After
88 teachers (and schools) in the control group, 87 teachers (and schools) in the treatment group.
|
|
Field
Power calculation: Minimum Detectable Effect Size for Main Outcomes
|
Before
MDE_TEACHERS = 0.23--0.30 standard deviations. Calculations based on formula by Bloom (2007) and the following parameter values: power=80%; alpha (level of significance)=0.05; R-squared : 0.5--0.7; P (share of control units)=0.5; n (total observations)=175.
MDE_STUDENTS = 0.14--0.17 standard deviations. Calculations based on formula by Bloom (2007) and the following parameter values: power=80%; alpha (level of significance)=0.05; R-squared (between): 0.32--0.53; R-squared (within): 0.03--0.24; Rho (Interclass correlation)=0.13--0.20; P (share of control units)=0.5; J (total clusters)=175; n (observations per cluster)=18
|
After
MDE_TEACHERS = 0.23--0.30 standard deviations. Calculations based on formula by Bloom (2007) and the following parameter values: power=80%; alpha (level of significance)=0.05; R-squared : 0.5--0.7; P (share of control units)=0.5; n (total observations)=175.
|
|
Field
Secondary Outcomes (End Points)
|
Before
Class cancellation rates and student attendance rates based on at least four unannounced visits per class throughout the school year 2020. This yields about 700 data points, 350 for the control group and 350 for the treatment group.
|
After
[Originally, we planned to collect monitoring data during the school year 2020, incl. class cancellation rates and student attendance. We cancelled these monitoring operations due to the COVID-19 pandemic.]
|