x

Please fill out this short user survey of only 3 questions in order to help us improve the site. We appreciate your feedback!
Comparing the Effectiveness of Two Teacher Training Programs: Experimental Evidence from Tanzania
Last registered on February 24, 2020

Pre-Trial

Trial Information
General Information
Title
Comparing the Effectiveness of Two Teacher Training Programs: Experimental Evidence from Tanzania
RCT ID
AEARCTR-0004959
Initial registration date
February 23, 2020
Last updated
February 24, 2020 3:42 PM EST
Location(s)

This section is unavailable to the public. Use the button below to request access to this information.

Request Information
Primary Investigator
Affiliation
University of Bern
Other Primary Investigator(s)
PI Affiliation
University of Bern
PI Affiliation
University of Bern
PI Affiliation
University of Bern
Additional Trial Information
Status
In development
Start date
2019-11-04
End date
2022-12-31
Secondary IDs
Abstract
Research on basic education in developing countries has recently been shifting its focus from accessibility to schooling towards the quality of schooling. Inevitably, this will put the performance of teachers at the center of the debate. While recent data from African countries, India and El Salvador document alarmingly low levels of teacher content knowledge, we lack rigorous evidence as to how this problem may be tackled. This field experiment aims at providing novel insights on the effectiveness of two types of teacher training programs. Both evaluated interventions target primary school math teachers: The first intervention, labelled as "SITT" (School Based In-Service Teacher Training), is built around a five-days workshop and detailed manuals that introduce new pedagogical techniques and offer an opportunity to refresh the participants' content knowledge. The second intervention, labelled as "SITT-D" (SITT-Digital), mixes computer-assisted self-learning (for teachers) and SITT. The design of the study allows to identify the causal effect of the implemented teacher training programs on the content knowledge of teachers and the math skills of their students.
External Link(s)
Registration Citation
Citation
Brunetti, Aymo et al. 2020. "Comparing the Effectiveness of Two Teacher Training Programs: Experimental Evidence from Tanzania." AEA RCT Registry. February 24. https://doi.org/10.1257/rct.4959-1.0.
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details
Interventions
Intervention(s)
Both interventions are in-service teacher training programs implemented by the Swiss NGO Helvetas, that target primary school teachers teaching math to 6th and 7th graders (among others).

The first intervention, which is labelled as "SITT" (School Based In-Service Teacher Training), builds around a 5-days workshop with local education experts, that aim to introduce new teaching techniques (pedagogy) but also repeat selected concepts from the official primary school curriculum. Moreover, all participants obtain specifically developed manuals covering Algebra (>100 pages), Numbers (>200 pages), Geometry (>100 pages), and Fractions (>100 pages). These manuals serve as basis for the teacher training courses and can be used by teachers as a guide and source of inspiration throughout the school year; the focus of these manuals lies on interactive and practice-oriented learning. SITT further aims at exploiting multiplier/cascade effects: In particular, participants are asked to form peer-learning groups at their school and introduce the new techniques to their fellow teachers. To ensure compliance, school authorities (incl. head teachers, quality assurance officers and district education officers) agreed to closely monitor the functioning of these peer-learning groups at their school(s); this close monitoring should catalyze trickle down effects of the intervention to all math teachers that are not directly treated.

The second intervention arm, which is labelled as "SITT-D", basically mirrors the original SITT-program, but adds a “Digital” component. Additionally to the core elements of SITT, the 65 participants of SITT-D will receive a tablet with computer-assisted learning software. They will be introduced to the software during the initial 5-days workshop, and receive assignments to refresh those concepts that were answered worst in the baseline assessment. In particular, the following elements to the original SITT-version are added: (i) An introduction to the software during the initial 5-days workshop, (ii) computer-assisted learning assignments as part of their action plans, (iii) two additional two-days meetings in 2020 (~April & ~September) to discuss the assignments and (iv) regular submission of automated outputs produced by the CAL-software to monitor their work with the software. Both the self-studying and the attendance in workshops are (weakly) incentivized, meaning that the participants will get awarded the IT-hardware, in case their participation rate crosses a certain threshold.
Intervention Start Date
2020-02-24
Intervention End Date
2020-10-31
Primary Outcomes
Primary Outcomes (end points)
Math skills of primary school teachers measured via pencil and paper assessments covering the local curriculum of grades 2 to 7. We also assess their students' math ability at the end of the school years 2021 and 2022 based on national standardized math exams in grade 7 conducted by the National Examimations Council of Tanzania (NECTA).
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Classroom observation will be conducted where we collect data on teacher attendance and pedagogical aspects during the lessons. These observations are based on the "Teach" observation tool of the World Bank and adapted to the context.
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
Starting point are 318 primary schools in four Tanzanian districts, i.e. Mbulu DC (Manyara), Mbulu TC (Manyara), Karatu (Arusha) and Siha (Kilimanjaro).

We exclude 49 of these 318 schools for one or several of the following reasons: (i) the school is run by a private entity, or (ii) the school did not participate in the Standard Four National Examination 2018, or (iii) less than 10 students participated in the Standard Four National Examination, or (iv) the school has less than 6 teachers.

After this first pre-slection, 269 primary schools remain in our sample. We sort these 269 schools along an accessibility index incorporating an ordinal assessment by the District Education Offices and Google Map driving times to the district center. Based on this ranking, we exclude the 49 least accessible schools.

Following this preselection, 220 schools remain in our sample. For these 220 schools the District Education Officers provided us with name lists of all teachers that (regularly) teach math to 4th, 5th, 6th, or 7th graders. The District Education Officers, in consultation with the schools, designated one of the math teachers as "Program Teacher". While we did not communicate any specific selection criteria for these "Program Teachers" the purpose of doing so was well explained: Only one teacher per school, that is the "Program Teacher" will potentially be assigned to Treatment 1 (i.e. SITT), or Treatment 2 (i.e. SITT-D), or the Control Group. Moreover, we asked the involved authorities and the selected teacher to sign a "Declaration of Intent", specifying that we expect these "Program Teachers" (i) to remain in the same school until the end of school year 2021, (ii) and that they will teach the same class of 6th graders (as of 2020) until they reach the Primary School Leaving Examination in 2021.

Moreover, we use the list of math teachers provided by the District Education Officer to sample one additional teacher per school as participant in our math assessments. We aim to use the data obtained from these teachers for two purposes: First, to gain insights on the math content knowledge for a representative sample of math teachers (standard 4--standard 7) at the pre-selected schools. Second, to be able to measure peer-to-peer spillovers/multiplier effects at the SITT- and SITT-D-treatment schools.

This procedure yields a list of 440 math teachers from 220 different schools. 220 of these teachers, i.e. one per school, was designated as "Program Teacher"; these 220 teachers are randomly assigned to Treatment 1 (SITT, 65 "Program Teachers"), Treatment 2 (SITT-D, 65 "Program Teachers"), or the Control Group (90 "Program Teachers"). The random assignment was implemented with STATA, and we stratify on average test scores of students in the National Standardized Math Assessment in grade 7, teacher baseline test scores, region.

To assess the impact of the two interventions on the teachers' math content knowledge, a baseline assessment with the 440 teachers will be conducted in November 2019 and an endline assessment in November 2020. The scores of the "Program Teachers" will inform us on the direct impact of the interventions on the teachers' content knowledge, while the scores of the additional (regular) teachers allows us to measure potential spillovers.

To assess the impact of the interventions on the students' math abilities, we will rely on the standardized national assessments conducted with 4th-graders (NSFA, "baseline") and 7th-graders (PSLE, "endline") administered by the The National Examinations Council of Tanzania (NECTA).



Experimental Design Details
Not available
Randomization Method
Randomization is done in the office using Stata.
Randomization Unit
We randomize on the school/teacher level stratifying on average test scores of students in the National Standardized Math Assessment in grade 7 (grouped in terciles), teacher baseline test scores (two groups) and on districts (Mbulu town, Mbulu district, Karatu and Siha). Note that each school nominates only one teacher for the program. Hence, for teachers the randomization of the treatments was not clustered, while for students it is clustered on the teacher level.
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
220 schools.

220 designated "Program Teachers", who are nominated by the Education Authorities to participate in the program; these are randomly assigned to Treatment 1 (SITT, 65), Treatment 2 (SITT-D, 65) or the Control Group (90).

220 randomly sampled math teachers, who regularly teach standard 4, standard 5, standard 6, or standard 7 classes. These allow us to measure within school spillovers.
Sample size: planned number of observations
220 "Program Teachers", 220 additional teachers to measure spillovers. The math abilities of their students are also assessed, as we plan to exploit the Primary School Leaving Examinations conducted by NECTA in 2021 and 2022. While class sizes vary considerably and change at the start of new school years, we expect on average 40-50 students per teacher, i.e. about 8'800-11'000 students per wave.
Sample size (or number of clusters) by treatment arms
65 teachers are assigned to Treatment 1 (SITT), 65 teachers are assigned to Treatment 2 (SITT-D) and 90 teachers are assigned to the control group.

As we expect on average 40-50 students per teacher, we estimate to evaluate the direct effect on students' math abilities based on the results of 8'800-11'000 Primary School Leaving Exams conducted by NECTA in 2020.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
MDE_TEACHERS=0.2 -- 0.32 standard deviations. Calculations based on formula by Bloom (2007) and following parameter values: power=80%; alpha (level of significance)=0.05; R-squared: 0.5--0.8 (based on empirical values in Brunetti et al. 2019, see RCT ID: AEARCTR-0004092); P (share of control units)=0.58; n (total observations)=155. MDE_STUDENTS=0.14 -- 0.19 standard deviations. Calculations based on formula by Bloom (2007) and following parameter values: power=80%; alpha (level of significance)=0.05; R-squared (between): 0.13--0.56; R-squared (within): 0.16--0.47; Rho (Interclass correlation)=0.19--0.20; P (share of control units)=0.58; J (total clusters)=155; n (observations per cluster)=40.
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
Ethikkommission der Wirtschafts- und Sozialwissenschaftlichen Fakultät der Universität Bern
IRB Approval Date
2019-11-04
IRB Approval Number
122019