Experimental Design
Our experiment is an impact evaluation that randomly assigned schools to either receive the complete PLP intervention, partial PLP intervention or to receive the current standard educational serves, serving as a control.
A. Site and School Selection
Eligible schools include primary schools from the Lango Sub-region that have two grade one classes (P1) and that agree to participate in the program. From this set of eligible schools, Mango Tree, in collaboration with the school administrations, district education officers and coordinating center tutors (CCTs), selected 36 target schools based on their location, availability of desks/tables in both the P1 classrooms, two P1 teachers for the two separate classrooms, willingness of the schools to sign a contract with Mango Tree and agree to place two best early primary teachers in the selected P1 classrooms.
B. Randomization - Treatment and Control Schools and Classrooms
12 of the 36 schools were randomly assigned as controls, 12 schools as partial treatment and another 12 schools were assigned to the treatment. Beyond the data used to select the schools, additional school data – available from district education officers – including location in the Lango sub-region, average performance on previous national exams, the share of female pupils, and total P1 pupils, was used for stratified assignment of the treatment.
The random assignment of the treatment was stratified across the 36 target schools in order to ensure a similar distribution of school characteristics across the control arm and the two treatment arms.
C. Selection and Assignment of Teachers
The PLP intervention involves asking the head teacher for the school to choose the two best early primary teachers in the school and assign them to the two P1 classrooms. Typically these are the highest skilled teachers in the school who are very committed to their work. In order to make each evaluation group equivalent, this was done in each of the 38 study schools. Head teachers agreed to assign the two best teachers to P1 classrooms, and to submit the names of those teachers at the stakeholder meeting prior to the public random assignment of the intervention. Baseline surveys of all P1 teachers were conducted at the beginning of the school. In addition, information on teacher attendance and performance will be collected during random checks over the course of the school year.
Compliance with this procedure has thus far been very high, but any deviations from the pre-assignment of teachers to classrooms are being noted by the data collection team. A related issue is that one of the Control schools, Alyec, had only a single P1 Stream instead of the two that were agreed upon in the contract. This means that our sample of P1 teachers has been reduced by one, to 75. It may also affect the validity of comparisons between Alyec and other schools in our sample, as large class sizes have a detrimental effect on learning; the single P1 stream at Alyec had over 200 pupils. Our analyses will take this potential issue into account, and Mango Tree staff are working with Alyec to correct the problem. Any other compliance issues will be noted as they come up, and either mitigated on the ground or accounted for in our data analyses.
D. Assignment of Pupils to Classrooms
Students were randomly assigned to one of the two classrooms by the school’s head teacher. This prevents any potential confounding from classroom-switching. It also helps ensure that the classroom assignments are fair, with each pupil having an equal chance of being paired with each teacher.
The random assignment was conducted by using specialized enrollment rosters for all 38 schools. Mango Tree staff carried pre-printed rosters with spaces for each pupils’ name and other details, along with an assignment to stream A or stream B. The assignments were generated at random using the runiform() function in Stata 11 SE. The head teachers were instructed to copy over the pupil names onto these rosters in order from their own records, and use the listed assignment for each pupil. Any late-enrolling pupils were added in the order they arrived. This process was validated by Mango Tree staff field visits, and a copy of each roster was collected and used to select pupils for examinations.