The Impact of Teaching at the Right Level in Zambia: With and Without Additional Continuous Professional Development for Teachers

Zambia's Ministry of Education has embraced the Teaching at the Right Level (TaRL) methodology since the 2016-17 academic year. The Zambian TaRL program focuses on grades three to five, and it is locally known as Catch Up (CU). It currently covers eight (out of ten) provinces, reaching 4,900 schools and impacting about 735,000 learners per year. The program divides children into groups based on their learning needs and pace and adds extra remedial lessons during which teachers provide differentiated instruction to each group. All of the program implementers are regular teachers and inspectors commonly assigned to the public primary schools.

Based on the findings from an earlier mixed-methods study and a subsequent year of iterative piloting on a small scale, the Research Practice Partnership between J-PAL, TaRL Africa, VVOB, and the Ministry of Education co-developed a continuous professional development program to better support Catch Up teachers. This program complements the standard Catch Up program by establishing and supporting communities of practice among teachers (both within schools and between schools via WhatsApp). During regular professional development meetings, teachers engage in discussions about the Catch Up program, receive additional guidance documents and videos that align with their responsibilities throughout the academic year, and are invited to collaborate with their colleagues to participate in biweekly “mastery challenges.” In addition, the Ministry recognizes teachers’ successful participation through non-monetary incentives and issues formal letters of commendation.

2023-01-01

2024-12-31

The study's main outcome of interest is children's learning of foundational skills in mathematics and literacy as measured with independent one-on-one assessments.

We use an item response theory (IRT) model to aggregate students' responses to test items to generate continuous estimates of student ability. We generate one overall score per subject and standardize the score as per the endline distribution of student ability (with a mean of zero for the control group).

Student-level end points
- Student subskills in math and literacy (including the subdomains targeted by the program vs other subdomains of foundational numeracy)
- Student socio-emotional skills and creativity
- Student non-subject-matter cognitive outcomes
- Student attitudes towards literacy, math, school

In addition, the study collects data on intermediate outcomes at the teacher level, program take-up, and implementation fidelity.

Additional explanation regarding measures of subdomains of math and literacy.

In mathematics, we generate one continuous, standardized score for those skills targeted by the program (i.e., number recognition and procedural arithmetic) and a similar score capturing the remaining domains (e.g., geometry). In addition, we calculate the proportion of correctly answered questions by content domain, cognitive domain, and questions' curricular grade level. Finally, we also report on the proportion of students who have mastered discrete levels of ability, as per the ASER tests. In number recognition and arithmetic, we focus on whether students can at least do two-digit subtraction with borrowing; in literacy, we focus on whether students can at least read a short paragraph of about 30 words (making three mistakes or fewer).

We created three experimental groups of schools for the study. We began by randomly assigning half of 182 zones to either receive the Catch Up program or not receive the program (and continue with business as usual). Those zones assigned to the Catch Up program are the same in which we sampled two government schools for data collection; in the remaining zones not assigned to the program, we sampled one school. We randomized zones within strata of four zones each. We generated these strata by grouping zones that (a) shared the same district and (b) had similar levels of average academic performance. After that, in the zones assigned to the program, we randomly assigned one school to receive the program together with the additional continuous professional development (CPD) program.

Not available

Randomization done in office by a computer

Zones: To either receive the Catch Up program or not receive the program (and continue with business as usual).
Schools: In the zones assigned to the program, one school to receive the program together with the additional continuous professional development (CPD) program or not (and continue with the usual program).

Yes

273 schools in 182 zones

8,025 students sampled at baseline (to be tracked over two years). Assuming a worst-case scenario of 20% attrition, the planned number of observations is between 6,420 and 8,025 students.

91 control schools, 91 schools with the regular Catch Up intervention, and 91 schools with the Catch Up intervention and additional continuous professional development for teachers

0.12 standard deviations