Back to History Current Version

Aprender+ Program: Impact Evaluation in Mozambique

Last registered on February 07, 2023

Pre-Trial

Trial Information

General Information

Title
Aprender+ Program: Impact Evaluation in Mozambique
RCT ID
AEARCTR-0010901
Initial registration date
February 01, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
February 07, 2023, 11:32 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Affiliation
The World Bank

Other Primary Investigator(s)

PI Affiliation
Georgetown University
PI Affiliation
Center for Global Development

Additional Trial Information

Status
In development
Start date
2023-02-27
End date
2024-11-29
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Research has shown that one of the most cost-effective interventions to improve student learning, is “structured pedagogy programs”, that provide teachers continuous professional development support (TCPD) – in the form of lesson plans, training, and ongoing support by expert coaches – and additional learning resources to be used in the classroom (Snilstveit et al., 2016). This model has been tested in many contexts in Africa (Cilliers et al., 2019).

However, there is limited evidence on how such programs can be implemented by the government at scale in a low-capacity environment. Most of the successful studies evaluated were implemented by NGOs and at a small scale. In fact, studies have shown that existing government-run teacher professional development programs fail to have a positive impact on teaching practices and student learning (Loyalka et al., 2019; Schaffner et al., 2021). And other studies have shown that successful programs implemented by NGOs at a small scale, often fail to successfully scale when implemented by the government (Tessa et al., 2018). More research is required to find models of teacher professional development that are scalable and can be successfully implemented by the government.

Despite positive elements of Mozambique’s new teacher professional development model, some aspects can be strengthened by: (i) providing resources to simplify lesson planning; (ii) recognizing that lesson study methodology (reviewing lessons plans among peer teachers) may not be effective for teachers who have not mastered the curriculum content or possess weak pedagogical skills; (iii) providing more structure and intensive support to teachers with low content and pedagogical skills; (iv) recognizing that the schedules and tasks teachers are responsible for have not been modified, despite having added additional responsibilities, which has resulted in increased time constraints that prevent them from effectively implementing the new model.

Most traditional in-service professional development is highly theoretical, with little practical application. Aprender+ will promote the use of active learning strategies and deliberate practice to help teachers improve their teaching. The literature on adult learning is clear on the need to replace lectured-based teaching with active learning strategies and hands-on application. The rationale is twofold. First, active learning strategies promote engagement and increase learning (Freeman et al., 2014). Second, practicing teaching skills that teachers need to use in their classrooms makes it more likely that teachers will modify their teaching habits (Lemov et al., 2012). Aprender+ will promote the use of these insights not just in the coaching protocol but also in other aspects of in-service teacher development (e.g., where teachers are brought together for a period of time, and presented with new materials or methods).

The study will be conducted in the province of Manica, working in 197 Pedagogical Influence Zones (ZIPs) that have 2-4 schools within the ZIP (356 schools in the 197 ZIPs). 99 ZIPs will be assigned to treatment, and 98 will be assigned to control. The treatment will take place in all the schools in the 99 treatment ZIPs. Data collection will take place in two randomly selected schools in each of the 172 sampled ZIPs (344 schools). This intervention is targeted at grade-1 students and their teachers.

This study will contribute to the literature on TCPD, and also the literature on improving state capacity. First, it will provide critical evidence on whether and how government can successfully implement teacher professional development programs in a low-capacity environment. As mentioned before, there is strong evidence for a successful model of TCPD. Second, it will also provide insight into how to improve state capacity to improve public service delivery in developing countries.
External Link(s)

Registration Citation

Citation
Cilliers, Jacobus, Lelys Dinarte and David Evans. 2023. "Aprender+ Program: Impact Evaluation in Mozambique." AEA RCT Registry. February 07. https://doi.org/10.1257/rct.10901-1.0
Sponsors & Partners

Partner

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
This study will evaluate the effect of a teacher professional development program (which includes teacher training, structured lesson plans, and institutional support), implemented by Mozambique’s Ministry of Education. The program has the following components:

1. Training provided to (i) Grade 1 teachers (4-day training); (ii) deputy school principal (Director Adjunto Pedagógico da Escola), who will be responsible for coaching the teachers (7-day training); and ZIP coordinators (Coordenador da Zona de Influencia Pedagógic), who will be responsible for monitoring the coaching activities (8-day training)
2. Provision of a structured daily lesson plan and reading materials to grade 1 and grade 2 teachers. These lesson plans are aligned with the desired curriculum and approved by government authorities.
3. Bi-weekly coaching activities performed by the deputy school principal, who will observe and assess teaching using a revised version of the Teach classroom observation tool in order to provide tailored feedback to each teacher and guide them on how to improve their practices.
4. Monthly Monitor visits, implemented by the ZIP coordinators. The ZIP coordinators will coordinate with coaches to visit a school together once a month to monitor the coaching activities.
Intervention Start Date
2023-03-06
Intervention End Date
2023-10-31

Primary Outcomes

Primary Outcomes (end points)
The two primary outcomes are:
1. Teaching practices: This outcome will be measured using data collected using the Teach classroom observation tool.
2. Grade 1 and 2 students' Portuguese literacy: This outcome will be measured using the test on Portuguese literacy that is used by the Ministry of Education and Human Development in Mozambique.
These outcomes will be estimated using data collected at baseline, at the end of the intervention (November 2023), and one year after (in November 2024). More details on the survey instruments that will be used are presented in the pre-analysis plan.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
The team will collect the following secondary outcomes from teachers using our own survey:
1. Teachers' intrinsic motivation to improve their own teaching practices and improve student learning
2. Teacher attendance at school
3. Perceptions of management and peer support
4. Motivation and wellbeing

Moreover, the team will collect the following secondary outcomes from deputy school principals (coaches):
1. Attendance
2. Motivation and management support
3. Workload

Lastly, the team will collect information from school principals and ZIP coordinators at baseline, to assess differences in the impact of the intervention by the quality of (school) management and peer support.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
To measure the impact of the Aprender+ program on outcomes of interest, the team will conduct a randomized control trial, where treatment is randomized at the ZIP (Pedagogical Influence Zones) level. (ZIPS are administrative clusters of 1-4 schools). Out of a sample of 197 ZIPs in Manica province, we will randomly assign half of the ZIPs to participate in the program. All the schools in the 99 treated ZIPs will be invited to participate. The remaining 98 ZIPs will be in the control.
Experimental Design Details
Not available
Randomization Method
Randomization was done at the ZIP level using a code written by the researchers.
Randomization Unit
The randomization unit is the ZIP (Pedagogical Influence Zone). These consist of administrative clusters of 1-4 schools.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
197 ZIPs
Sample size: planned number of observations
356 schools and 4,272 students distributed across the 197 ZIPs
Sample size (or number of clusters) by treatment arms
We expect a distribution of 178 schools in the treatment and 178 schools in the control groups.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
MDE = 0.25-0.30 SD assuming alpha = 0.05, power = 80%, correlation between baseline and endline (ANCOVA model) = 0.40.
IRB

Institutional Review Boards (IRBs)

IRB Name
Georgetown University-MedStar IRB System
IRB Approval Date
2023-01-10
IRB Approval Number
00005950