Malawi National Numeracy Programme Impact Evaluation

Last registered on October 04, 2023
View Trial History

Pre-Trial

Trial Information

General Information

Title
Malawi National Numeracy Programme Impact Evaluation
RCT ID
AEARCTR-0011964
Initial registration date
September 29, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
October 04, 2023, 4:51 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
Malawi
Region

Primary Investigator

Name
Todd Pugatch
Affiliation
Oregon State University
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Lontum Nchadze
PI Affiliation
Oregon State University
Contact Investigator
PI Name
Evaewero French
PI Affiliation
Oregon State University
Contact Investigator
PI Name
Alperen Acikol
PI Affiliation
Oregon State University
Contact Investigator
PI Name
Esme Kadzamira
PI Affiliation
University of Malawi
Contact Investigator

Additional Trial Information

Status
In development
Start date
2023-09-11
End date
2024-06-19
Keywords
Education
Additional Keywords
foundational learning; numeracy; active learning
JEL code(s)
I21, I25, O15
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
The Malawi Ministry of Education is implementing the National Numeracy Programme (NNP). The NNP aims to improve early grade mathematics outcomes. This study will assess the impact of the NNP during the 2023/24 school year. The evaluation will use a randomized controlled trial design with 150 primary schools. Schools will be randomly assigned to treatment and control groups within matched sets of schools. Treatment schools will implement the revised NNP materials and training. Student learning outcomes will be measured using the Early Grade Mathematics Assessment in grades S1-S4. The study will also evaluate impacts on instructional quality using classroom observations. The evaluation will provide rigorous evidence on the impact of the NNP model on foundational numeracy skills, to inform decisions about national scale-up.
External Link(s)

Registration Citation

Citation
Acikol, Alperen et al. 2023. "Malawi National Numeracy Programme Impact Evaluation." AEA RCT Registry. October 04. https://doi.org/10.1257/rct.11964-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
Led by the Malawi Ministry of Education (MoE), funded by the United Kingdom’s Foreign, Commonwealth & Development Office (FCDO), and delivered by Cambridge Education, the four-year NNP aims to improve outcomes in mathematics so girls and boys have a solid foundation in basic skills to succeed in the rest of their schooling and fulfil their potential. The NNP aims to develop a new vision for teaching and learning mathematics in Malawi in which children experience mathematics as a meaningful, sense-making and problem-solving activity. Learners will be expected not only to know mathematics, but also to understand the mathematics they know, apply it to solve unfamiliar problems and reason and argue using the mathematics they develop. The NNP includes: a) revision of maths curriculum for the first year of primary school and subsequent training; 2) one-to-one ratio workbooks that are given to the children and that children can bring home; 3) semi-structured teacher guides guiding teachers to teach one page of workbook per day.
Intervention Start Date
2023-09-11
Intervention End Date
2024-06-19

Primary Outcomes

Primary Outcomes (end points)
numeracy (Early Grade Mathematics Assessment (EGMA))
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
take-up of teacher training
curriculum implementation (based on classroom observations)
teacher knowledge
teacher attitudes
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Cluster RCT, with treatment assignment at school level.

From a sample of 150 public primary schools in Malawi, half will be randomly assigned to the National Numeracy Program (NNP; treatment group), with the other half keeping the status quo mathematics curriculum (control group).

Details: We will begin with a sampling frame of 300 schools (100 in each of the three sample districts). Randomization will be stratified by district and school characteristics. School characteristics will be measured by the first principal component of the following school characteristics from the 2021 EMIS:

• age of school
• enrollment
• pupil-teacher ratio (PTR)
• female enrollment %
• female teacher %
• remoteness categories (from Asim et al 2019)
• share with ECD exposure
• share of repeaters in S1-S4
• passing rate on PSLCE exam
• management index, calculated as the proportion of the following reported as active at the school:
o PTA
o school management committee
o community volunteers
o School Improvement Plan

We will use these criteria to group schools into groups of 4.

Within each group of 4 schools, we will randomly assign two to treatment and two to control. This method of stratification follows recommendation by Athey and Imbens (2017).

Within treatment and control pairs, we will randomly assign one school for inclusion in the sample, the other as replacement.

This will result in an evaluation sample of 150 schools (75T/75C). It will also generate a replacement sample of equal size, to be used as necessary.
Experimental Design Details
Not available
Randomization Method
randomization done in office by computer (Stata "randtreat" command)
Randomization Unit
school
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
150 schools
Sample size: planned number of observations
64 students per school (16 students each in grades S1-S4) for a total of 9,600 total students.
Sample size (or number of clusters) by treatment arms
75 schools treatment (NNP), 75 schools control
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
MDE = 0.23 standard deviations. The analysis is based on the following considerations: • We assume power of 80% and test size of 5%. • We envisage this to be a cluster randomized control trial, with the school as the treatment unit and learning outcomes measured at the student level. • We are assuming two study groups: a treatment and a control group. • We have updated our analysis to focus on grade-specific student learning outcomes. We therefore present analysis assuming data on 16 students per grade, to account for attrition We assume an intra-cluster correlation of 0.2, based on pilot results: – The grade-specific ICCs in the pilot endline data for the control group are: • S1 = 0.19 • S2 = 0.20 • S3 = 0.11 • S4 = 0.22
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
University of Malawi Research Ethics Committee
IRB Approval Date
2023-09-20
IRB Approval Number
P.09/23/298
Analysis Plan