Integrating Educational Technology with Structured Pedagogy to Improve Learning Outcomes for Every Student

The intervention to be implemented is “Tools for Foundational Learning Improvement” (TFLI). TFLI is a structured pedagogy program which uses a “science of reading” approach, combined with assessment-informed targeted instruction for early-grade reading. In addition to paper materials, training, and coaching, the program provides a companion app with video explainers and tools for running 1-1 literacy assessments, tracking student progress and managing coaching. This app (SmartCoach) adds a digital layer to the program, enabling high-frequency data collection on program delivery and student learning outcomes. This enables data-driven adaptive program management and is intended to cultivate a more purposeful learning centered approach to program delivery.

2024-09-16

2025-07-04

English EGRA score (in SDs)
- Score is the weighted average of the subtest scores, where the weights are the first
principal component of the control-group data across all English EGRA components we tested in this wave of data collection, for every student in the relevant cohort (BS1 students in the 2024-25 school year). We will standardize each subtest score by the control-group mean and SD before running PCA. See the PAP for more details on how the scores are measured.

See PAP.

Besides EGRA scores, we will also study the following outcomes from data collected during the school visits:
a. Other questions from the student survey. Specifically, we plan to analyze how the intervention affects students’ perceived class ranks, career, and academic aspirations. We will also test if the intervention affects whether students practice reading and writing outside of school.
b. Quality of teaching instruction, as measured via classroom observation.
c. The distributional effects of the intervention using quantile regression, distributional regression, and K–S Tests.
d. Effects on individual subtests of the EGRA (including distributional effects)
We will also do three other secondary analyses. First, we will study the effect of the type of enumerator (SISO vs. outside contractor) on tests scores and estimated treatment effects.
Second, we will validate the data collected by enumerators. This will be validated by audio recording the EGRA evaluations and re-scoring them using an AI tool.
Third, the treatment effects of the intervention will also be expressed as Equivalent Years of Schooling (EYS). Since we have no baseline test scores, we will explore other ways of measuring typical progress on the EGRA each year.
Finally, we will look at treatment effect heterogeneity analyses by student gender, teacher gender and teacher-student gender match. We may also do other exploratory analyses of the data; this list is not comprehensive.

We will randomly assign a total of 80 schools to two study arms, with 40 schools in the control group and 40 schools in the treatment group. Schools in the treatment group will receive the structured pedagogy program over the 2024/2025 academic year. For this academic year, the focus is only on grade 1 students. There will be no baseline data. Within the intervention period, we will collect student lists for every school at the beginning of the 2024-25 school year. We will run a separate analysis studying whether the type of enumerator who collects the data affects measured test scores. Specifically, we have two groups of enumerators: 1) people with teaching certifications (which we call “teachers”, although they are not currently working as teachers) and 2) , and outside contractors hired directly by our organization and who are have no other not currently working in connection the education sector (“non-teachers”). Each group will have approximately 12 members. We will randomly assign an equal number of teachers and non-teachers to each school and randomize at the student level which enumerator collects each student’s data.

Randomization was done in the office by computer. Specifically, the randomization was run in R using the randomizr package.

The treatment is randomized at the school level. The type of data collector (i.e. SISO or contractor) is randomized at the individual level.

Yes

80 schools

80 schools (2400 pupils in total)

40 schools in treatment, 40 schools in control

The MDE at 90% power for our main analysis is 0.39SDs. See the power calculations document under "Supporting Documents".

Endline 2024-2025 Analysis Plan