|
Field
Last Published
|
Before
June 10, 2025 06:02 AM
|
After
June 10, 2025 06:09 AM
|
|
Field
Intervention (Public)
|
Before
The intervention is to be implemented is "Tools for Foundational Learning Improvement" (TFLI). TFLI is a structured pedagogy program which uses a ‘science of reading’ approach, combined with assessment-informed targeted instruction for early-grade reading. In addition to paper materials, training, and coaching, teachers, school leaders, and field staff delivering the program get a companion app with video explainers and tools for running 1-1 literacy assessments, tracking student progress and managing coaching. This app (SmartCoach) adds a digital layer to the program, enabling high-frequency data collection on program delivery and student learning outcomes. This enables data-driven adaptive program management, and is intended to cultivate a more purposeful learning centered approach to program delivery.
|
After
The intervention to be implemented is “Tools for Foundational Learning Improvement” (TFLI). TFLI is a structured pedagogy program which uses a “science of reading” approach, combined with assessment-informed targeted instruction for early-grade reading. In addition to paper materials, training, and coaching, the program provides a companion app with video explainers and tools for running 1-1 literacy assessments, tracking student progress and managing coaching. This app (SmartCoach) adds a digital layer to the program, enabling high-frequency data collection on program delivery and student learning outcomes. This enables data-driven adaptive program management and is intended to cultivate a more purposeful learning centered approach to program delivery.
|
|
Field
Primary Outcomes (End Points)
|
Before
English early grade reading assessment (EGRA) scores in control-group standard deviations. EGRA scores will be computed as the first principal component of all the EGRA subtest scores, as measured in the control group at endline.
|
After
English EGRA score (in SDs)
- Score is the weighted average of the subtest scores, where the weights are the first
principal component of the control-group data across all English EGRA components we tested in this wave of data collection, for every student in the relevant cohort (BS1 students in the 2024-25 school year). We will standardize each subtest score by the control-group mean and SD before running PCA. See the PAP for more details on how the scores are measured.
|
|
Field
Experimental Design (Public)
|
Before
We will randomly assign a total of 80 schools in two experimental arms, with 40 schools in control and 40 schools in treatment. Schools in the treatment group will receive the structured pedagogy program over the 2024/2025 academic year. For this academic year, the focus is only on grade 1 students. Within the intervention period, we will run a series of A/B tests on all the treatment schools, testing out different features of TFLI to inform future designs in a planned scale-up of the study. These rapid A/B tests include having teachers do more re-assessing and re-grouping of students based on their progress, testing the impact of parent-facing report cards, and training school leaders to oversee the program. The pupils in treatment schools will receive internal weekly whole-class assessments as well as one-on-one assessments planned for every fifth week in the program. This will be administered on the SmartCoach App- a tool that the implementing partner has developed and is currently using to administer reading tests at a high frequency. At the end of the academic year, we will implement an external EGRA tests to monitor the main outcome of interest- EGRA scores, for the whole study sample.
|
After
We will randomly assign a total of 80 schools to two study arms, with 40 schools in the control group and 40 schools in the treatment group. Schools in the treatment group will receive the structured pedagogy program over the 2024/2025 academic year. For this academic year, the focus is only on grade 1 students. There will be no baseline data. Within the intervention period, we will collect student lists for every school at the beginning of the 2024-25 school year. We will run a separate analysis studying whether the type of enumerator who collects the data affects measured test scores. Specifically, we have two groups of enumerators: 1) school improvement support officers (SISOs) who work for the Ghanaian government, and contractors hired directly by our organization and whose only connection to the education sector is this project. Each group will have approximately 10 members. We will randomly assign an equal number of SISOs and contractors to each school and randomize at the student level which enumerator collects each student’s data.
|
|
Field
Randomization Method
|
Before
Randomization was done in office by a computer. Specfically, the randomization was run in R using the randomizr package.
|
After
Randomization was done in the office by computer. Specifically, the randomization was run in R using the randomizr package.
|
|
Field
Randomization Unit
|
Before
We randomized at the school level.
|
After
The treatment is randomized at the school level. The type of data collector (i.e. SISO or contractor) is randomized at the individual level.
|
|
Field
Planned Number of Clusters
|
Before
80 schools.
|
After
80 schools
|
|
Field
Sample size (or number of clusters) by treatment arms
|
Before
40 schools in both treatment and control.
|
After
40 schools in treatment, 40 schools in control
|
|
Field
Power calculation: Minimum Detectable Effect Size for Main Outcomes
|
Before
Based on a pilot study of this program, we estimate an ICC of 0.25. Since two schools closed before the intervention began, we expect to have 78 schools with an estimated 30 children per school for a total expected sample size of 2340 students. This gives an estimated MDE of 0.38 standard deviations at 90% power.
|
After
See the power calculations document under "Supporting Documents"
|
|
Field
Secondary Outcomes (End Points)
|
Before
|
After
Besides EGRA scores, we will also study the following outcomes from data collected during the school visits:
a. Other questions from the student survey. Specifically, we plan to analyze how the intervention affects students’ perceived class ranks, career, and academic aspirations. We will also test if the intervention affects whether students practice reading and writing outside of school.
b. Quality of teaching instruction, as measured via classroom observation.
c. The distributional effects of the intervention using quantile regression, distributional regression, and K–S Tests.
d. Effects on individual subtests of the EGRA (including distributional effects)
We will also do three other secondary analyses. First, we will study the effect of the type of enumerator (SISO vs. outside contractor) on tests scores and estimated treatment effects.
Second, we will validate the data collected by enumerators. This will be validated by audio recording the EGRA evaluations and re-scoring them using an AI tool.
Third, the treatment effects of the intervention will also be expressed as Equivalent Years of Schooling (EYS). Since we have no baseline test scores, we will explore other ways of measuring typical progress on the EGRA each year.
Finally, we will look at treatment effect heterogeneity analyses by student gender, teacher gender and teacher-student gender match. We may also do other exploratory analyses of the data; this list is not comprehensive.
|