Integrating Educational Technology with Structured Pedagogy to Improve Learning Outcomes for Every Student

Last registered on December 03, 2024
View Trial History

Pre-Trial

Trial Information

General Information

Title
Integrating Educational Technology with Structured Pedagogy to Improve Learning Outcomes for Every Student
RCT ID
AEARCTR-0014627
Initial registration date
November 26, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
December 03, 2024, 1:29 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Name
Erik Andersen
Affiliation
University of Washington
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Jason Kerwin
PI Affiliation
University of Washington
Contact Investigator
PI Name
Simon Graffy
PI Affiliation
Inspiring Teachers
Contact Investigator
PI Name
Monica Lambon-Quayefio
PI Affiliation
University of Ghana
Contact Investigator

Additional Trial Information

Status
On going
Start date
2024-09-16
End date
2025-07-04
Keywords
Education
Additional Keywords
JEL code(s)
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
We are studying Inspiring Teacher's Tools for Foundational Learning Improvement (TFLI). TFLI uses best practices from early-grade reading
instruction in a structured pedagogy format, in which teachers are given teacher guides with semi-scripted, high-quality lesson plans that tell them exactly how to teach their classes. These plans are linked to workbooks and also to training and support from Inspiring Teachers employees, who are expert teachers and teacher trainers. This basic approach has been proven to be highly effective in many other interventions and contexts across Africa. TFLI adds a digital layer to this structured pedagogy approach, with teachers accessing program material and training videos via the Smart Coach smartphone app. The app will also collect high-frequency data on student outcomes, which Inspiring Teachers can use to modify the intervention on the fly, capturing the best aspects of teaching at the right level (TaRL) interventions as well.

In this study we aim to use the Inspiring Teachers TFLI approach to explore how support for assessment-informed targeted instruction can be built into structured pedagogy and how technology can be used to support that process to enable cost-effective and data driven teacher and school support. Using grade 1 and 2 students in primary schools in Cape Coast, Ghana, as our study context, we seek to determine four things:1) how TFLI affects learning outcomes, 2) the optimal way to deliver TFLI (i.e., how best to deploy the digital layer to improve learning outcomes), 3) how the benefits of structured pedagogy and teaching at the right level (TaRL) substitute for or complement each other, and 4) how best to integrate the two approaches.
External Link(s)

Registration Citation

Citation
Andersen, Erik et al. 2024. "Integrating Educational Technology with Structured Pedagogy to Improve Learning Outcomes for Every Student." AEA RCT Registry. December 03. https://doi.org/10.1257/rct.14627-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
The intervention is to be implemented is "Tools for Foundational Learning Improvement" (TFLI). TFLI is a structured pedagogy program which uses a ‘science of reading’ approach, combined with assessment-informed targeted instruction for early-grade reading. In addition to paper materials, training, and coaching, teachers, school leaders, and field staff delivering the program get a companion app with video explainers and tools for running 1-1 literacy assessments, tracking student progress and managing coaching. This app (SmartCoach) adds a digital layer to the program, enabling high-frequency data collection on program delivery and student learning outcomes. This enables data-driven adaptive program management, and is intended to cultivate a more purposeful learning centered approach to program delivery.
Intervention Start Date
2024-09-16
Intervention End Date
2025-07-04

Primary Outcomes

Primary Outcomes (end points)
English early grade reading assessment (EGRA) scores in control-group standard deviations. EGRA scores will be computed as the first principal component of all the EGRA subtest scores, as measured in the control group at endline.
Primary Outcomes (explanation)
See PAP.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We will randomly assign a total of 80 schools in two experimental arms, with 40 schools in control and 40 schools in treatment. Schools in the treatment group will receive the structured pedagogy program over the 2024/2025 academic year. For this academic year, the focus is only on grade 1 students. Within the intervention period, we will run a series of A/B tests on all the treatment schools, testing out different features of TFLI to inform future designs in a planned scale-up of the study. These rapid A/B tests include having teachers do more re-assessing and re-grouping of students based on their progress, testing the impact of parent-facing report cards, and training school leaders to oversee the program. The pupils in treatment schools will receive internal weekly whole-class assessments as well as one-on-one assessments planned for every fifth week in the program. This will be administered on the SmartCoach App- a tool that the implementing partner has developed and is currently using to administer reading tests at a high frequency. At the end of the academic year, we will implement an external EGRA tests to monitor the main outcome of interest- EGRA scores, for the whole study sample.
Experimental Design Details
Not available
Randomization Method
Randomization was done in office by a computer. Specfically, the randomization was run in R using the randomizr package.
Randomization Unit
We randomized at the school level.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
80 schools.
Sample size: planned number of observations
80 schools (2400 pupils in total)
Sample size (or number of clusters) by treatment arms
40 schools in both treatment and control.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Based on a pilot study of this program, we estimate an ICC of 0.25. Since two schools closed before the intervention began, we expect to have 78 schools with an estimated 30 children per school for a total expected sample size of 2340 students. This gives an estimated MDE of 0.38 standard deviations at 90% power.
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
University of Ghana Ethics Committee for the Humanities
IRB Approval Date
2024-07-04
IRB Approval Number
ECH 278/23-24
IRB Name
Heartland IRB
IRB Approval Date
2024-06-24
IRB Approval Number
062424-573
IRB Name
University of Washington Human Subjects Division
IRB Approval Date
2024-09-03
IRB Approval Number
STUDY00021036
Analysis Plan