Back to History Current Version

AI, implementation and academic outcomes in poor resource settings: experimental evidence from Ghana

Last registered on December 09, 2024

Pre-Trial

Trial Information

General Information

Title
AI, implementation and academic outcomes in poor resource settings: experimental evidence from Ghana
RCT ID
AEARCTR-0014476
Initial registration date
December 04, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
December 09, 2024, 4:52 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
University of Ghana, Legon

Other Primary Investigator(s)

PI Affiliation
Rising Academies Network
PI Affiliation
Rising Academies Network

Additional Trial Information

Status
In development
Start date
2025-01-01
End date
2025-06-30
Secondary IDs
Prior work
This trial is based on or builds upon one or more prior RCTs.
Abstract
Low-and-middle income countries (LMIC) are experiencing a learning crisis, with less than 15% of sub-Saharan African students reaching minimum mathematics proficiency by the end of middle school (UNESCO, 2017). Students are often taught content far above their ability level and lack opportunities for practice and timely feedback. While high-quality one-on-one instruction has long been advocated to address these issues (Bloom 1984; Chi et al., 2001), the low supply and high cost of quality tutors in West Africa, pose significant barriers to access (Bray 2021). Consequently, research is needed to identify affordable and scalable interventions tailored to this context. Leveraging technology-supported learning, such as Intelligent Tutoring Systems (ITS) (Vanlehn, 2011) has shown promise in addressing diverse learning needs in LMICs. However, students in LMICs often don’t have access to home computers and the internet, though mobile phone usage is high (International Telecommunication Union 2022). The University of Ghana and Rising Academies, an educational network based in West Africa, seek to improve learning in primary and junior high schools in Ghana. Rising Academies has developed an AI-driven solution, 'Rori' to help children improve their math performance. Rori is a chat-based math tutor on WhatsApp, which can be accessed on affordable mobile phones. Rori has the potential to generate benefits for students living in poverty who cannot afford extra in-person tutorials. There are many possible avenues to using Rori as it does not require extensive bandwidth or large amounts of costly data, unlike platforms that provide video lessons. Building on encouraging results with students attending Rising Academy schools (Henkel et al., 2024), our proposed pilot will encompass a broader spectrum of schools (low-cost private schools and government-run public schools) to deepen the understanding of implementation challenges and intervention effectiveness. If Rori effectively improves math proficiency in all school types, it could support policy change and the widespread adoption of Rori, contributing to poverty reduction through enhanced educational outcomes. This pilot will also support the planning of a future RCT by helping us understand how to implement Rori in diverse contexts, ultimately contributing to developing scalable and sustainable solutions for improving educational outcomes in LMICs.
External Link(s)

Registration Citation

Citation
Asiedu, Dr. Edward, Owen Henkel and Hannah Horne-robinson. 2024. "AI, implementation and academic outcomes in poor resource settings: experimental evidence from Ghana." AEA RCT Registry. December 09. https://doi.org/10.1257/rct.14476-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
Rori is designed to increase access to resources and choices for low-income students as it can be used by students in or outside of school. Rori can improve classroom learning, remediate key skills and concepts, or help with test preparation. It is inspired by successful elements of TaRL (Abdul Latif Jameel Poverty Action Lab) and ITSs (Banerjee, et al., 2016). Rori first assesses students’ ability and then encourages students to work through lessons at their level, each with a brief explanation and a series of scaffolded practice questions. When students struggle, Rori provides first a hint and then a complete solution. Rori will also direct students to more suitable lessons if a lesson is too challenging or too easy. There are over 500 lessons structured around the Global Proficiency Framework, an internationally recognized set of mathematics learning standards. Students can chat with Rori about their mathematics goals and encourage metacognitive skills as it uses various natural language processing (NLP) methods, including specialized language models (LLMs). This also enables Rori to better interpret students’ answer attempts and questions.
Intervention (Hidden)
Intervention Start Date
2025-01-06
Intervention End Date
2025-04-30

Primary Outcomes

Primary Outcomes (end points)
Increase in learning gains
Primary Outcomes (explanation)
The main outcome measure will be learning gains, which will be assessed with items from Trends in International Mathematics and Science Study (TIMSS), an externally validated assessment.

Secondary Outcomes

Secondary Outcomes (end points)
Poverty Reduction,
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We plan to recruit eight schools, for a total of 24 schools with relevant observable characteristics, such as in similar communities and graduation rates. Students in grades 6-9 will participate in this pilot and all participants will receive their regular math instruction. The classes of the schools will be randomly assigned into two conditions and Rising Academies will provide all treatment classes with phones. Students in the Treatment group will use Rori for an hour a week during this time usually reserved for extracurriculars or study hall. Students in the control group will not receive access to Rori during this time. Our partners will provide clear instructions to the teacher who will be in charge to ensure that the mobile phones are kept in a safe place and children have access to them when it is time for Rori class.
Experimental Design Details
Randomization Method
We randomized in the office amongst 4 people, each individual was assigned a class for each school. whatever one picks
Randomization Unit
Classroom Level Randomization
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
24 Schools
Sample size: planned number of observations
1440 Pupils
Sample size (or number of clusters) by treatment arms
12 Schools in Control , 12 Schools in Treatment
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
For our power calculation, we used an Intraclass Correlation Coefficient (ICC) of 0.09, based on findings by Hedges and Hedberg (2007) for low-achievement schools, and a standard error (SE) of 0.01, as per Kelcey et al. (2016) for cluster randomized trials in Sub-Saharan Africa.
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
Noguchi Memorial Institute for Medical Research IRB
IRB Approval Date
2024-10-01
IRB Approval Number
#35/24-25
Analysis Plan

Analysis Plan Documents

AI, implementation and academic outcomes in poor resource settings: experimental evidence from Ghana

MD5: 483c5989c710029ee970a6ef6f59a82e

SHA1: 4e271fc6d3aaf0a060d3587ce1936700da24b475

Uploaded At: December 04, 2024

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials