x

Please fill out this short user survey of only 3 questions in order to help us improve the site. We appreciate your feedback!
After-School Khan Academy Program in Cape Town
Last registered on November 11, 2013

Pre-Trial

Trial Information
General Information
Title
After-School Khan Academy Program in Cape Town
RCT ID
AEARCTR-0000104
Initial registration date
Not yet registered
Last updated
November 11, 2013 3:50 AM EST
Location(s)
Region
Primary Investigator
Affiliation
Innovations for Poverty Action
Other Primary Investigator(s)
PI Affiliation
University of Cape Town
PI Affiliation
University of Cape Town
Additional Trial Information
Status
On going
Start date
2013-01-19
End date
2014-03-01
Secondary IDs
Abstract
This is an evaluation of the Numeric after-school maths program for Grade 8 students from low-income areas around Cape Town, South Africa. The program, which takes place in school computer labs, uses the free online learning platform called Khan Academy. Globally, Khan Academy is a high-profile education tool, having delivered over 260 million video lessons as of August 2013. Numeric’s added value is the creation of an offline video browser which allows the organization to stream high quality video in a low (or no) bandwidth environment.

The evaluation is designed to test the ability of a computer-based intervention that uses large amounts of refined content (in the form of video lessons) and data-driven feedback to improve students’ maths performance. The evaluation takes place in a country that performs at or near the bottom of 8th grade math performance in the Trends in International Mathematics and Science Study (TIMSS) despite a relatively high level of public sector expenditure on education. If successful, the intention is for Numeric to increase the scale of its after-school programs.

The evaluation will use creative maths testing procedures to overcome the oft-cited problem in education research of low-stakes assessment tests actually measuring motivation rather than ability. The evaluation will also seek to identify the channels through which the program improves maths performance, thereby equipping education policymakers with valuable information on how to structure similar interventions. Finally, the evaluation will measure impacts on non-math outcomes, including a measure of nonverbal cognitive ability and computer literacy.

The evaluation team aims to register this trial prior to endline data collection, cleaning, and analysis.
External Link(s)
Registration Citation
Citation
Bohmer, Bianca, Justine Burns and Lucas Crowley. 2013. "After-School Khan Academy Program in Cape Town." AEA RCT Registry. November 11. https://doi.org/10.1257/rct.104-2.0.
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details
Interventions
Intervention(s)
Numeric sessions run twice a week for 1.5 hours at a time during the course of the school year. The sessions are attended by Grade 8 students. The sessions are overseen by "coaches", most of whom are working towards a higher education degree. The sessions run immediately after school, in school computer labs. Students normally sit one to a computer. The sessions are not aligned with the South African curriculum. The focus is on improving numeracy. The sessions use online and offline versions of the Khan Academy program.
Intervention Start Date
2013-02-13
Intervention End Date
2013-11-14
Primary Outcomes
Primary Outcomes (end points)
Use of Khan Academy, math test scores, other subject marks, general cognitive skills, computer skills, English language skills
Primary Outcomes (explanation)
In addition to the overall score, the math section will be broken down into: arithmetic, fractions, decimals, patterns, and word problems, as well as grade-level questions. General cognitive ability will be measured by the Raven's Progressive Matrices test. The English measure tests vocabulary, oral comprehension, and reading comprehension. The computer measures include basic knowledge questions and correct keystrokes per minute.
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
The study employs a student-level randomization, stratifying on school as well as above/below median baseline total math score. We also checked for balance on the math subsections listed above, gender, home computer access, and travel time to school. We re-randomized if the random assignment created unbalance in these measures.
Experimental Design Details
Randomization Method
Stata command
Randomization Unit
Student
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
Not applicable
Sample size: planned number of observations
472 students
Sample size (or number of clusters) by treatment arms
236 treatment, 236 control
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
If we assume perfect compliance with random assignment, we should expect between a 0.26 and a 0.28 effect size. However, we expect take-up to be around 80% or less.
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
Faculty of Commerce Ethics in Research Committee at the University of Cape Town
IRB Approval Date
2013-07-01
IRB Approval Number
UCT/COM/202/2013
Analysis Plan

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
No
Is data collection complete?
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports, Papers & Other Materials
Relevant Paper(s)
REPORTS & OTHER MATERIALS