After-School Khan Academy Program in Cape Town

Last registered on November 11, 2013

View Trial History

Pre-Trial

Trial Information

General Information

Title

After-School Khan Academy Program in Cape Town

RCT ID

AEARCTR-0000104

First published

November 08, 2013, 7:03 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated

November 11, 2013, 3:50 AM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Country

South Africa

Region

Cape Town

Primary Investigator

Name

Lucas Crowley

Affiliation

Innovations for Poverty Action

Contact Primary Investigator

Other Primary Investigator(s)

PI Name

Justine Burns

PI Affiliation

University of Cape Town

Contact Investigator

PI Name

Bianca Bohmer

PI Affiliation

University of Cape Town

Contact Investigator

Additional Trial Information

Status

On going

Start date

2013-01-19

End date

2014-03-01

Keywords

Education

Additional Keywords

math, after-school, Khan Academy

JEL code(s)

C93, I21, I24

Secondary IDs

Abstract

This is an evaluation of the Numeric after-school maths program for Grade 8 students from low-income areas around Cape Town, South Africa. The program, which takes place in school computer labs, uses the free online learning platform called Khan Academy. Globally, Khan Academy is a high-profile education tool, having delivered over 260 million video lessons as of August 2013. Numeric’s added value is the creation of an offline video browser which allows the organization to stream high quality video in a low (or no) bandwidth environment.

The evaluation is designed to test the ability of a computer-based intervention that uses large amounts of refined content (in the form of video lessons) and data-driven feedback to improve students’ maths performance. The evaluation takes place in a country that performs at or near the bottom of 8th grade math performance in the Trends in International Mathematics and Science Study (TIMSS) despite a relatively high level of public sector expenditure on education. If successful, the intention is for Numeric to increase the scale of its after-school programs.

The evaluation will use creative maths testing procedures to overcome the oft-cited problem in education research of low-stakes assessment tests actually measuring motivation rather than ability. The evaluation will also seek to identify the channels through which the program improves maths performance, thereby equipping education policymakers with valuable information on how to structure similar interventions. Finally, the evaluation will measure impacts on non-math outcomes, including a measure of nonverbal cognitive ability and computer literacy.

The evaluation team aims to register this trial prior to endline data collection, cleaning, and analysis.

External Link(s)

Registration Citation

Citation

Bohmer, Bianca, Justine Burns and Lucas Crowley. 2013. "After-School Khan Academy Program in Cape Town." AEA RCT Registry. November 11. https://doi.org/10.1257/rct.104-2.0

Former Citation

Bohmer, Bianca, Justine Burns and Lucas Crowley. 2013. "After-School Khan Academy Program in Cape Town." AEA RCT Registry. November 11. https://www.socialscienceregistry.org/trials/104/history/499

Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Experimental Details

Interventions

Intervention(s)

Numeric sessions run twice a week for 1.5 hours at a time during the course of the school year. The sessions are attended by Grade 8 students. The sessions are overseen by "coaches", most of whom are working towards a higher education degree. The sessions run immediately after school, in school computer labs. Students normally sit one to a computer. The sessions are not aligned with the South African curriculum. The focus is on improving numeracy. The sessions use online and offline versions of the Khan Academy program.

Intervention Start Date

2013-02-13

Intervention End Date

2013-11-14

Primary Outcomes

Primary Outcomes (end points)

Use of Khan Academy, math test scores, other subject marks, general cognitive skills, computer skills, English language skills

Primary Outcomes (explanation)

In addition to the overall score, the math section will be broken down into: arithmetic, fractions, decimals, patterns, and word problems, as well as grade-level questions. General cognitive ability will be measured by the Raven's Progressive Matrices test. The English measure tests vocabulary, oral comprehension, and reading comprehension. The computer measures include basic knowledge questions and correct keystrokes per minute.

Secondary Outcomes

Secondary Outcomes (end points)

Secondary Outcomes (explanation)

Experimental Design

The study employs a student-level randomization, stratifying on school as well as above/below median baseline total math score. We also checked for balance on the math subsections listed above, gender, home computer access, and travel time to school. We re-randomized if the random assignment created unbalance in these measures.

Experimental Design Details

Randomization Method

Stata command

Randomization Unit

Student

Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters

Not applicable

Sample size: planned number of observations

472 students

Sample size (or number of clusters) by treatment arms

236 treatment, 236 control

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

If we assume perfect compliance with random assignment, we should expect between a 0.26 and a 0.28 effect size. However, we expect take-up to be around 80% or less.

Supporting Documents and Materials

IRB