The Impact of Khan Academy on proficiency and attitudes towards math: Experimental Evidence from Brazil

Last registered on October 03, 2017

Pre-Trial

Trial Information

General Information

Title
The Impact of Khan Academy on proficiency and attitudes towards math: Experimental Evidence from Brazil
RCT ID
AEARCTR-0002456
Initial registration date
October 03, 2017

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
October 03, 2017, 3:03 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
Sao Paulo School of Economics - FGV

Other Primary Investigator(s)

PI Affiliation
FGV São Paulo School of Business
PI Affiliation
Yale
PI Affiliation
São Paulo School of Economics - FGV

Additional Trial Information

Status
On going
Start date
2017-03-13
End date
2018-12-21
Secondary IDs
Abstract
Researchers will carry out a randomized evaluation in partnership with the Lemann Foundation to measure the impacts of the implementation and use of a math platform (Khan Academy) in a sample of Brazilian public schools on students' proficiency in math and attitudes towards math.
External Link(s)

Registration Citation

Citation
Ferman, Bruno et al. 2017. "The Impact of Khan Academy on proficiency and attitudes towards math: Experimental Evidence from Brazil." AEA RCT Registry. October 03. https://doi.org/10.1257/rct.2456-1.0
Former Citation
Ferman, Bruno et al. 2017. "The Impact of Khan Academy on proficiency and attitudes towards math: Experimental Evidence from Brazil." AEA RCT Registry. October 03. https://www.socialscienceregistry.org/trials/2456/history/22009
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
The intervention is characterized by promoting the use of Khan Academy, an online platform which offers mathematics instructional videos and exercises in a personalized environment, during math classes, following the teachers' instructions. Math teachers of the grades allocated to treatment receive a one-day training to get familiar with the platform and explore their functionalities. All students allocated to treatment grades are assigned a username and password to access the platform. Schools that have less than 0.5 computer per student (irrespectively of treatment status) are granted new ones from the Lemann Foundation.
Intervention (Hidden)
Intervention Start Date
2017-04-03
Intervention End Date
2017-11-10

Primary Outcomes

Primary Outcomes (end points)
Students' proficiency in math (measured through the national standardized exam grades) and attitudes towards math
Primary Outcomes (explanation)
Attitudes towards math will be measured through the instrument developed by Aiken Jr & Dreger (1961) and translated to Portuguese and validated by Brito (1998).

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The study is composed by 166 primary education schools in the cities of Pelotas (30 schools), Manaus (64 schools), Barueri (25 schools), Mogi das Cruzes (30 schools) and Sao Bernando do Campo (17 schools), all of which have voluntarily applied to participate in the program.The randomization was undertaken in the beginning of the school year at grade level and was stratified in the schools' previous math scores at the national standard test (Prova Brasil). 3rd, 5th, 6th and 9th grades were eligible for receiving the treatment. Schools may receive one or two treated grades, depending on whether they offer only one or two cycles of primary education. Every school in the sample receives at least one treatment and one control grade. The evaluation will be focused on students in the 5th and 9th grades only, since for these students we will have a national exam based on which we will measure proficiency (Prova Brasil 2017). Follow-up data collection will ask about knowledge of the intervention to allow the analysis to potentially focus on subgroups with less contamination if necessary.



Experimental Design Details

Randomization Method
Randomization done in office by computer, carried out by team of researchers.
Randomization Unit
School grade level
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
166 schools with 229 school grades in the evaluation
Sample size: planned number of observations
14 000 students
Sample size (or number of clusters) by treatment arms
There is only one treatment arm. All 166 schools have at least one treatment and one control grade, but only 5th and 9th grades are part of this evaluation. There is a total of 229 school grades in the evaluation.

The 166 schools are divided into 88 schools of cycle 1(1st to 5th grades), 63 of cycles 1 and 2 (all grades) and 15 of cycle 2 (5th-9th grades). Among these, we have:
Schools of cycle 1: 44 schools with treatment in 5th grade and 44 schools with control in 5th grade.
Schools of cycle 1 and 2: 32 schools with treatment in 5th grade and control in 9th grade, 31 schools with control in 5th grade and treatment in 9th grade.
Schools of cycle 2: 8 schools with treatment in 5th grade and 7 schools with control in 5th grade.

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Attitudes towards math index: MDE = 0.041 standard deviation Grade in Math Exam on the National Standardized exam (Prova Brasil): MDE = 0.11 standard deviation (2.40 points in SAEB scale)
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB - EAESP - FGV
IRB Approval Date
2016-10-04
IRB Approval Number
n/a

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials