The Impact of Khan Academy on proficiency and attitudes towards math: Experimental Evidence from Brazil

The intervention is characterized by promoting the use of Khan Academy, an online platform which offers mathematics instructional videos and exercises in a personalized environment, during math classes, following the teachers' instructions. Math teachers of the grades allocated to treatment receive a one-day training to get familiar with the platform and explore their functionalities. All students allocated to treatment grades are assigned a username and password to access the platform. Schools that have less than 0.5 computer per student (irrespectively of treatment status) are granted new ones from the Lemann Foundation.

2017-04-03

2017-11-10

Students' proficiency in math (measured through the national standardized exam grades) and attitudes towards math

Attitudes towards math will be measured through the instrument developed by Aiken Jr & Dreger (1961) and translated to Portuguese and validated by Brito (1998).

The study is composed by 166 primary education schools in the cities of Pelotas (30 schools), Manaus (64 schools), Barueri (25 schools), Mogi das Cruzes (30 schools) and Sao Bernando do Campo (17 schools), all of which have voluntarily applied to participate in the program.The randomization was undertaken in the beginning of the school year at grade level and was stratified in the schools' previous math scores at the national standard test (Prova Brasil). 3rd, 5th, 6th and 9th grades were eligible for receiving the treatment. Schools may receive one or two treated grades, depending on whether they offer only one or two cycles of primary education. Every school in the sample receives at least one treatment and one control grade. The evaluation will be focused on students in the 5th and 9th grades only, since for these students we will have a national exam based on which we will measure proficiency (Prova Brasil 2017). Follow-up data collection will ask about knowledge of the intervention to allow the analysis to potentially focus on subgroups with less contamination if necessary.

Randomization done in office by computer, carried out by team of researchers.

School grade level

Yes

166 schools with 229 school grades in the evaluation

14 000 students

There is only one treatment arm. All 166 schools have at least one treatment and one control grade, but only 5th and 9th grades are part of this evaluation. There is a total of 229 school grades in the evaluation.

The 166 schools are divided into 88 schools of cycle 1(1st to 5th grades), 63 of cycles 1 and 2 (all grades) and 15 of cycle 2 (5th-9th grades). Among these, we have:
Schools of cycle 1: 44 schools with treatment in 5th grade and 44 schools with control in 5th grade.
Schools of cycle 1 and 2: 32 schools with treatment in 5th grade and control in 9th grade, 31 schools with control in 5th grade and treatment in 9th grade.
Schools of cycle 2: 8 schools with treatment in 5th grade and 7 schools with control in 5th grade.

Attitudes towards math index: MDE = 0.041 standard deviation Grade in Math Exam on the National Standardized exam (Prova Brasil): MDE = 0.11 standard deviation (2.40 points in SAEB scale)