Khollaboration with Khan Academy: Leveraging Computer Assisted Learning, Teachers, and Parents to Facilitate Personalized Learning at Scale
Last registered on June 17, 2021

Pre-Trial

Trial Information
General Information
Title
Khollaboration with Khan Academy: Leveraging Computer Assisted Learning, Teachers, and Parents to Facilitate Personalized Learning at Scale
RCT ID
AEARCTR-0007682
Initial registration date
June 17, 2021
Last updated
June 17, 2021 10:33 AM EDT
Location(s)

This section is unavailable to the public. Use the button below to request access to this information.

Request Information
Primary Investigator
Affiliation
University of Notre Dame
Other Primary Investigator(s)
PI Affiliation
PI Affiliation
PI Affiliation
Additional Trial Information
Status
In development
Start date
2021-06-21
End date
2023-07-31
Secondary IDs
Abstract
This research investigates the effectiveness of providing teachers with supports for integrating Khan Academy programming with their math instruction. The study will be conducted as a randomized control trial comparing math achievement of students whose teachers have been offered support from a Khan Academy “Khollaborator” to students whose teachers do not receive the Khollaborator support and conduct their math instruction as usual. Teachers in the treatment group will also receive regular progress reports and support for communicating with parents and students. Results will provide information about how to most effectively align Khan Academy with classroom efforts, and how doing so impacts students’ math achievement.
External Link(s)
Registration Citation
Citation
Brough, Rebecca et al. 2021. "Khollaboration with Khan Academy: Leveraging Computer Assisted Learning, Teachers, and Parents to Facilitate Personalized Learning at Scale." AEA RCT Registry. June 17. https://doi.org/10.1257/rct.7682-2.0.
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details
Interventions
Intervention(s)
The program, called “Khollaboration With Khan Academy” (KWiK), aims to increase student engagement with Khan Academy to provide students with a greater personalized learning experience, allowing students to receive more immediate feedback, progress at their own pace, and focus on mastering course material. To effectively integrate Khan Academy into teachers’ classrooms, treatment group teachers will participate in training prior to the start of the school year and university student coaches (Khollaborators) will regularly communicate with teachers to offer support and guidance. Khollaborators will also provide weekly progress updates to students in treatment group classrooms and their parents.
Intervention Start Date
2021-07-15
Intervention End Date
2022-06-30
Primary Outcomes
Primary Outcomes (end points)
1) Math skill development as measured through Khan Academy progress and performance
2) Math achievement as measured through interim and year-end standardized tests
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Secondary outcomes of interest include student attendance in school, disciplinary incidents, on-time grade progression, and student and teachers’ satisfaction.
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
The research team will employ a randomized control trial to answer the research question: What is the impact of Khollaboration with Khan Academy (KWiK) training for teachers on students’ math achievement? To answer this, researchers will randomize assignment to KWiK at the grade-school level. Students whose teachers are assigned to KWiK will comprise the treatment group, whereas students assigned to teachers not offered training are the control group. Teachers who sign up to participate and are not offered KWiK in the 2021-22 school year will receive the intervention the following year (SY 2022-23).

The primary outcome for both is students’ performance on math standardized assessments. The research team will also use data from Khan Academy to measure usage (in minutes) and performance on the Khan Academy platform (number of level-ups). Secondary outcomes of interest include student attendance in school, and student and teachers’ satisfaction.
Experimental Design Details
Not available
Randomization Method
Teachers will be randomized at the school-grade level. Researchers will use Stata to assign each school-grade level unit to a random number uniformly distributed between 0 and 1. This number will be replicable using the associated seed number. Randomization will be stratified at the grade level, meaning that each grade-school unit will be sorted by its random number and ranked in order relative to other grade-school units within the same grade. Grade-school units with a ranking below the 50 percentile will be considered treated, grade-school units with a ranking above the 50th percentile will be considered control.

Randomization Unit
Randomization will occur at the school-grade level. Randomization will be stratified at the grade level.
Was the treatment clustered?
Yes
Experiment Characteristics
Sample size: planned number of clusters
Based on current interest survey data, there will be approximately 180 grade-school units.
Sample size: planned number of observations
Based on current survey data, the study will have approximately 300 teacher participants. Assuming each classroom has approximately 25 students, we expect a study size of 5500 students.
Sample size (or number of clusters) by treatment arms
We plan to randomize school-grade level clusters using a 50-50 ratio.

We expect to have approximately 4500 students in treatment group classrooms and 4500 students in control group classrooms.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Given the clustered design, the study is sufficiently powered to detect an MDE on test-score outcomes ranging from 0.13-0.17 standard deviations (rho ranging from 0.15-0.30; R^2 of covariates = 0.5, n in each cluster = 50), which is in line with effects on math achievement in the existing literature from similar or related interventions.
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
University of Notre Dame Research Compliance
IRB Approval Date
2021-05-05
IRB Approval Number
21-04-6558
Analysis Plan

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information