Teachers, Tutors, and Technology: A New Recipe for Facilitating Personalized Learning

Last registered on June 28, 2023

Pre-Trial

Trial Information

General Information

Title
Teachers, Tutors, and Technology: A New Recipe for Facilitating Personalized Learning
RCT ID
AEARCTR-0011639
Initial registration date
June 23, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
June 28, 2023, 4:32 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Affiliation
University of Toronto

Other Primary Investigator(s)

Additional Trial Information

Status
In development
Start date
2023-09-04
End date
2024-09-30
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Tutoring is one of the most effective policies for improving education but is often impractical because of scalability and cost constraints. One solution is to teach teachers to utilize more effectively Computer Assisted Learning (CAL) for simulating the tutoring experience at low cost. A recent evaluation of this approach (called Khoaching with Khan Academy) showed that elementary students with teachers randomly selected to receive this assistance did significantly better on standardized math test scores compared to students with teachers that did not receive the help. While successful in raising average performance, the study also revealed wide variance in CAL practice time within and across treated classrooms. A free after-school virtual tutoring program is proposed to further help this smaller subset of low-practicing students. Teachers will randomly invite parents of these children to receive weekly supervised CAL practice at home (through Khan Academy’s sister organization, Schoolhouse.World). It is expected that integrating CAL time during school, supervised by teachers, with CAL time after school, supervised by tutors, will substantially increase personalized learning and performance among students most in need of individualized help.
External Link(s)

Registration Citation

Citation
Oreopoulos, Philip. 2023. "Teachers, Tutors, and Technology: A New Recipe for Facilitating Personalized Learning." AEA RCT Registry. June 28. https://doi.org/10.1257/rct.11639-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
With support form superintendents of the Toronto District School Board, invitations will be sent to Grades 5-9 math teachers to participate in the experiment. All teachers will be assigned a khoach and participate in the regular KWiK program. The randomized tutoring component will happen concurrently from the third of week of school onwards. Among the lowest ~10 practicing students, approximately five students from each class will be randomly selected for the treatment group, offering them free virtual tutoring sessions once a week as an extra effort to improve their performance. Teachers will email parents of selected students to consider participating in the tutoring component of the program, with the student meeting once a week throughout the school year. The email will contain a special link to the program, housed at Schoohouse World. Schoolhouse is Khan Academy’s sister organization that offers free online volunteer tutoring using volunteers from around the world. Selected students will either be assigned a group tutoring session or a one-on-one tutoring session.

The primary objective of the tutoring sessions is to create a supportive learning environment where students can receive assistance in completing their current assignments. Typically, when students come to the sessions, they have already completed one or two out of their three current assignments, as they are given dedicated time to work on Khan Academy during class. However, some students may progress at a slower pace or require guidance, particularly if they have weaker foundations in math. Tutors are trained to prioritize the use of Khan Academy resources, such as hints, videos, and articles, instead of directly teaching or providing answers. If a student requires additional guidance, the tutor encourages them to start over in order to demonstrate their understanding and gain more practice. If a student has already completed all their current assignments before the tutoring session, tutors will assist the student in completing past exercises.

At the end of each session, the tutor will encourage the student to choose a past assignment to complete independently before the start of their next session. This practice is intended to promote accountability and foster independent learning in math. The overall goal of the tutoring program is to help students become confident, independent math learners, leading to positive long-term learning progress.
Intervention Start Date
2023-09-04
Intervention End Date
2024-06-30

Primary Outcomes

Primary Outcomes (end points)
Academic assessments (math grades in particular), and if possible survey data on attitudes towards school and mathematics subjects.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
With support form superintendents of the Toronto District School Board, invitations will be sent to Grades 5-9 math teachers to participate in the experiment. All teachers will be assigned a khoach and participate in the regular KWiK program. The randomized tutoring component will happen concurrently from the third of week of school onwards. Among the lowest ~10 practicing students, approximately five students from each class will be randomly selected for the treatment group, offering them free virtual tutoring sessions once a week as an extra effort to improve their performance. Teachers will email parents of selected students to consider participating in the tutoring component of the program, with the student meeting once a week throughout the school year. The email will contain a special link to the program, housed at Schoohouse World. Schoolhouse is Khan Academy’s sister organization that offers free online volunteer tutoring using volunteers from around the world. Selected students will either be assigned a group tutoring session or a one-on-one tutoring session.
Experimental Design Details
Not available
Randomization Method
randomization done in office by a computer
Randomization Unit
student
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
no clusters
Sample size: planned number of observations
Target is 1500 students
Sample size (or number of clusters) by treatment arms
750 control, 325 one on one tutoirng, 325 group tutoring
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
University of Toronto
IRB Approval Date
2023-04-06
IRB Approval Number
42223