The effects of high-dosage online tutoring on student outcomes.

Last registered on September 14, 2021

Pre-Trial

Trial Information

General Information

Title
The effects of high-dosage online tutoring on student outcomes.
RCT ID
AEARCTR-0008156
Initial registration date
August 26, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
August 30, 2021, 1:50 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
September 14, 2021, 9:38 PM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Primary Investigator

Affiliation
Brown University

Other Primary Investigator(s)

Additional Trial Information

Status
On going
Start date
2021-03-05
End date
2022-03-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
While a large body of research finds that in-person tutoring increases student achievement, little evidence exists concerning the effectiveness of online tutoring. This experiment assesses the effectiveness of a pilot high-dosage online tutoring program in Metro Nashville Public Schools on student achievement and attendance. Students assigned to treatment were paired with a volunteer college student or community volunteer and attended online tutoring sessions 3 times per week lasting 45 minutes during the school day for 5 to 6 weeks with their tutor. Tutors worked with students on relationship building and provided personalized academic tutoring. We will examine the impacts of this program on student achievement, grades, attendance, and behavior outcomes. Additionally, we will determine whether high-dosage online tutoring was more effective when delivered while kids are attending school in person or while they are attending school remotely.
External Link(s)

Registration Citation

Citation
Kraft, Matthew. 2021. "The effects of high-dosage online tutoring on student outcomes.." AEA RCT Registry. September 14. https://doi.org/10.1257/rct.8156-2.0
Experimental Details

Interventions

Intervention(s)
Students assigned to Treatment were paired with a trained volunteer college student (108 tutors) or community volunteer (20 tutors). Students attended online tutoring sessions 3 times per week lasting 45 minutes during the school day for 5 to 6 weeks with their tutor. Tutors worked with students on relationship building and provided personalized academic tutoring. Tutoring sessions occurred during the student’s Personalized Learning Time (PLT) which is 50 minutes during each school day where students are individually assigned to a learning intervention determined by their academic and social development. Students assigned to Control attended PLT as usual.

Three schools participated in the trial, one elementary school, one middle school, and one high school. In the elementary school, students in grades 3, 4, and 5 participated in the trial. Students in grades 6 and 7 attending the middle school and students in 9th grade at the high school participated in the study. In the middle and high schools, math was the subject that was the focus of tutoring. In the elementary school, literacy was the focus of tutoring.
Intervention Start Date
2021-03-08
Intervention End Date
2021-05-21

Primary Outcomes

Primary Outcomes (end points)
The primary outcomes are standardized test scores in the tutored subject (ELA or Math) measured using administrative data and course grades in tutored subjects. We believe the assessments will be the internal tests administered by the schools (e.g., MAP ELA and Math). We will also examine the state standardized test (e.g., Tennessee Comprehensive Assessment Program (TCAP/TNReady).
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
• School attendance between the beginning of the intervention period and the end of the school year.
• Number of behavior incidents between the beginning of the intervention period and the end of the school year.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Students were randomly assigned to Treatment or Control using a blocked randomization design. We randomly assigned students within school-grade cells, stratifying by students self-reported intention of finishing the school year either in-person or remotely. 139 (50%) students were assigned to treatment and 139 (50%) students were assigned to Control.
Experimental Design Details
Randomization Method
Randomization done in office by a computer using Stata.
Randomization Unit
Students
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
278 students. 148 (53%) students attended school virtually, 111 (40%) students attended school in person, and 19 (7%) students did not state their modality.
Sample size: planned number of observations
278 students. 148 (53%) students attended school virtually, 111 (40%) students attended school in person, and 19 (7%) students did not state their modality.
Sample size (or number of clusters) by treatment arms
139 (50%) students were assigned to treatment and 139 (50%) students were assigned to Control. Treatment compliance was not perfect. 7 students opted out of the program, 1 student left the program because of a move, and 2 students never showed up.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
At the currently available sample size (N = 278, 139 offered treatment), the minimum detectable effect size for an unconditional two-tailed t-test with power 1 – β = 0.8 and α = 0.05 of the difference in means for a variable that is standardized to have a mean of 0 and a standard deviation of 1 is 0.34 standard deviation for our baseline model. When we include student-level covariates, like prior test scores, that explain 50% of the variation in our outcome (0.5 R-squared), the minimum detectable effect size is 0.24 standard deviation.
IRB

Institutional Review Boards (IRBs)

IRB Name
Brown University, Office of Research Integrity, Human Research Protection Program
IRB Approval Date
2021-03-30
IRB Approval Number
N/A
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials