The effects of high-dosage tutoring at-scale on student outcomes.

Last registered on September 14, 2021

Pre-Trial

Trial Information

General Information

Title
The effects of high-dosage tutoring at-scale on student outcomes.
RCT ID
AEARCTR-0008233
Initial registration date
September 13, 2021
Last updated
September 14, 2021, 4:42 PM EDT

Locations

Primary Investigator

Affiliation
Brown University

Other Primary Investigator(s)

Additional Trial Information

Status
In development
Start date
2021-08-14
End date
2023-07-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Although evaluations of tutoring programs find large effects on student achievement, few programs have been successfully implemented at-scale. We examine the effectiveness of a district-wide high-dosage tutoring initiative in a large urban school district that will serve 7,000 students by the end of the 2021-22 school year. Students participating in the program during the Fall 2021 semester will be paired with a teacher, paraprofessional, college student, or community volunteer and attend tutoring sessions 3 times per week for 30-45 minutes per session and on a recurring basis for 10 weeks. To determine the effectiveness of this tutoring program, we randomize 68 schools to either begin participation in the tutoring program in Fall 2021 or Spring 2022. We estimate the impacts of participating in this program in Fall 2021 on Fall semester achievement, attendance, and behavior outcomes.
External Link(s)

Registration Citation

Citation
Kraft, Matthew. 2021. "The effects of high-dosage tutoring at-scale on student outcomes. ." AEA RCT Registry. September 14. https://doi.org/10.1257/rct.8233-1.0
Experimental Details

Interventions

Intervention(s)
68 schools were chosen to either begin participation in a tutoring program in Fall 2021 or Spring 2022 at random. Schools assigned to treatment will begin participation in Fall 2021, between September 13th and September 24th. Control group schools will participate in the tutoring program in Spring 2022.

1st, 2nd, and 3rd grade students in the trial schools are eligible to participate in the trial if they score in the 15th-60th percentiles of the Spring 2021 reading assessment. 8th and 9th grade students in the trial schools are eligible to participate in the trial if they score in the 15th-60th percentiles of the Spring 2021 math assessment. 1st, 2nd, and 3rd grade students in the treatment schools will receive tutoring in reading while 8th and 9th grade students in the treatment schools will be tutored in math during the Fall 2021 semester.
Due to a rolling tutor recruitment approach as well as a limited supply of tutors, the school district will be entering eligible students into the student-tutor matching pool in batches from lowest to highest Spring 2021 test score percentile within each school until there are no more tutors. Specifically, we will start with students with the lowest test scores (at or above the 15th percentile) within each school and release blocks of students going up the test score distribution at each school until there are no more tutors left in the pool. In practice, this may mean that eligible students with higher Spring 2021 test scores will not be offered a tutor. Tutor recruitment will be ongoing until September 24th.

Students participating for tutoring will be assigned one of the following tutors: a district teacher, a district paraprofessional, or a volunteer that is background checked and trained. The volunteer tutor sources include: college undergraduates, graduate students, and business volunteers. Teacher/student ratios will be no more than 4:1, paraprofessional/student ratios will be no more than 2:1, and volunteer/student ratios will be 1:1. Tutoring will take place online during the student’s daily personalized learning time and the teacher/paraprofessional’s planning period.

All tutor/student pairs will meet for three sessions a week for 30-45 minutes per session and on a recurring basis for 10 weeks in each semester the school is participating in the program. Tutors will utilize a curriculum that contains daily lesson plans and practice problems that will drive the content of the tutoring sessions.
Intervention Start Date
2021-09-13
Intervention End Date
2022-05-20

Primary Outcomes

Primary Outcomes (end points)
The primary outcomes are fall semester standardized test scores and course grades in the tutored subject (ELA or Math). Test scores will be measured using administrative data for students in 2nd, 3rd, 8th, and 9th grades ( we expect test scores in 1st grade will not be available). We believe these will be the internal tests administered by the schools in November 2021 (e.g., MAP ELA and Math).
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
• School attendance between the beginning of the intervention period and the end of the Fall 2021 semester.
• Number of behavior incidents between the beginning of the intervention period and the Fall 2021 semester.
• Student survey responses to questions about socio-emotional skills, sense of belonging at school, school engagement, and student-teacher relationships for students in 3rd, 8th, and 9th grades.
• Contingent on program implementation in Spring 2022, we will use Spring 2022 test scores, grades, attendance, and behavior incidents as secondary outcomes.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
90 schools (13 high schools, 25 middle schools, and 52 elementary schools) were selected to participate in the 2021-22 high-dosage tutoring program. Target schools were identified by developing a model to rank each school based on need (as defined by Spring 2020 achievement scores and capacity to offer tutoring). Using the ranking as a prioritized list, the 90 schools with the highest need were selected for participation in the trial.

45 schools will start the tutoring program in Fall 2021 and 45 schools will start in Spring 2022. 16 schools with the highest need were selected to start in Fall 2021. 6 schools participating in another tutoring program were assigned to begin the program in Spring 2022. The other 68 schools that were randomly assigned to start the tutoring program in Fall 2021, the treatment, or beginning participation in Spring 2022, the control, using a blocked randomization design. We randomly assigned schools within school type, elementary, middle, or high school. In accordance with district goals, we used an unbalanced probability of assignment to treatment. 29 (43%) of schools were selected for treatment. 39 schools were selected for the control group (57%).
Experimental Design Details
Not available
Randomization Method
Randomization done in office by a computer using Stata.
Randomization Unit
Schools
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
68 schools
Sample size: planned number of observations
5,733 students
Sample size (or number of clusters) by treatment arms
29 (43%) of schools were selected for treatment. 39 school were selected for the control group (57%).
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
At the estimated sample size (N = 5,733 students nested in 68 schools randomized within 3 blocks, 29 offered treatment), the minimum detectable effect size for an unconditional two-tailed t-test with power 1 – β = 0.8, α = 0.05, and an inter-cluster correlation of 0.1 of the difference in means for a variable that is standardized to have a mean of 0 and a standard deviation of 1 is 0.23 standard deviation for our baseline model. When we include student-level covariates, like prior test scores, that explain 50% of the variation in our outcome (0.5 R-squared), and school covariates that explain 20% of the variation in our outcome (0.2 R-squared) the minimum detectable effect size is 0.20 of a standard deviation.
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information