Reading Partners Evaluation

Last registered on February 06, 2017

Pre-Trial

Trial Information

General Information

Title
Reading Partners Evaluation
RCT ID
AEARCTR-0001908
Initial registration date
January 19, 2017

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
January 19, 2017, 11:04 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
February 06, 2017, 2:21 PM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
University of Michigan

Other Primary Investigator(s)

Additional Trial Information

Status
Completed
Start date
2012-08-01
End date
2015-03-01
Secondary IDs
Abstract
The Reading Partners program uses community volunteers to provide one-on-one tutoring to struggling readers in underresourced elementary schools. Established in 1999 in East Menlo Park, California, Reading Partners' mission is to help children become lifelong readers by empowering communities to provide individualized instruction with measurable results. At each school, Reading Partners transforms a dedicated space into a "reading center," places a full-time team member on site to manage day-to-day operations, and recruits a corps of 40 to 100 community volunteers to work one-on-one with students in pull-out sessions during the school day or after school in kindergarten through grade 5.

In March 2011, Reading Partners received a three-year True North Fund investment of up to $3.5 million in grants from the Edna McConnell Clark Foundation and the Social Innovation Fund, matched by $3.5 million from True North Fund co-investors, to further expand its early-intervention literacy program to elementary schools throughout the country and evaluate its effectiveness. The evaluation was conducted by MDRC during the 2012-2013 school year in 19 schools in three states, with more than 1,100 students randomly assigned to the study's program and control groups. This evaluation included only students in grades 2 through 5.

Key Findings

-- Despite the myriad difficulties inherent in operating a program whose direct service providers are volunteers, Reading Partners was implemented in the schools with a relatively high degree of fidelity to the program model. On average, students in the study received approximately 1.5 tutoring sessions per week, and spent 28 weeks in the Reading Partners program.

-- Reading Partners had a positive and statistically significant impact on three different measures of student reading proficiency. These impacts are equivalent to approximately one and a half to two months of additional growth in reading proficiency among the program group relative to the control group and are robust across a range of student characteristic subgroups as well as across groups of students who had different levels of reading comprehension skills at the start of the study.

-- Reading Partners is a low-cost option for underresourced schools because a majority of the costs are in-kind contributions, primarily from community volunteers. On average, schools bear only about 20 percent ($710 per program group student) of the total cost of the resources required to implement the program, and over half of these costs are in-kind contributions of space and staff time from the school.

Registration Citation

Citation
Jacob, Robin . 2017. "Reading Partners Evaluation." AEA RCT Registry. February 06. https://doi.org/10.1257/rct.1908-2.0
Former Citation
Jacob, Robin . 2017. "Reading Partners Evaluation." AEA RCT Registry. February 06. https://www.socialscienceregistry.org/trials/1908/history/13790
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
The Reading Partners program uses community volunteers to provide one-on-one tutoring to struggling readers in underresourced elementary schools. At each school, Reading Partners transforms a dedicated space into a "reading center," places a full-time team member on site to manage day-to-day operations, and recruits a corps of 40 to 100 community volunteers to work one-on-one with students twice a week for 45 minutes each in pull-out sessions during the school day or after school in kindergarten through grade 5. (This evaluation included only students in grades 2 through 5.)
Intervention Start Date
2012-09-01
Intervention End Date
2013-06-15

Primary Outcomes

Primary Outcomes (end points)
reading comprehension, fluency, sight word reading efficiency
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The impact study was designed to test the effect of the Reading Partners program on student reading proficiency -- that is, to determine whether the tutoring provided by the Reading Partners' volunteers increased the reading skills of the students who participated.

Although Reading Partners typically serves students in kindergarten through grade 5, the funding priorities of the granting institution, EMCF, typically focus on older children and adolescents. For that reason, the decision was made to include only second- through fifth-graders in this evaluation.

The sample for this study consists of 19 schools with Reading Partners reading centers that were recruited for the evaluation after Reading Partners senior staff identified them as potential participants. Only schools in which Reading Partners had been operating for at least one year before the study began were eligible, although the staff in these schools included both new and returning staff members. Schools were also eligible to participate only if Reading Partners anticipated that they would not be able to provide services to all the students who needed help with reading. This stipulation ensured that each program site was able to serve the same number of eligible students as it typically would, and did not deny services as a result of random assignment. A total of 31 schools were contacted during recruitment, and 19 of them joined the study. The 19 participating schools were spread across 12 school districts and 6 Reading Partners programmatic regions (New York City; Washington, DC; and, in California, East Bay, South Bay, Sacramento, and Los Angeles).

The study team used a systematic process to obtain and randomly assign the student sample at each school. At the start of the school year, schools and Reading Partners staff provided the study team with a list of students they wished to refer to the Reading Partners program, who had parental permission to participate in the program and the study. These lists were developed by individual schools and were based on a combination of test score review and recommendations by teachers or school staff (or both). Within two days of receiving the list, the study team randomly assigned students within schools (using a random number generator) to the program group that was eligible to participate in the Reading Partners program or to an "as is" control group that would not participate in Reading Partners but was eligible to receive any other supplemental reading services. A total of 1,265 students in the 19 participating schools were randomly assigned. Students were randomized within grade groups (grades 2 through 3 and grades 4 through 5) to ensure that equal numbers of upper and lower elementary school students were represented in the sample, since one of the research questions examines the differential impact of Reading Partners by grade level.

At the start of the school year, three assessments (SAT-10, TOWRE-2, AIMSweb) were given to all students in the study sample. The assessments measured reading comprehension, fluency, and sight word efficiency, all key components of early reading. These same three assessments were administered to students again in the spring, as close to the end of the school year as possible. In addition, in the spring of 2013, teachers were asked to complete a survey about the academic behavior and performance of each student in their classrooms who was part of the study sample (in both the program and control groups) and to report on the full range of supplemental reading services that each student had received during the school year.
Experimental Design Details
Randomization Method
Random number generator
Randomization Unit
individual (randomized within schools and within two grade groups: grades 2-3 and grades 4-5)
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
n/a
Sample size: planned number of observations
1,265 students
Sample size (or number of clusters) by treatment arms
619 students control, 646 students Reading Partners
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Yes
Data Collection Completion Date
Final Sample Size: Number of Clusters (Unit of Randomization)
n/a
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
1,166 students
Final Sample Size (or Number of Clusters) by Treatment Arms
554 students control, 580 students Reading Partners
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Abstract
Competence and confidence in reading constitute the foundation for all educational achievement. Students who struggle with reading inevitably struggle with all academic course work, and those who begin school behind their peers rarely catch up without significant intervention. Given the centrality of reading skills, the national statistics on literacy attainment are deeply distressing: two out of three American fourth-graders are reading below grade level, and almost one-third of children nationwide lack even basic reading skills. For children in low-income families, the numbers are even more alarming, with 80 percent reading below grade level. Despite several decades of educational reform efforts, only incremental progress has been made in addressing this crisis. From 1998 to 2013, the number of low-income fourth-graders reading at a proficient level increased by only 7 percentage points.

There are a plethora of literacy and reading programs that use a variety of methods and approaches designed to improve students' ability to read. Research suggests that among these, one-on-one tutoring has shown the greatest promise in improving reading performance. Such tutoring delivered by a certified teacher has consistently demonstrated large impacts on the reading proficiency of struggling readers. Yet its high cost means that it is often not a viable option for already underresourced schools. Using volunteers or paraprofessionals may be a more cost-effective approach to the problem, but to date little rigorous evidence exists regarding the efficacy of such an approach. Though prior research suggests that tutoring by volunteers can improve the reading proficiency of students who are falling behind, most of the studies that have been conducted have used very small sample sizes (generally fewer than 100 students, with only half receiving tutoring). Thus, even if their effectiveness has been established in studies using smaller samples, there is only limited evidence that such programs can be expanded to a large scale (for example, delivered to hundreds of students in multiple locations). In addition, research on the implementation and effectiveness of volunteer programs suggests that expanding them to a large scale might be quite difficult.

This policy brief tells the story of Reading Partners, a successful one-on-one volunteer tutoring program that serves struggling readers in low-income elementary schools and that has already been taken to a large scale. In the years since its inception, Reading Partners has grown to serve more than 7,000 students in over 130 schools throughout California, Colorado, New York, Oklahoma, Maryland, South Carolina, Texas, and Washington, DC. In March 2011, the program was awarded a three-year investment of up to $3.5 million from the Edna McConnell Clark Foundation and the Social Innovation Fund (SIF), matched by $3.5 million in grants from the True North Fund and co-investors, to further expand its literacy program to elementary schools throughout the country and to evaluate the effectiveness of the program. (Reading Partners has also been expanding with the support of AmeriCorps, a program of the Corporation for National and Community Service. AmeriCorps members provide teaching, mentoring, after-school support, and other services to students in more than 10,000 public schools, including one in three persistently low-achieving schools.)

This policy brief summarizes the early results of that evaluation, which was conducted during the 2012-2013 school year in 19 schools in three states, and which involved 1,265 students. The evaluation finds positive impacts of the program on three different measures of reading proficiency. These encouraging results demonstrate that Reading Partners, when delivered on a large scale and implemented with fidelity, can be an effective tool for improving reading proficiency.
Citation
Jacob, Robin Tepper, Thomas J. Smith, Jacklyn Willard, Rachel E. Rifkin. 2014. Reading Partners: The Implementation and Effectiveness of a One-on-One Tutoring Program Delivered by Community Volunteers. New York: MDRC.
Abstract
This study reports on an evaluation of the Reading Partners program, which uses community volunteers to provide one-on-one tutoring to struggling readers in underresourced elementary schools. Established in 1999 in East Menlo Park, California, Reading Partners' mission is to help children become lifelong readers by empowering communities to provide individualized instruction with measurable results. At each school, Reading Partners transforms a dedicated space into a "reading center," places a full-time team member on site to manage day-to-day operations, and recruits a corps of 40 to 100 community volunteers to work one-on-one with students in pull-out sessions during the school day or after school in kindergarten through grade 5. (This evaluation included only students in grades 2 through 5.)

In March 2011, Reading Partners received a three-year True North Fund investment of up to $3.5 million in grants from the Edna McConnell Clark Foundation and the Social Innovation Fund, matched by $3.5 million from True North Fund co-investors, to further expand its early-intervention literacy program to elementary schools throughout the country and evaluate its effectiveness. This report is the second publication from that evaluation. A policy brief released by MDRC in June 2014 reported the initial findings from the evaluation, which was conducted during the 2012-2013 school year in 19 schools in three states, with more than 1,100 students randomly assigned to the study's program and control groups.

This report builds on those initial findings by describing the Reading Partners program and its implementation in greater detail, exploring whether the program is more or less effective for particular subgroups of students, and assessing some of the potential explanations for the program's success to date. In addition, this report includes an analysis of the cost of implementing the Reading Partners program in 6 of the 19 sites.

Key Findings

-- Despite the myriad difficulties inherent in operating a program whose direct service providers are volunteers, Reading Partners was implemented in the schools with a relatively high degree of fidelity to the program model. On average, students in the study received approximately 1.5 tutoring sessions per week, and spent 28 weeks in the Reading Partners program.

-- Reading Partners had a positive and statistically significant impact on three different measures of student reading proficiency. These impacts are equivalent to approximately one and a half to two months of additional growth in reading proficiency among the program group relative to the control group and are robust across a range of student characteristic subgroups as well as across groups of students who had different levels of reading comprehension skills at the start of the study.

-- Reading Partners is a low-cost option for underresourced schools because a majority of the costs are in-kind contributions, primarily from community volunteers. On average, schools bear only about 20 percent ($710 per program group student) of the total cost of the resources required to implement the program, and over half of these costs are in-kind contributions of space and staff time from the school.
Citation
Jacob, Robin Tepper, Catherine Armstrong, and Jacklyn Altuna Willard, with A. Brooks Bowden and Yilin Pan. 2015. Mobilizing Volunteer Tutors to Improve Student Literacy: Implementation, Impacts, and Costs of the Reading Partners Program. New York: MDRC.

Reports & Other Materials