The Enhanced Reading Opportunities Study

Last registered on September 30, 2016

Pre-Trial

Trial Information

General Information

Title
The Enhanced Reading Opportunities Study
RCT ID
AEARCTR-0001490
Initial registration date
September 30, 2016

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
September 30, 2016, 2:54 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
MDRC

Other Primary Investigator(s)

PI Affiliation
MDRC

Additional Trial Information

Status
Completed
Start date
2005-08-01
End date
2010-07-01
Secondary IDs
Abstract
The Enhanced Reading Opportunities Study tests the effectiveness of two supplemental literacy interventions targeted to striving ninth-grade readers — those with reading comprehension skills that are two to five years below grade level. The interventions that were selected before the start of the 2005 school year were (1) Reading Apprenticeship for Academic Literacy from WestEd and (2) Xtreme Reading from the University of Kansas Center for Research on Learning. This national study has unfolded over a five-year period and addresses the following questions:

• What challenges and successes did the high schools experience in implementing the reading interventions?
• What are the effects of these interventions on students’ reading skills and on other academic outcomes?
• For which subgroups of students are the programs most effective?
• What are the costs of these programs?

Overall, the findings from these reports show that over the course of ninth grade, the ERO programs improved students’ reading comprehension skills and helped them perform better academically in their high school course work. However, these benefits did not persist in the following school year, when students were no longer receiving the supports provided by the ERO programs. The key findings from the study follow (the statistical significance of all impact estimates in this report is evaluated at the 5 percent level):

• The ERO programs improved students’ reading comprehension skills over the course of ninth grade. Across both cohorts of participating ninth-grade students, the ERO programs improved students’ reading comprehension scores by an effect size of 0.09, corresponding to an improvement from the twenty-third percentile to the twenty-fifth percentile nationally. However, 77 percent of students assigned to the ERO classes were still reading at two or more years below grade level at the end of ninth grade.

• During the ninth grade, the ERO programs also had a positive impact on students’ academic performance in core subject areas. Students’ GPA in core subject areas (English language arts, social studies, science, and mathematics) was 0.06 point higher (out of a maximum of 4 points) as a result of being assigned to the ERO program (effect size = 0.07). The programs also helped students earn 0.6 percentage point more of the core credits that they need to graduate (effect size = 0.06). In the subset of high schools located in states where standardized tests are administered in ninth grade, students also scored higher on their English language arts and mathematics tests as a result of having been assigned to the ERO program; the estimated effect size of these impacts are 0.11 and 0.07, respectively.

• However, in the school year following students’ participation in the ERO programs, the programs no longer had an impact on academic performance. Estimated impacts on students’ GPA in core subject areas, credit accumulation, and standardized state test scores are not statistically significant in the school year following program participation (tenth grade for most students).

• The ERO programs did not increase students’ vocabulary scores, nor did the programs affect students’ reading behaviors or their school behaviors. The programs did not have a statistically significant impact on students’ vocabulary scores at the end of ninth grade. Nor did the programs have a statistically significant effect on how often students read school-related or non-school-related texts, or on how often students use the reading strategies taught by the two programs. Similarly, impacts on student attendance and suspensions were not statistically significant, in either the program year or the following school year.
External Link(s)

Registration Citation

Citation
Corrin, William and Marie-Andree Somers. 2016. "The Enhanced Reading Opportunities Study." AEA RCT Registry. September 30. https://doi.org/10.1257/rct.1490-1.0
Former Citation
Corrin, William and Marie-Andree Somers. 2016. "The Enhanced Reading Opportunities Study." AEA RCT Registry. September 30. https://www.socialscienceregistry.org/trials/1490/history/10949
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
In Enhanced Reading Opportunities (ERO), schools were randomly assigned to one of two supplemental literacy programs — Reading Apprenticeship Academic Literacy and Xtreme Reading — that aim to improve the reading comprehension skills and school performance of struggling ninth-grade readers. The programs were selected through a competitive applications process based on ratings by an expert panel. The supplemental literacy programs are full-year courses targeted to students whose reading skills are two to five years below grade level as they enter high school. The ERO class, designed to serve 12-15 students, replaces a ninth-grade elective, and it is offered in addition to students’ regular English language arts classes. The programs seek to help ninth-grade students learn and employ the strategies used by proficient readers, improve their comprehension skills, and increase their motivation to read more and to enjoy what they read. For this demonstration, one teacher at each school was trained to teach the literacy program, and she or he exclusively taught the course to four sections of students.
Intervention Start Date
2005-08-15
Intervention End Date
2007-06-15

Primary Outcomes

Primary Outcomes (end points)
Reading comprehension test scores, vocabulary test scores, and self-reported reading behaviors at the end of ninth grade; grade point average, credit accumulation, state test scores, school attendance and suspensions during ninth grade and in the following school year.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The study sample consisted of two cohorts of ninth-grade students from 34 high schools and 10 school districts (2,916 students in Cohort 1 and 2,679 students in Cohort 2). Districts and schools were selected by ED’s Office of Vocational and Adult Education through a special Small Learning Communities Grant competition. Students were selected based on reading comprehension test scores that were between two and five years below grade level.

Within each district, high schools were randomly assigned to use either the RAAL program or the Xtreme Reading program during two school years (2005-2006 and 2006-2007). Within each high school, students were randomly assigned to enroll in the ERO class (ERO group) or to remain in a regularly scheduled elective class (non-ERO group). Because students were randomly assigned to the ERO program, the impact of the programs can be estimated by comparing the outcomes of students in the ERO and the non-ERO group. Impact estimates are regression-adjusted for the blocking of random assignment as well as random baseline differences between the ERO and non-ERO group.

Classroom observations in the first and second semester of the school year were used to measure implementation fidelity. A reading comprehension test — the Group Reading Assessment and Diagnostic Examination (GRADE) — and a survey were administered to students in the spring of eighth grade or at the start of ninth grade prior to random assignment, and again at the end of ninth grade. School records data were collected for students’ ninth-grade and tenth-grade year (and for Cohort 1’s eleventh-grade year); these data include course transcripts, state test scores, attendance, and disciplinary outcomes.
Experimental Design Details
Randomization Method
Randomization was conducted by MDRC staff members in the office by computer.
Randomization Unit
High schools were randomly assigned to one of the two reading interventions. Within each school, students were randomly assigned to the treatment or control group.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
Two cohorts of ninth-grade students from 34 high schools and 10 school districts.
Sample size: planned number of observations
5,595 ninth-grade students (2,916 students in Cohort 1 and 2,679 students in Cohort 2)
Sample size (or number of clusters) by treatment arms
In the RAAL group, there were 1,602 students in the ERO group and 1,160 in the control group. In the Xtreme Reading group, there were 1,615 students in the ERO group and 1,218 students in the control group.

Clusters: 17 schools RAAL, 17 schools Xtreme Reading.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Yes
Data Collection Completion Date
Final Sample Size: Number of Clusters (Unit of Randomization)
Was attrition correlated with treatment status?
Final Sample Size: Total Number of Observations
Final Sample Size (or Number of Clusters) by Treatment Arms
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
No
Reports, Papers & Other Materials

Relevant Paper(s)

Abstract
The key findings from the study follow (the statistical significance of all impact estimates in this report is evaluated at the 5 percent level):
•The ERO programs improved students’ reading comprehension skills over the course of ninth grade. Across both cohorts of participating ninth-grade students, the ERO programs improved students’ reading comprehension scores by an effect size of 0.09, corresponding to an improvement from the twenty-third percentile to the twenty-fifth percentile nationally. However, 77 percent of students assigned to the ERO classes were still reading at two or more years below grade level at the end of ninth grade.
•During the ninth grade, the ERO programs also had a positive impact on students’ academic performance in core subject areas. Students’ GPA in core subject areas (English language arts, social studies, science, and mathematics) was 0.06 point higher (out of a maximum of 4 points) as a result of being assigned to the ERO program (effect size = 0.07). The programs also helped students earn 0.6 percentage point more of the core credits that they need to graduate (effect size = 0.06). In the subset of high schools located in states where standardized tests are administered in ninth grade, students also scored higher on their English language arts and mathematics tests as a result of having been assigned to the ERO program; the estimated effect size of these impacts are 0.11 and 0.07, respectively.
•However, in the school year following students’ participation in the ERO programs, the programs no longer had an impact on academic performance. Estimated impacts on students’ GPA in core subject areas, credit accumulation, and standardized state test scores are not statistically significant in the school year following program participation (tenth grade for most students).
•The ERO programs did not increase students’ vocabulary scores, nor did the programs affect students’ reading behaviors or their school behaviors. The programs did not have a statistically significant impact on students’ vocabulary scores at the end of ninth grade. Nor did the programs have a statistically significant effect on how often students read school-related or non-school-related texts, or on how often students use the reading strategies taught by the two programs. Similarly, impacts on student attendance and suspensions were not statistically significant, in either the program year or the following school year.
Citation
Somers, Marie-Andrée, William Corrin, Susan Sepanik, Terry Salinger, Jesse Levin, and Courtney Zmach. 2010. The Enhanced Reading Opportunities Study Final Report: The Impact of Supplemental Literacy Courses for Struggling Ninth-Grade Readers (NCEE 2010-4022). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.

Reports & Other Materials