Impact of Short-Term Tutoring on Cognitive and Non-Cognitive Skills in Chile

Last registered on February 22, 2017

Pre-Trial

Trial Information

General Information

Title
Impact of Short-Term Tutoring on Cognitive and Non-Cognitive Skills in Chile
RCT ID
AEARCTR-0001480
Initial registration date
February 22, 2017

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
February 22, 2017, 7:36 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
PUC-Chile

Other Primary Investigator(s)

PI Affiliation
University of Chicago (formerly J-PAL)
PI Affiliation
PUC-Chile

Additional Trial Information

Status
Completed
Start date
2010-08-02
End date
2010-12-31
Secondary IDs
Abstract
A randomized evaluation in Chile …finds that providing a 3-month program of small group tutoring to fourth graders using college student volunteers raise language scores and measures of attitudes towards reading for some subgroups of the population. In particular, students from low-performing and poor schools in areas in which the program was implemented increased their performance in a reading test by between 0.15 and 0.20 standard deviations and improved significantly their self-perceptions as readers. Researchers also present cost-effectiveness analyses and …find that for the students for which the program was effective the program was in the range of the current estimates.
External Link(s)

Registration Citation

Citation
Cabezas, Veronica, José I. Cuesta and Francisco Gallego. 2017. "Impact of Short-Term Tutoring on Cognitive and Non-Cognitive Skills in Chile." AEA RCT Registry. February 22. https://doi.org/10.1257/rct.1480-1.0
Former Citation
Cabezas, Veronica, José I. Cuesta and Francisco Gallego. 2017. "Impact of Short-Term Tutoring on Cognitive and Non-Cognitive Skills in Chile." AEA RCT Registry. February 22. https://www.socialscienceregistry.org/trials/1480/history/14334
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
In order to improve reading comprehension and attitudes toward reading in fourth grade students from vulnerable schools, the Chilean Ministry of Education partnered with the Fundación para la Superación de la Pobreza (FSP) in 2010 to implement Servicio Pas en Educación (SPE). In the poorest counties of the Santiago and Bío-Bío regions, the Chilean Ministry of Education selected 85 public and private schools that had scored below the average on the language portion of Chile' s national standardized test. From these 85 schools, 45 schools were randomly selected to receive SPE and the remaining 40 served as comparison schools.

SPE brought college student volunteers into primary schools, where they conducted 90-minute reading sessions with groups of 5 or 6 fourth grade students. The sessions included a mix of traditional stories and instructional texts appropriate for the students' age. Volunteers were managed by a paid employee of FSP stationed permanently in each program school to ensure accurate implementation and provide assistance on effective teaching techniques. The program lasted for 15 weeks.

Surveys gathered information on reading comprehension and attitudes towards reading, as well as program implementation details, and individual school and student characteristics. The implementation of the program was monitored through random visits to observe tutoring sessions and administrative data on student, volunteer, and supervisor attendance.
Intervention Start Date
2010-09-01
Intervention End Date
2010-12-31

Primary Outcomes

Primary Outcomes (end points)
1) Scores on the Prueba de Comprensión Lectora y Producción de Textos (Reading Comprehension and Texts Production Test, CLPT), which measures formal reading skills. (2) Scores on the Gusto por la Lectura (Taste for Reading, GPL), which measures attitudes towards reading.
Primary Outcomes (explanation)
Measure reading cognitive skills and attitudes towards reading

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The Servicio País en Educación (SPE) program emerged from a partnership between the Chilean Ministry of Education (Mineduc) and the Fundación para la Superación de la Pobreza (Poverty Alleviation Foundation, FSP) and was first implemented during the period between September and December of 2010. The main objective of the program was to improve both attitudes toward reading (IR) and reading comprehension (RC) in 4th graders from vulnerable schools. Additionally, the program aimed at intervening in two additional dimensions: to generate new learning environments in which to implement the program, being this aspect influenced by the earthquake that affected the central region of Chile in February, 2010. Secondly, SPE intended to include college students as direct intervention agents. However, this evaluation only answers the questions related to the impact of the program on IR and RC.

The SPE program targeted vulnerable schools of the central region of Chile. Additionally, a number of logistic restrictions were put by FSP. Researchers excluded all schools from counties in which the FSP either was not able to work because they had no human resources in them or had already committed with some schools in it, which made randomization impossible there. This reduced the number of counties from which schools were included in the evaluation to 10: Santiago, Estación Central, Lo Espejo, Maipú, La Florida and San Bernardo from the Great Santiago region (Metropolitan Region, RM), and Concepción, Coronel, Hualpén and Talcahuano from the Biobio Region (VIII). In some of those counties, an additional restriction was set in terms of the administrative dependence of the schools, restricting us to include either only public schools (P) in some counties or only private subsidized schools (PS) in other ones. In addition, to fit the operational model designed by the FSP, each of the included schools had to have at least 90 students in fourth grade.

Using this sample, schools were randomly assigned to treatment and control groups, stratifying by county, socioeconomic group and SIMCE test scores. As the eligible school set was larger than the number of required schools, only some of the schools assigned to each group were included in the evaluation, decision that was random too. The remaining schools were kept as replacement lists for the eventual rejection of schools to take part of the evaluation. With the results of this assignment, schools were contacted and invited to take part of the evaluation in their corresponding group. Five of them rejected the program but, except in two cases, all of them accepted to be evaluated anyway. Additionally, two schools in the control group rejected to be evaluated. All of these schools were randomly replaced by another schools coming from the replacement lists. The treatment and control groups were finally composed of 45 and 40 schools respectively, grouped in 25 and 24 units.

Baseline data on the students were collected before the start of the program in August 2010, and follow-up data were collected after the program finished in December 2010. Researchers included two instruments. (1) The Prueba de Comprensión Lectora y Producción de Textos (Reading Comprehension and Texts Production Test, CLPT) measures formal reading skills, i.e., Reading Comprehension (RC), Texts Production (TP), and use of Language (UL). (2) The Gusto por la Lectura (Taste for Reading, GPL) measures attitudes towards reading along the four dimensions: Interest for Reading (IR), Self- perception as a Reader (SPR), Enjoyableness for Reading (ER) and Perception of Reading at School (PRS). These four indexes move discretely between 0 and 3, where 0 is the most negative of the offered alternatives and 3 is the most positive of the offered alternatives. Researchers also collected information on the program operation to understand the reasons behind the heterogeneous impacts of the program in different subpopulations.

First, researchers monitored the implementation of the program with random visits to observe the actual tutoring sessions. Second, they gathered administrative information about student, tutor, and professional assistance to tutoring sessions and about the numbers of sessions received by each student in the program. Finally, researchers collected data from the Ministry of Education on both schools’ and students’ and characteristics. Regarding schools, researchers collected data on Language and Math SIMCE test scores, average mothers’ years of schooling, average household income, socioeconomic level, a school vulnerability index (IVE), and administrative dependence. Regarding students, researchers collected information on gender, grades for 2009 and school attendance for 2009 and 2010, coming them from Mineduc records too.

Researchers estimated the effect of the program by comparing average outcomes of the treatment and the control group. Additionally, they perform some statistical exercise in order to understand the relationship between the two tests results. To assess the Intention-to-Treat (ITT) and the Treatment-on-the-Treated (TT) effects on the CLPT scores, researchers use OLS models. For TT, researchers use number of sessions of the program received by a student, which stands as a measure of the intensity of the program, and which researchers instrument using the intention-to-treat dummy as an instrumental variable (IV). To estimate the ITT and TT effects of the program on GPL test scores, researchers used ordered logit models. To estimate the TT effect, researchers used the number of sessions of the program received by the student to control for endogeneity. Researchers also present cost effectiveness analyses.
Experimental Design Details
Randomization Method
Computer
Randomization Unit
School
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
85 schools
Sample size: planned number of observations
6,136 students
Sample size (or number of clusters) by treatment arms
Treatment group: 45 (3171) schools (students). Control group: 40 (2965) remaining schools (students).
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Study has received IRB approval. Details not available.
IRB Approval Date
Details not available
IRB Approval Number
Details not available

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
December 31, 2010, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
December 31, 2010, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
85 schools
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
6,136 students
Final Sample Size (or Number of Clusters) by Treatment Arms
Treatment group: 45 (3171) schools (students). Control group: 40 (2965) remaining schools (students).
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Abstract
A randomized evaluation in Chile …finds that providing a 3-month program of small group tutoring to fourth graders using college student volunteers raise language scores and measures of attitudes towards reading for some subgroups of the population. In particular, students from low-performing and poor schools in areas in which the program was implemented increased their performance in a reading test by between 0:15 and 0:20 standard deviations and improved signi…cantly their self-perceptions as readers. Researchers also present cost-effectiveness analyses and …find that for the students for which the program was effective the program was in the range of the current estimates.
Citation
Cabezas, Veronica, Jose Cuesta, and Francisco Galleg. "Effects of Short-Term Tutoring on Cognitive and Non-Cognitive Skills: Evidence from a Randomized Evaluation in Chile." Working Paper, May 2011.

Reports & Other Materials