Evidence of tutoring, mentoring and support programs for the improvement of reading and mathematics competences.

Last registered on April 25, 2023

View Trial History

Pre-Trial

Trial Information

General Information

Title

Evidence of tutoring, mentoring and support programs for the improvement of reading and mathematics competences.

RCT ID

AEARCTR-0010448

Initial registration date

December 16, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

January 03, 2023, 4:34 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated

April 25, 2023, 12:17 PM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Country

Spain

Region

Catalonia

Primary Investigator

Name

Caterina Calsamiglia

Affiliation

Institute of Political Economy and Governance

Contact Primary Investigator

Other Primary Investigator(s)

PI Name

Giacomo De Giorgi

PI Affiliation

IEE/GSEM, University of Geneva

Contact Investigator

PI Name

Davide Viviano

PI Affiliation

Stanford GSB and Harvard University

Contact Investigator

PI Name

Jenifer Ruiz-Valenzuela

PI Affiliation

Universitat de Barcelona

Contact Investigator

PI Name

Javier Garcia-Brazales

PI Affiliation

Bocconi (LEAP) and University of Exeter Business School

Contact Investigator

Additional Trial Information

Status

In development

Start date

2022-11-23

End date

2024-03-31

Keywords

Behavior, Education

Additional Keywords

Mentoring and Tutoring Academic Programs

JEL code(s)

I24, J18

Secondary IDs

Prior work

This trial does not extend or rely on any prior RCTs.

Abstract

Reading and math skills have a direct impact on people's lives. Lacking these skills constitutes an important barrier against effective communication, and therefore against the development of the individual within society. An early deficiency in reading and mathematics levels is correlated with lower academic performance, a higher level of dropout, and worse labor market outcomes.

Resolving these deficiencies at early ages has two main potential benefits. First, it unravels the development of the person, making it easier for him/her to progress in their learning (dynamic gains and complementarities of future investments). Second, it improves the ability to work in the classroom on the part of the whole group (spillovers). The sooner such gaps and inequities are identified and addressed, the better individual and classroom educational performance will be.

The available evidence, generated from intervention projects developed in countries other than Spain (our context of interest), points to tutoring and mentoring programs as one of the most effective means to increase the academic performance of students, especially those who start at a greater disadvantage. In addition, the existing evidence points to this type of intervention as triggering other positive effects on learning (personal and transversal competences, school adherence, reduction in absenteeism, etc.).

The objective of this study is to provide evidence about the effectiveness of two interventions providing educational support, one for reading and one for mathematics, in low-socioeconomic schools located in Spain. The reading program consist of one-on-one mentoring sessions for grade 4 students aimed at fostering the pleasure for autonomous reading, The mathematics program consists of four-group tutorial sessions for grade 6 students. Within the classes of the grade for which the school is treated, only a subset of students will actually participate in our tutoring/mentoring programs.

Assigning half of the pool of participating schools into the reading program and the other half into the mathematics program, we will compare the outcomes of the students from the grade for which the school has been treated with the outcomes of the students in the same grade from schools that got assigned to the other program. Within a given treated class, students who did not actually participate in the mentoring/tutoring programs might still have their outcomes impacted through in-class spillovers.

Naturally, we will explore the impact of our intervention on reading and mathematics cognitive abilities. We will pay particular attention to how these programs affect the classroom depending on the position in the network of friends where the treated kids belong. We also explore other dimensions such as improvements in socio-emotional skills, classroom climate and behavior, and changes in social relations and friendships.

External Link(s)

Registration Citation

Citation

Calsamiglia, Caterina et al. 2023. "Evidence of tutoring, mentoring and support programs for the improvement of reading and mathematics competences.." AEA RCT Registry. April 25. https://doi.org/10.1257/rct.10448-2.0

Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Experimental Details

Interventions

Intervention(s)

Intervention Start Date

2023-01-09

Intervention End Date

2023-06-30

Primary Outcomes

Primary Outcomes (end points)

Academic scores (mathematics and reading); non-cognitive measures (academic interest and aspirations, locus-of-control, self-confidence), class environment and friendship networks.

Primary Outcomes (explanation)

Academic scores will be obtained from two tests, MONK and TEA, whose scoring systems are already pre-defined by the test developers.

Non-cognitive outcomes will be constructed from multiple Likert-scale survey items within the various non-cognitive dimensions outlined above. Aggregation methods within each dimension will include dichotomization of the responses, averaging, and standardarizations. Multiple correspondence analysis will also be employed.

Measures of friendship networks will be constructed regarding both the number of reported classmates considered good friends and the "quality" of those networks (friends perceived to have given characteristics).

Secondary Outcomes

Secondary Outcomes (end points)

Secondary Outcomes (explanation)

Experimental Design

We will evaluate the impact of two programs targeted at primary education students in disadvantaged areas that take place outside school hours. One is a reading mentoring program for Grade 4 students consisting of one-on-one sessions with a volunteer. The other one is a mathematics tutorial program delivered in groups of four among Grade 6 students.

We have committed to implement one of the two programs in each and every of the schools that have agreed to participate in our study. For this, we randomly assign half of the schools to the reading treatment and the other half to the mathematics treatment. Within each treated class, only a subset of students will participate in the program due to budgetary reasons. We will identify the effects of a given program by comparing the outcomes of the students in the relevant grade in schools that were assigned to treatment for that program (some of these students received the treatment directly and others through classroom spillovers) and the outcomes of the students in that grade from students of schools that got assigned to the other treatment. These control students did not receive the treatment in any form (we assume that cross-grade spillovers are non-existent).

Experimental Design Details

Randomization Method

The randomization algorithm was written by the PIs and then executed (including the selection of the seed for random number generation) under the supervision of our implementation partner, Fundació Bofill.

Randomization Unit

Randomization takes place at two levels. In the first one, the pool of participating schools gets split into two: those that will receive the reading treatment and those that will receive the mathematics treatment.

In the second level, a subset of the students in each class gets assigned to participate in the program. For the selection of these students, their headteacher provided a ranking based on how he/she perceives the needs of each student for participating in our program to be. We follow this ranking until completing the pre-determined number of participants per class/program and only intervene by randomizing which student to receive treatment in the case of oversubscription within a class of students with the same level of perceived needs.

Was the treatment clustered?

Yes

Experiment Characteristics

Sample size: planned number of clusters

A total of 134 schools initially agreed verbally to participate in our study. A subset of them might nevertheless decide not to proceed with the signing of a formal agreement of participation and will then be excluded from our investigation. The allocation of schools to treatment arms was performed before knowing which schools will decide to discontinue their participation.

Sample size: planned number of observations

Within each school, all students in grades 4 and 6 belong to our target population (as explained, only a subset of them actually receives our treatment). If all the 134 schools participated and all the students in those grades did so as well, approximately 10,000 students would be expected to participate in our study. This number is, however, prone to be significantly lower due to multiple reasons, including: 1) student absenteeism on the days of information collection; 2) students' inability to speak the language; 3) lack of parental consent; 4) schools eventually deciding not to participate as a whole.

Sample size (or number of clusters) by treatment arms

50% of our schools will be assigned to the reading treatment and 50% to the mathematics one. This amounts to 67 schools in each treatment.

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

Supporting Documents and Materials

IRB