Peer effects and tracking, the role of boosting group cohesion.

Last registered on August 25, 2022

Pre-Trial

Trial Information

General Information

Title
Peer effects and tracking, the role of boosting group cohesion.
RCT ID
AEARCTR-0009947
Initial registration date
August 18, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
August 25, 2022, 2:42 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation

Other Primary Investigator(s)

Additional Trial Information

Status
On going
Start date
2022-08-15
End date
2023-07-15
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
We propose an RCT to study first whether a socio-emotional learning intervention can boost social cohesion in a group setting. In a second stage we seek to understand whether this increase in social cohesion can boost peer effects where students in the group learn more from peers with more knowledge. We will carry out this study in the setting of an online tutoring program in Mexico. Tutors meet with groups of 5 students twice a week for several months to revise math topics. Students are all of the same grade but from different schools and don't know each other in advance.

We will first randomly assign students to groups that will either be tracked or not tracked by initial performance. From the literature of tracking, we know that tracked groups tend to outperform non-tracked groups because tracking makes it easier for the teacher to teach at the right level at the expense of peer effects that can occur in diverse group settings.

We will then randomly assign these groups to a socio-emotional learning intervention (SEL) or a control condition. In the SEL intervention, tutors will use 10 hours of class to revise SEL activities instead of mathematics. To keep the time spent with the tutor constant, tutors in the control condition will use 10 hours of class to revise Mexican history topics. From previous pilots, we expect that the SEL condition will boost group cohesion as well as the attachment of students to the tutor.

Among educators it is well discussed that building rapport between teachers and students is a key ingredient of learning but and this study can document the importance of rapport quantitatively. We expect groups under the SEL intervention to perform better. We also expect groups in the tracking condition to perform better than in the non-tracked condition and we are interested in the interaction between the SEL treatment and the non-tracked groups as we think that peer effects might be boosted in groups with a strong social cohesion.

We are also interested to document the a priori preferences of children to be in a tracked or non-tracked group. We will ask students before assignments whether they would rather be in a tracked or non-tracked group then we will look for heterogeneous treatment effects between both groups. Students may have private information of whether they will reap the benefits of tracking and we would be able to measure these benefits.

We are also interested in studying heterogeneous treatment effects by initial performance as peer effects will go in different directions whenever the student outperforms or underperforms their peers.

External Link(s)

Registration Citation

Citation
Aguilar Llanes, Salome and Bernardo Garcias Bulle Bueno. 2022. "Peer effects and tracking, the role of boosting group cohesion. ." AEA RCT Registry. August 25. https://doi.org/10.1257/rct.9947-1.0
Experimental Details

Interventions

Intervention(s)
We will work with students in grades primary 3 to secondary 3rd. We have partnered with the states of Aguascalientes, Baja California, Sinaloa and Yucatán. The states will help us distribute the invitation to the program among the students enrolled in the state. Randomization is done at the student level with little to no concerns for spillovers because students are meeting on average less than 3 hours a week online with each other to hear the teacher’s lecture. We will do a stratified randomization according to the grade of the students as well as the baseline score. Also, since most of our tutors take more than one group we will make sure tutors after being assigned to the Affective or Non-Affective treatment are assigned a tracked and a no-tracked group.
The intervention would start in September 2022 and last a whole year until the summer of 2023.
Given that in the spring of 2022 we are working with 7000 students and 1400 tutors we expect to be working with a sample of at least 10 000 students, each interaction would have 2500 students.
In the spring semester of march 2021 we piloted the SEL treatment. We had a pure control group, a group that received only math classes and a group that received math and SEL classes. We observed and increase of 0.14 standard deviations in math and an increase in the student reported attachment to the tutor in the SEL intervention (see appendix). If we assume a standard error of 0.073 (observed in the pilot) we would have a minimum detectible effect (MDE) of 0.029 standard deviations. This MDE would allow us to identify any relevant effect of the any of the interactions across treatments. We are not planning on having a pure control group but in case the suscription of children is higher than the suscription of tutors we will add a pure control group.
JANN works as follows: we recruit tutors, volunteers of universities and normal schools seeking to meet their social service requirements of 480 hours to graduate. We assign the tutors into WhatsApp groups of 5 students that will meet 2 times a week. Tutors receive a 4 hour initial training on how to teach math, a psychopedagogic training, a training on how to handle difficult situations in case they find out of cases of abuse in the family of children. After the initial training sessions, all tutors have an optional series of training in math content. There will be two different interventions and the corresponding interactions among these interventions:

Track vs no-track: We will randomize students into being tracked or not. Then for the tracked students, we’ll rank them according to their performance in the baseline evaluation to create similar performing groups. Among non-tracked groups we will randomly assign them into groups. Given that we are pooling students across schools the heterogeneity in non-tracked groups can be considerable in some cases, whereas the tracked groups will be very homogenous.
Affective vs non affective. We hypothesize that peer effects can be boosted if we have a treatment that enhances the social cohesion of the groups as well as the rapport between the tutor and the students. The idea is that tutors in the affective treatment will use a total of 10 of their classes to cover socio emotional learning material. Tutors in the non-affective treatment will cover Mexican history material that is aimed at making sure that tutors and students in both treatment arms spends the same amount of time together as well as receive the same percentage of math classes. All tutors will be given a set of 10 activities in SEL and History respectively to cover during 10 of their classes that will be spread throughout their time with the students.
We would estimate the following equation: y_i=α+β_1 SEL_i+β_2 T_i+β_3 SEL_i*T_i+δX_i+ω_it
With y_i the score in math for the endline evaluation , SEL_i a dummy indicating the group received socio emotional learning classes, T_i a dummy indicating that the student was assigned to a tracked group and the interaction of both SEL_i*T_i and X_i a vector of baseline controls that can potentially be excluded. We hypothesize that β_1>0 and β_2>0 but potentially β_3<0 as the SEL_i treatment would boost peer effects that are weaker in tracked groups. We can also estimate the heterogenous effect of these treatments for students in different quartiles of the distribution at baseline similar to Duflo, Dupas, & Kremer 2011.
Intervention (Hidden)
Intervention Start Date
2022-08-15
Intervention End Date
2023-07-15

Primary Outcomes

Primary Outcomes (end points)
Our main outcome is the academic learning of students measured by the math and spanish score on an evaluation designed by us and it´s an adaptive exam. Students will answer a version of this evaluation at baseline and at endline. The second main outcome is the student-tutor attachment measured using the student version of the teacher–student relationship inventory (S-TSRI) (Ang, Ong, & Li 2020), in the pilot we observed that the affective intervention had a strong effect on this measure and the third main outcome will be the social cohesion of the group measured using the group environment questionnaire (Eys, Carron, Bray, & Brawley, 2007) commonly used in sports literature. Finally, we will survey the tutors on each of their groups using the strengths and difficulties questionnaire (Muris, Meesters, & van den Berg 2003) to measure whether they perceived any differential teaching difficulties when faced with groups that were tracked or not and between tutors that received the affective treatment or not.
Another important outcome will be the features we will extract from the recordings of the audio of tutoring sessions. We want to do some exploratory work using Natural Language Processing techniques to study the interactions between the students and tutors and see how these interactions relate to the success of the tutoring sessions. For instance, variables that we are considering are the tone in which the tutor speaks (encouraging, scolding, etc.), the frequency of each student’s interventions among others. Currently, to study the effectiveness of teaching we rely on the annotations done by observers (Kane, McCaffrey, Miller & Staiger 2013). The problem is that both measurements are subjective and potentially biased. We want to automatize the task of observing and put a “neutral” or at least “constant” algorithm to analyze what is happening in the tutoring session. Preliminary analysis of the recordings in our first implementation shows that a Latent Dirichlet Allocation algorithm is able to identify topics discussed in the classroom that are related to math as well as topics rather related to personal matters.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We first randomize students into being assigned to a tracked group or a non-tracked group. Then we cross randomize the groups to the Socio-emotional learning condition (SEL) or a control condition.
Experimental Design Details
Randomization Method
By a computer program
Randomization Unit
We do randomization at the student level between tracked and non-tracked groups. And then at the group level between the SEL and the control condition.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
At least 2000 groups
Sample size: planned number of observations
at least 10000 students
Sample size (or number of clusters) by treatment arms
We have 4 treatment arms 2500, about students per treatment arm.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Committee on the Use of Humans as Experimental Subjects (COUHES) at MIT
IRB Approval Date
2021-08-03
IRB Approval Number
2101000304A005

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials