Peer effects and tracking, the role of boosting group cohesion.

We will work with students in grades primary 3 to secondary 3rd. We have partnered with the states of Aguascalientes, Baja California, Sinaloa and Yucatán. The states will help us distribute the invitation to the program among the students enrolled in the state. Randomization is done at the student level with little to no concerns for spillovers because students are meeting on average less than 3 hours a week online with each other to hear the teacher’s lecture. We will do a stratified randomization according to the grade of the students as well as the baseline score. Also, since most of our tutors take more than one group we will make sure tutors after being assigned to the Affective or Non-Affective treatment are assigned a tracked and a no-tracked group.
The intervention would start in September 2022 and last a whole year until the summer of 2023.
Given that in the spring of 2022 we are working with 7000 students and 1400 tutors we expect to be working with a sample of at least 10 000 students, each interaction would have 2500 students.
In the spring semester of march 2021 we piloted the SEL treatment. We had a pure control group, a group that received only math classes and a group that received math and SEL classes. We observed and increase of 0.14 standard deviations in math and an increase in the student reported attachment to the tutor in the SEL intervention (see appendix). If we assume a standard error of 0.073 (observed in the pilot) we would have a minimum detectible effect (MDE) of 0.029 standard deviations. This MDE would allow us to identify any relevant effect of the any of the interactions across treatments. We are not planning on having a pure control group but in case the suscription of children is higher than the suscription of tutors we will add a pure control group.
JANN works as follows: we recruit tutors, volunteers of universities and normal schools seeking to meet their social service requirements of 480 hours to graduate. We assign the tutors into WhatsApp groups of 5 students that will meet 2 times a week. Tutors receive a 4 hour initial training on how to teach math, a psychopedagogic training, a training on how to handle difficult situations in case they find out of cases of abuse in the family of children. After the initial training sessions, all tutors have an optional series of training in math content. There will be two different interventions and the corresponding interactions among these interventions:

Track vs no-track: We will randomize students into being tracked or not. Then for the tracked students, we’ll rank them according to their performance in the baseline evaluation to create similar performing groups. Among non-tracked groups we will randomly assign them into groups. Given that we are pooling students across schools the heterogeneity in non-tracked groups can be considerable in some cases, whereas the tracked groups will be very homogenous.
Affective vs non affective. We hypothesize that peer effects can be boosted if we have a treatment that enhances the social cohesion of the groups as well as the rapport between the tutor and the students. The idea is that tutors in the affective treatment will use a total of 10 of their classes to cover socio emotional learning material. Tutors in the non-affective treatment will cover Mexican history material that is aimed at making sure that tutors and students in both treatment arms spends the same amount of time together as well as receive the same percentage of math classes. All tutors will be given a set of 10 activities in SEL and History respectively to cover during 10 of their classes that will be spread throughout their time with the students.
We would estimate the following equation: y_i=α+β_1 SEL_i+β_2 T_i+β_3 SEL_i*T_i+δX_i+ω_it
With y_i the score in math for the endline evaluation , SEL_i a dummy indicating the group received socio emotional learning classes, T_i a dummy indicating that the student was assigned to a tracked group and the interaction of both SEL_i*T_i and X_i a vector of baseline controls that can potentially be excluded. We hypothesize that β_1>0 and β_2>0 but potentially β_3<0 as the SEL_i treatment would boost peer effects that are weaker in tracked groups. We can also estimate the heterogenous effect of these treatments for students in different quartiles of the distribution at baseline similar to Duflo, Dupas, & Kremer 2011.

2022-08-15

2023-07-15

Our main outcome is the academic learning of students measured by the math and spanish score on an evaluation designed by us and it´s an adaptive exam. Students will answer a version of this evaluation at baseline and at endline. The second main outcome is the student-tutor attachment measured using the student version of the teacher–student relationship inventory (S-TSRI) (Ang, Ong, & Li 2020), in the pilot we observed that the affective intervention had a strong effect on this measure and the third main outcome will be the social cohesion of the group measured using the group environment questionnaire (Eys, Carron, Bray, & Brawley, 2007) commonly used in sports literature. Finally, we will survey the tutors on each of their groups using the strengths and difficulties questionnaire (Muris, Meesters, & van den Berg 2003) to measure whether they perceived any differential teaching difficulties when faced with groups that were tracked or not and between tutors that received the affective treatment or not.
Another important outcome will be the features we will extract from the recordings of the audio of tutoring sessions. We want to do some exploratory work using Natural Language Processing techniques to study the interactions between the students and tutors and see how these interactions relate to the success of the tutoring sessions. For instance, variables that we are considering are the tone in which the tutor speaks (encouraging, scolding, etc.), the frequency of each student’s interventions among others. Currently, to study the effectiveness of teaching we rely on the annotations done by observers (Kane, McCaffrey, Miller & Staiger 2013). The problem is that both measurements are subjective and potentially biased. We want to automatize the task of observing and put a “neutral” or at least “constant” algorithm to analyze what is happening in the tutoring session. Preliminary analysis of the recordings in our first implementation shows that a Latent Dirichlet Allocation algorithm is able to identify topics discussed in the classroom that are related to math as well as topics rather related to personal matters.

We first randomize students into being assigned to a tracked group or a non-tracked group. Then we cross randomize the groups to the Socio-emotional learning condition (SEL) or a control condition.

By a computer program

We do randomization at the student level between tracked and non-tracked groups. And then at the group level between the SEL and the control condition.

Yes

At least 2000 groups

at least 10000 students

We have 4 treatment arms 2500, about students per treatment arm.