|
Field
Trial End Date
|
Before
August 31, 2024
|
After
August 31, 2025
|
|
Field
Last Published
|
Before
January 23, 2023 06:02 AM
|
After
February 26, 2024 12:45 PM
|
|
Field
Intervention End Date
|
Before
December 23, 2022
|
After
December 23, 2023
|
|
Field
Planned Number of Clusters
|
Before
85 schools
|
After
85 schools ----- DELETE
230 schools in total (85 schools in the first year, 150 additional schools in the second year)
|
|
Field
Planned Number of Observations
|
Before
Around 4,000 pupils, 600 teachers.
|
After
Around 4,000 pupils, 600 teachers. ----- DELETE
Around 1400 teachers and 10,000 students in total (Around 4,000 pupils and 600 teachers in the first year of implementation, 780 teachers and 6,000 students in the second year).
|
|
Field
Sample size (or number of clusters) by treatment arms
|
Before
Around 2,000 students and 300 teachers in each of the treatment arms
|
After
Around 2,000 students and 300 teachers in each of the treatment arms ----- DELETE
In the first year of implementation, around 300 teachers in control group and 300 in treatment group.
In the second year of implementation, around 180 additional teachers in control, 300 in first treatment group, 300 in second treatment group.
|
|
Field
Power calculation: Minimum Detectable Effect Size for Main Outcomes
|
Before
Considering ICC of 0.07, 85 schools, 40 students per school, the MDE is 0.185 SD or 0.085 in terms of percentage points change assuming 20% of students are recommended toward a top-tier track. In case of promising results, we plan to expand the scale of the project during the school year 2024-25.
|
After
----- DELETE ------ Considering ICC of 0.07, 85 schools, 40 students per school, the MDE is 0.185 SD or 0.085 in terms of percentage points change assuming 20% of students are recommended toward a top-tier track. In case of promising results, we plan to expand the scale of the project during the school year 2024-25. ----- DELETE
Considering ICC of 0.07, 230 schools, 40 students per school, the MDE is 0.113 SD or 0.05 in terms of percentage points change assuming 20% of students are recommended toward a top-tier track.
|
|
Field
Intervention (Hidden)
|
Before
We implement the following interventions:
(1) Experiment in teacher survey. We show a series of vignettes with hypothetical students' profiles to teachers and ask them to provide a track recommendation for these hypothetical students. We randomize students' gender and SES status across vignettes to study if they affect teachers' recommendations. We implement two teachers' online surveys: the baseline survey was sent by email to the official school email address in November to recruit the teachers. The second survey will be sent by email to the recruited teachers, through the email address they provided in the first questionnaire.
(2) Information intervention at the school level. Between November 27 and December 17, teachers in treatment and control schools received a personalized report with aggregate statistics about the performance in high school of their past students, along with the province and national average. All teachers received information about their past students' average grades in math, Italian, and English in high school, the average share of students admitted in grade 10, and the average number of subjects in which they failed in grade 9. They also received information about these outcomes separately for students who followed and did not follow their track recommendation, and for students who chose different high school tracks. In addition to this information, teachers in treatment schools received information about inequality in their past track recommendations. In particular, they were shown the percentage of students that they recommended to each track by students' SES; the same statistics were shown separately for students with high and low GPA. Moreover, they were shown their believed and actual share of students from low SES assigned to the high tracks, and the actual performance in high school of students with high GPA in middle school but different SES. The individualized reports were sent to teachers via email. We will evaluate the impact of this intervention on teachers' track recommendations and students' choices. We chose to randomize at the school level rather than at the teacher level in order to avoid contamination between teachers who received the additional information on inequality in recommendations by SES.
|
After
We implement the following interventions:
(1) Experiment in teacher survey. We show a series of vignettes with hypothetical students' profiles to teachers and ask them to provide a track recommendation for these hypothetical students. We randomize students' gender and SES status across vignettes to study if they affect teachers' recommendations. We implement two teachers' online surveys: the baseline survey was sent by email to the official school email address in November to recruit the teachers. The second survey will be sent by email to the recruited teachers, through the email address they provided in the first questionnaire.
(2) Information intervention at the school level. Between November 27 and December 17 2023, teachers in treatment and control schools received a personalized report with aggregate statistics about the performance in high school of their past students, along with the province and national average. All teachers received information about their past students' average grades in math, Italian, and English in high school, the average share of students admitted in grade 10, and the average number of subjects in which they failed in grade 9. They also received information about these outcomes separately for students who followed and did not follow their track recommendation, and for students who chose different high school tracks. In addition to this information, teachers in treatment schools received information about inequality in their past track recommendations. In particular, they were shown the percentage of students that they recommended to each track by students' SES; the same statistics were shown separately for students with high and low GPA. Moreover, they were shown their believed and actual share of students from low SES assigned to the high tracks, and the actual performance in high school of students with high GPA in middle school but different SES. The individualized reports were sent to teachers via email. We will evaluate the impact of this intervention on teachers' track recommendations and students' choices. We chose to randomize at the school level rather than at the teacher level in order to avoid contamination between teachers who received the additional information on inequality in recommendations by SES.
In the academic year 2023-24, we have expanded the intervention to include additional schools. The intervention was implemented in November and December 2023. The information treatment remains similar to the previous year, with the following modifications. In addition to the control group, there will be two treatment groups. In the first treatment, teachers receive information on the gaps in track recommendations for their past students, along with information on teachers' beliefs regarding whether students should receive different recommendations based on their socio-economic status (SES). In the second treatment group, teachers receive additional information aimed at correcting their misconceptions about the probability of success for low SES students in high school. We present teachers with reported beliefs on the probability of failure in grade 9 for low SES students (on average and conditional on a high GPA in middle school), comparing this with the truth based on administrative data for the same variables.
|