An Algorithmic Approach to Reducing Gender Discrimination: Evidence from a Large-Scale Experiment on Tracking Decisions

Last registered on February 19, 2025

Pre-Trial

Trial Information

General Information

Title
An Algorithmic Approach to Reducing Gender Discrimination: Evidence from a Large-Scale Experiment on Tracking Decisions
RCT ID
AEARCTR-0015384
Initial registration date
February 13, 2025

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
February 19, 2025, 9:29 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Affiliation

Other Primary Investigator(s)

PI Affiliation
Harvard Kennedy School
PI Affiliation
Harvard Kennedy School
PI Affiliation
Cornell University

Additional Trial Information

Status
On going
Start date
2024-03-01
End date
2027-02-01
Secondary IDs
This trial does not extend or rely on any prior RCTs.
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
We study how to equalize students’ educational opportunities in a tracked high school system. We will use a randomized control trial in around 400 Italian middle schools to test two conceptually different ways to close gender gaps in high school track recommendations. To help teachers predict future performance, our first treatment provides teachers with a simple algorithmic recommendation that indicates whether students are likely to excel in the more rigorous scientific high school tracks. To address behavioral biases and awareness issues, our second treatment provides teachers with real-time feedback on the diversity of their track recommendations compared to an algorithmic benchmark, which they can then adjust before communicating their recommendations to students. We hypothesize that the first intervention will lead to fairer track recommendations if teacher biases are driven by inaccurate mental models of how individual students will perform in high school. Meanwhile, the second intervention will lead to fairer recommendations if teachers are best able to correct behavioral biases when considering the aggregate diversity of their track recommendations.
External Link(s)

Registration Citation

Citation
Carlana, Michela et al. 2025. "An Algorithmic Approach to Reducing Gender Discrimination: Evidence from a Large-Scale Experiment on Tracking Decisions." AEA RCT Registry. February 19. https://doi.org/10.1257/rct.15384-1.0
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2024-11-04
Intervention End Date
2025-12-19

Primary Outcomes

Primary Outcomes (end points)
The primary outcome will include:
(1) Administrative outcomes on the gender gap in the teachers' scientific track recommendations, for baseline high-achievers.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
The secondary outcomes will include:
(1) Administrative outcomes on the gender gap in the students' track enrollment decisions, for baseline high-achievers.
(2) Administrative outcomes on the gender gaps in the teachers' scientific track recommendations and students' track enrollment decisions, for the general population of eighth graders.
(3) Administrative outcomes on the proportion of baseline high-achievers recommended to and enrolled in the scientific track, compared to baseline low-achievers, both in aggregate and by gender.
(4) Administrative outcomes on students' academic performance in high school, conditional on scientific track enrollment, both in aggregate and by gender.
(5) Data collected from a teacher questionnaire on teachers' beliefs about the gender gaps in their own track recommendations and their students' track enrollment decisions.
(6) Data collected from a teacher questionnaire on scientific track recommendations for hypothetical students' profiles with randomized genders.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Information intervention with teachers
Experimental Design Details
Not available
Randomization Method
Randomization was done in an office by a computer.
Randomization Unit
The unit of randomization is the school system. Because some teachers reported teaching classes in multiple participating middle schools, we linked some schools to form school systems before randomizing, to avoid exposure to multiple treatments. The school systems were then randomized into a control group or one of three treatment groups, with 25% probability each.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
Around 400 school systems
Sample size: planned number of observations
Around 3,000 eligible eighth-grade teachers and 40,000 eighth-graders in participating school systems.
Sample size (or number of clusters) by treatment arms
The school systems will be randomized into the control group or one of three treatment groups, with 25% probability each.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Considering an ICC of 0.07, with approximately 400 school systems (100 school systems per treatment arm) and around 100 students per school system (approximately 30 baseline high-achievers and 70 baseline low-achievers), the MDE for the high-achieving students is 0.127 SD for each treatment arm without including covariates, or 0.06 percentage points change, assuming that 42% of high-achieving students are recommended for a scientific track.
IRB

Institutional Review Boards (IRBs)

IRB Name
Harvard
IRB Approval Date
2024-10-01
IRB Approval Number
IRB24-1047