Race Discrimination in Internet Advertising: Evidence From a Field Experiment

Last registered on March 31, 2022
View Trial History

Pre-Trial

Trial Information

General Information

Title
Race Discrimination in Internet Advertising: Evidence From a Field Experiment
RCT ID
AEARCTR-0009168
Initial registration date
March 30, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
March 31, 2022, 3:26 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
United States of America
Region

Primary Investigator

Name
Neil Sehgal
Affiliation
Harvard University
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Dan Svirsky
PI Affiliation
Uber Technologies, Inc.
Contact Investigator

Additional Trial Information

Status
Completed
Start date
2020-08-18
End date
2021-09-04
Keywords
Behavior, Other
Additional Keywords
Social Networks, Discrimination
JEL code(s)
J71, J15
Secondary IDs
Prior work
This trial is based on or builds upon one or more prior RCTs.
Abstract
We present the results of an experiment that documents racial bias on Facebook’s Advertising Platform. We find that darker skin complexions are penalized, leading to real economic consequences. For every $1,000 an advertiser spends on ads with models with light skin complexions, that advertiser would have to spend $1,159 to achieve the same level of engagement using a photo of a darker skin complexion model. Facebook’s budget optimization tool reinforces these viewer biases. When pictures of models with light and dark complexions are allocated a shared budget, Facebook funnels roughly 64% of the budget towards photos featuring lighter skin complexions. Our findings show how neutral algorithmic decisions can complement user bias, and also demonstrate that racial disparities persist in settings where statistical discrimination is a less natural explanation than simple animus.
External Link(s)
original OSF preregistration documents

Registration Citation

Citation
Sehgal, Neil and Dan Svirsky. 2022. "Race Discrimination in Internet Advertising: Evidence From a Field Experiment." AEA RCT Registry. March 31. https://doi.org/10.1257/rct.9168-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
We will conduct a 2x2 experiment to test the role of racial bias in whether Instagram users like advertisements for wedding pictures. We test whether images of people with dark complexions receive less favorable engagement, relative to a baseline, when the advertisement zooms in. Consider two photographs that are highly similar but differ in the skin complexion of the people being pictured. The hypothesis is: if an Instagram ad zooms in on both pictures, in a way that makes the skin of the models take up more of the image, does this lead to racial disparities? Specifically, let P_L be equal to the Number of Likes for a photograph of a model with a lighter complexion divided by the Total Number of People Who Viewed that photograph, P_D be the same for a photograph of a model with a dark complexion, P_LZ be the same for the zoomed-in photograph of the model with a light complexion, and P_DZ be the same for a zoomed-in photograph of the model with a dark complexion. Is P_LZ – P_L > P_DZ – P_D?
Intervention Start Date
2020-08-18
Intervention End Date
2021-09-04

Primary Outcomes

Primary Outcomes (end points)
The key dependent variables – P_LZ, P_L, P_DZ, and P_D – will be measured as the number of likes left for each photograph divided by the number of “reaches”, which Instagram defines as the number of people who saw the photograph in their Instagram feed.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
For our measure of potential racial disparity ( [P_LZ – P_L] – [P_DZ – P_D] ), we will have geographic breakdowns at the state level.
We will calculate the geographic correlation at the state level of this measure of racial disparity with two other measures of racial disparities: the IAT self-survey of racial attitudes, and the measure of racial resentment in the ANES survey, and black maternal mortality per 1,000,000 minus white maternal mortality.
Our hypothesis is that our measure of racial disparity will be positively correlated with these measures.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
For the likes experiment, users are assigned to one of four conditions:
1) A zoomed-out wedding photograph with people with light complexions
2) A zoomed-out wedding photograph with people with dark complexions
3) A zoomed-in wedding photograph with people with light complexions
4) A zoomed-in wedding photograph with people with dark complexions
There are three approaches to finding pairs of photographs that are similar to each other but have models of different skin complexion:
1) We could find pictures that look highly similar – same pose, same theme – but where the models photographed have different skin complexions
2) We could take a picture of someone with a light complexion and darken it using photo editing software
3) We could take a picture of someone with a dark complexion and lighten it using photo editing software
We use all three approaches, and we use each approach two times.
Experimental Design Details
Randomization Method
The Facebook advertisement algorithm randomly selects users to view our advertisements.
Randomization Unit
Individuals are randomized to advertisements. Facebook's advertisement platform randomly selects users to view our ads based on targeting parameters.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
We will collect roughly 75,000 observations.
Sample size: planned number of observations
We will collect roughly 75,000 observations.
Sample size (or number of clusters) by treatment arms
There are 6 sets of ads, each made of 2 zoomed out images and 2 zoomed in images. We aim for about 3,125 observations for each image, or 18,750 observations for each unique treatment (picture complexion*cropped).
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials

Documents

Document Name
OSF Preregistration
Document Type
other
Document Description
Our original preregistration was done via OSF
File
OSF Preregistration

MD5: 111c2702e57f375b54865f04166fb186

SHA1: 37103a1c43f1be26df4d13ce85fbc9cb6dff7142

Uploaded At: March 30, 2022

IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials