Experiment Registration for The Consumer Welfare Effects of Online Ads: Evidence from a 9-Year Experiment

Last registered on June 11, 2025
View Trial History

Pre-Trial

Trial Information

General Information

Title
Experiment Registration for The Consumer Welfare Effects of Online Ads: Evidence from a 9-Year Experiment
RCT ID
AEARCTR-0016160
Initial registration date
June 02, 2025

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
June 11, 2025, 6:28 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
Belgium
Region
Country
Canada
Region
Country
Germany
Region
Country
Spain
Region
Country
France
Region
Country
United Kingdom of Great Britain and Northern Ireland
Region
Country
Ireland
Region
Country
Japan
Region
Country
Korea (Democratic People's Republic of)
Region
Country
Mexico
Region
Country
Norway
Region
Country
Romania
Region
Country
United States of America
Region

Primary Investigator

Name
Haritz Garro
Affiliation
Meta
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Asad Liaqat
PI Affiliation
Meta Platforms
Contact Investigator
PI Name
Daley Kutzman
PI Affiliation
Meta Platforms
Contact Investigator
PI Name
Daniel Deisenroth
PI Affiliation
Meta Platforms
Contact Investigator
PI Name
Nils Wernerfelt
PI Affiliation
Northwestern University
Contact Investigator
PI Name
Avinash Collis
PI Affiliation
Carnegie Mellon University
Contact Investigator
PI Name
Erik Brynjolfsson
PI Affiliation
Stanford University
Contact Investigator

Additional Trial Information

Status
Completed
Start date
2013-07-01
End date
2025-06-01
Keywords
Other
Additional Keywords
JEL code(s)
Secondary IDs
Prior work
This trial is based on or builds upon one or more prior RCTs.
Abstract
Research on the effects of online advertising on consumer welfare is limited due to challenges in running large-scale field experiments. We analyze a long-running field experiment on Facebook in which a random subset of users received no ads in their newsfeeds. Using an incentive - compatible deactivation experiment, we find no significant differences in users’ valuation of Facebook across a representative sample of 53,083 Facebook users in the ads and no ads groups. Our sample size allows for precise estimates, suggesting that either the disutility of ads is relatively small or that there are offsetting benefits, such as product discovery.
External Link(s)

Registration Citation

Citation
Brynjolfsson, Erik et al. 2025. "Experiment Registration for The Consumer Welfare Effects of Online Ads: Evidence from a 9-Year Experiment ." AEA RCT Registry. June 11. https://doi.org/10.1257/rct.16160-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
We analyze a long-running field experiment on Facebook in which a random subset of users received no ads in their newsfeeds experiment to understand how users in the ads and no ads groups value Facebook. We measure users' valuation of Facebook across a representative sample of 53,083 Facebook users using an incentive-compatible deactivation experiment.
Intervention (Hidden)
Intervention Start Date
2022-03-25
Intervention End Date
2022-04-07

Primary Outcomes

Primary Outcomes (end points)
User valuation of Facebook
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We leveraged a long-running and pre-existing field experiment on Facebook in which a random subset of users received no ads in their newsfeeds experiment.
Experimental Design Details
Randomization Method
When a user joins Facebook, she is given a unique user ID. At the same time, a hash of that user ID will randomly determine whether that user is in the no ads or ads condition. Whatever the assignment is, the user will stay in that condition for the entire duration of her time on Facebook. This means that the users in our no ads condition have never seen ads in their newsfeed. Further, as the Facebook population grows, new users are continually added to both arms of the experiment.
Randomization Unit
Facebook user
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
1
Sample size: planned number of observations
53,083 Facebook users
Sample size (or number of clusters) by treatment arms
39,647 in the ads arm, and 13,436 in the no-ads arm
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
April 07, 2022, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
Final Sample Size: Number of Clusters (Unit of Randomization)
1
Was attrition correlated with treatment status?
Final Sample Size: Total Number of Observations
53,083 Facebook users
Final Sample Size (or Number of Clusters) by Treatment Arms
39,647 in the ads arm, and 13,436 in the no-ads arm
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
No
Reports, Papers & Other Materials

Relevant Paper(s)

Abstract
Research on the effects of online advertising on consumer welfare is limited due to challenges in running large-scale field experiments. We analyze a long-running field experiment on Facebook in which a random subset of users received no ads in their newsfeeds. Using an incentive - compatible deactivation experiment, we find no significant differences in users’ valuation of Facebook across a representative sample of 53,083 Facebook users in the ads and no ads groups. Our sample size allows for precise estimates, suggesting that either the disutility of ads is relatively small or that there are offsetting benefits, such as product discovery.
Citation
"The Consumer Welfare Effects of Online Ads: Evidence from a 9-Year Experiment" Erik Brynjolfsson, Avinash Collis, Daniel Deisenroth, Haritz Garro, Daley Kutzman, Asad Liaqat, Nils Wernerfelt - American Economic Review: Insights (Forthcoming)
URL
https://www.aeaweb.org/articles?id=10.1257/aeri.20240452&&from=f

Reports & Other Materials