x

NEW UPDATE: Completed trials may now upload and register supplementary documents (e.g. null results reports, populated pre-analysis plans, or post-trial results reports) in the Post Trial section under Reports, Papers, & Other Materials.
Measuring Consumer Sensitivity to Advertising: A Field Experiment on Pandora Internet Radio
Last registered on April 25, 2018

Pre-Trial

Trial Information
General Information
Title
Measuring Consumer Sensitivity to Advertising: A Field Experiment on Pandora Internet Radio
RCT ID
AEARCTR-0002932
Initial registration date
April 23, 2018
Last updated
April 25, 2018 3:01 PM EDT
Location(s)
Region
Primary Investigator
Affiliation
Pandora
Other Primary Investigator(s)
Additional Trial Information
Status
Completed
Start date
2014-06-01
End date
2016-06-01
Secondary IDs
Abstract
A randomized experiment with almost 35 million Pandora listeners enables us to measure the sensitivity of consumers to advertising, an important topic of study in the era of ad-supported digital content provision. The experiment randomized listeners into nine treatment groups, each of which received a different level of audio advertising interrupting their music listening, with the highest treatment group receiving more than twice as many ads as the lowest treatment group. By keeping consistent treatment assignment for 21 months, we are able to measure long-run demand effects and compare them to the short-run effects we might obtain with only a month-long experiment. We estimate
a demand curve: number of hours listened as a function of the number of ads per hour received. We also measure the impact on the number of days listened and on the probability of listening at all in the final month of the experiment. We separately vary the number of commercial interruptions per hour and the number of ads per commercial interruption, to measure the extent to which these influence consumer listening behavior. Lastly, we measure the impact on demand for a substitute good: the number of paid subscriptions to the ad-free version of Pandora.
External Link(s)
Registration Citation
Citation
Reiley, David. 2018. "Measuring Consumer Sensitivity to Advertising: A Field Experiment on Pandora Internet Radio." AEA RCT Registry. April 25. https://doi.org/10.1257/rct.2932-1.0.
Former Citation
Reiley, David. 2018. "Measuring Consumer Sensitivity to Advertising: A Field Experiment on Pandora Internet Radio." AEA RCT Registry. April 25. http://www.socialscienceregistry.org/trials/2932/history/28793.
Experimental Details
Interventions
Intervention(s)
We vary the audio ad capacity for each individual listener.
Intervention Start Date
2014-06-01
Intervention End Date
2016-04-07
Primary Outcomes
Primary Outcomes (end points)
Hours listened in the final month of the experiment.
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Days listened in the final month of the experiment. Probability of listening at all in the final month of the experiment. Probability of subscribing to ad-free Pandora One in the final month of the experiment.
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
3x3 experimental design. We separately vary the number of commercial interruptions per hour (3, 4, or 6) and the number of ads per commercial interruption (1, 1.5, or 2, where 1.5 means alternating between 1 and 2). We also have a control group 10x as large as each treatment group, and this control group matches the values at the center of the matrix (4x1.5), matching the previous default values listeners received at Pandora before the experiment.
Experimental Design Details
Randomization Method
Randomization based on a hash of the listener ID number at Pandora.
Randomization Unit
Individual listener.
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
35 million listeners.
Sample size: planned number of observations
35 million listeners.
Sample size (or number of clusters) by treatment arms
10% of listeners assigned to control (default previous ad load of 1.5x4 per hour), 1% to each of the nine treatments.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
IRB Approval Date
IRB Approval Number
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
No
Is data collection complete?
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports, Papers & Other Materials
Relevant Paper(s)
REPORTS & OTHER MATERIALS