Measuring Consumer Sensitivity to Advertising: A Field Experiment on Pandora Internet Radio

Last registered on April 25, 2018

Pre-Trial

Trial Information

General Information

Title
Measuring Consumer Sensitivity to Advertising: A Field Experiment on Pandora Internet Radio
RCT ID
AEARCTR-0002932
Initial registration date
April 23, 2018

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
April 25, 2018, 3:01 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
Pandora

Other Primary Investigator(s)

Additional Trial Information

Status
Completed
Start date
2014-06-01
End date
2016-06-01
Secondary IDs
Abstract
A randomized experiment with almost 35 million Pandora listeners enables us to measure the sensitivity of consumers to advertising, an important topic of study in the era of ad-supported digital content provision. The experiment randomized listeners into nine treatment groups, each of which received a different level of audio advertising interrupting their music listening, with the highest treatment group receiving more than twice as many ads as the lowest treatment group. By keeping consistent treatment assignment for 21 months, we are able to measure long-run demand effects and compare them to the short-run effects we might obtain with only a month-long experiment. We estimate
a demand curve: number of hours listened as a function of the number of ads per hour received. We also measure the impact on the number of days listened and on the probability of listening at all in the final month of the experiment. We separately vary the number of commercial interruptions per hour and the number of ads per commercial interruption, to measure the extent to which these influence consumer listening behavior. Lastly, we measure the impact on demand for a substitute good: the number of paid subscriptions to the ad-free version of Pandora.
External Link(s)

Registration Citation

Citation
Reiley, David. 2018. "Measuring Consumer Sensitivity to Advertising: A Field Experiment on Pandora Internet Radio." AEA RCT Registry. April 25. https://doi.org/10.1257/rct.2932-1.0
Former Citation
Reiley, David. 2018. "Measuring Consumer Sensitivity to Advertising: A Field Experiment on Pandora Internet Radio." AEA RCT Registry. April 25. https://www.socialscienceregistry.org/trials/2932/history/28793
Experimental Details

Interventions

Intervention(s)
We vary the audio ad capacity for each individual listener.
Intervention Start Date
2014-06-01
Intervention End Date
2016-04-07

Primary Outcomes

Primary Outcomes (end points)
Hours listened in the final month of the experiment.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Days listened in the final month of the experiment. Probability of listening at all in the final month of the experiment. Probability of subscribing to ad-free Pandora One in the final month of the experiment.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
3x3 experimental design. We separately vary the number of commercial interruptions per hour (3, 4, or 6) and the number of ads per commercial interruption (1, 1.5, or 2, where 1.5 means alternating between 1 and 2). We also have a control group 10x as large as each treatment group, and this control group matches the values at the center of the matrix (4x1.5), matching the previous default values listeners received at Pandora before the experiment.
Experimental Design Details
Randomization Method
Randomization based on a hash of the listener ID number at Pandora.
Randomization Unit
Individual listener.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
35 million listeners.
Sample size: planned number of observations
35 million listeners.
Sample size (or number of clusters) by treatment arms
10% of listeners assigned to control (default previous ad load of 1.5x4 per hour), 1% to each of the nine treatments.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials