Guiding Consumer Search and Seller Competition with Ranking Algorithms

Last registered on July 16, 2024
View Trial History

Pre-Trial

Trial Information

General Information

Title
Guiding Consumer Search and Seller Competition with Ranking Algorithms
RCT ID
AEARCTR-0013988
Initial registration date
July 09, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
July 16, 2024, 2:32 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
United States of America
Region
Online

Primary Investigator

Name
Gaoyang Cai
Affiliation
Northwestern
Contact Primary Investigator

Other Primary Investigator(s)

Additional Trial Information

Status
In development
Start date
2024-09-01
End date
2024-09-22
Keywords
Welfare
Additional Keywords
Digital Platform, Ranking Algorithm, Product Discovery, Consumer Search
JEL code(s)
C93, D12, D83
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This research studies the impact of platform ranking algorithms on consumer beliefs, and behaviors in product discovery, search, and purchase, as seller price competition. I conduct a field experiment focusing on the demand side to disentangle the impacts on consumer beliefs from consumer search costs. The experiment divides participants into four groups, varying in the belief elicitation process and product ranking algorithm. The experiment data will be used to estimate a structural model that incorporates two-sided markets and platform design in ranking algorithms.
External Link(s)

Registration Citation

Citation
Cai, Gaoyang. 2024. "Guiding Consumer Search and Seller Competition with Ranking Algorithms." AEA RCT Registry. July 16. https://doi.org/10.1257/rct.13988-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
This field experiment has three types of interventions. The first is different belief elicitation processes. The second is the different product ranking algorithms facing consumers who intend to search for and purchase products. The third is different information disclosures.

The first intervention involves two types of belief elicitation processes: the first is to incentivize participants to fill out a prior belief survey and install data collection apps to collect their search and purchase data in the Amazon mobile app. After consumers discover, search for, or purchase products on Amazon, they will be finally incentivized to complete a posterior belief survey, while the second is to sequentially alternate between showing them products according to different ranking algorithms and eliciting beliefs.

The second intervention involves two types of product ranking algorithms: the first is Amazon's algorithm (with no additional change); the second is the random ranking algorithm, specifically, randomizing the order of products shown on the first result page under search terms consumers intended to search on Amazon.

The third intervention involves different information disclosed to consumers: consumers in the control group will face Amazon's ranking algorithm with full information (one control group uses the first belief elicitation process; the other control group uses the second); consumers in one treatment group will face the random ranking algorithm with no information, while consumers in the other treatment group will face the random ranking algorithm with full information.
Intervention (Hidden)
Intervention Start Date
2024-09-08
Intervention End Date
2024-09-15

Primary Outcomes

Primary Outcomes (end points)
Consumer prior and posterior beliefs on rank-specific utilities and joint distribution of product prices and qualities; consumer scrolling and clicking on the product listing page; consumer product purchase
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
I randomly divide participants into 4 groups, each has 1/4 of the total participants.

1/4 of the subjects will be randomly selected into the control group C1. Subjects in this group will not have any interventions; they will be directly incentivized to fill out a prior belief survey and install data collection apps to collect their search and purchase data in the Amazon mobile app. After consumers discover, search for, or purchase products on Amazon, they will be finally incentivized to complete a posterior belief survey.

Another 1/4 of the subjects will be randomly selected into the control group C2, they will not be incentivized to install data collection apps. Instead, we will sequentially ask them to report their prior beliefs on the benefits they can obtain from discovering products on Amazon, and show them products under the search term they intend to search according to Amazon’s product rankings. And then ask participants to report their posterior beliefs on the benefits they can obtain from discovering products at subsequent search ranks.

The remaining 1/2 subjects will be assigned to the treatment groups, among them 1/2 (1/4 out of the total number of participants) will be randomly assigned to treatment group 1. They will be asked to complete a prior belief survey before being incentivized to install the data collection apps. However, we will randomize the product orders displayed on the first page of Amazon mobile apps under the corresponding search term consumers intend to search. Participants in this group will not be informed of the random ranking algorithm. After consumers discover, search for, or purchase products on Amazon, they will be finally incentivized to complete a posterior belief survey.
The other 1/2 (1/4 out of the total number of participants) will be randomly assigned to treatment group 2. The only difference between this group and treatment group 1 is that participants in this group will be informed of the random ranking algorithm.
Experimental Design Details
Randomization Method
Randomization is done in office by a computer
Randomization Unit
Individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
1000 individuals
Sample size: planned number of observations
1000
Sample size (or number of clusters) by treatment arms
2 control groups and 2 treatment groups
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
The minimum detectable effect size is 1%, since each group is expected to have about 100 responsive participants (250 * 40%).
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials