Price Information and Competitive Spillovers in an Online Platform in Pakistan
Last registered on June 14, 2021

Pre-Trial

Trial Information
General Information
Title
Price Information and Competitive Spillovers in an Online Platform in Pakistan
RCT ID
AEARCTR-0007537
Initial registration date
June 13, 2021
Last updated
June 14, 2021 11:35 AM EDT
Location(s)
Region
Primary Investigator
Affiliation
UC Davis
Other Primary Investigator(s)
PI Affiliation
Lahore University of Management Sciences
PI Affiliation
Lahore University of Management Sciences
Additional Trial Information
Status
In development
Start date
2021-06-14
End date
2021-09-20
Secondary IDs
Abstract
Information and search frictions in developing markets may induce agents to make distorted pricing and advertising choices, which in turn may generate further negative externalities that reinforce such frictions. While direct effects of information interventions to alleviate some of these frictions are extensively studied (e.g. Aker 2010; Aker and Mbiti 2010; Jensen 2007), potential spillovers and their mechanisms remain relatively under-explored. We wish to provide empirical insights into the ways in which new, individual price information in a developing economy alters market conditions, or ways in which the effects of new information are altered by such market conditions. To this end, we will conduct a private price information intervention to sellers on a listing platform for used vehicles in Pakistan, PakWheels.com, from June to August 2021. We will generate variation in the timing of treatment by market sub-sections via a two-step cluster-randomization design, in order to separate direct treatment effects from spillovers. In our primary analysis, we will measure direct and spillover effects on a) changes to the listing price, b) occurrence of transaction, c) transaction price, d) an advertising index, and e) page views as a proxy for buyer interest. In our secondary analysis, we will identify ways in which the treatment effects interact with, or in turn affect, market efficiency and structure.
External Link(s)
Registration Citation
Citation
Hasanain, Syed Ali, Shotaro Nakamura and Adeel Tariq. 2021. "Price Information and Competitive Spillovers in an Online Platform in Pakistan." AEA RCT Registry. June 14. https://doi.org/10.1257/rct.7537-1.0.
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details
Interventions
Intervention(s)
In our intervention, we will privately provide sellers price information on a listing platform for used vehicles in Pakistan, called PakWheels. The price information, which PakWheels calls “the Price Calculator”, is an estimate of transaction prices, based on a machine-learning model designed predict the self-reported transaction price of previous listings from the firm’s database. The estimate is conditional on a) the occurrence of transaction and b) observable characteristics of the vehicle, but not of sellers’ characteristics. Our hypothesis is that this information will help sellers identify realistic transaction prices, and set listing prices accordingly.

We will conduct the intervention to a subset of the flow of new posts on PakWheels, as part of the post-making process. On PakWheels’ web platform and mobile apps, sellers can create a new post by clicking on “Post an Ad.” They are first asked to log in, so that we can identify the user-ID associated with each post. Users would neither know their user-ID (it is internal to PakWheels) nor for which last digits we are providing the Price Calculator. Once logged in, users are asked to provide information on the vehicle they intend to sell, and then to set the listing price. If the seller is assigned to treatment, they will then be shown the Price Calculator estimate, i.e. the machine-learning based transaction price forecast, as well as the 10th and 90th percentiles of reported transaction prices for the make-model-model year (or the make-model-model year-version level for popular model-versions). Treated sellers will then be given a chance to update their listing price.
Intervention Start Date
2021-06-14
Intervention End Date
2021-08-09
Primary Outcomes
Primary Outcomes (end points)
• log(absolute difference between listing price and Price Calculator estimate)
• binary transaction outcome
• log(transaction price)
• indexed measure of advertisement usage
• buyer-attention index
• spillover effect on all outcomes above
Primary Outcomes (explanation)
The indexed measure of advertising will be constructed from the following variables:
• number of “bumps” the seller applies to the post
• number of weeks the seller “features” the post
• 1 if the seller requests PakWheels to have the vehicle inspected.

The buyer-attention index will consist of the following variables:
• number of times the post was viewed
• number of times viewers clicked to view the seller’ phone number.

Further detail on data sources and the definitions of component variables is provided in the analysis plan document.
Secondary Outcomes
Secondary Outcomes (end points)
• Survey-verified transaction outcome and price
• Expected transaction price at the time of initial posting (survey-based)
• Post’s duration on the platform
• Alternative measures of list price and changes to it
• Cluster-level outcomes
• Spillovers based on the listing order
Secondary Outcomes (explanation)
Please see the pre-analysis plan for detail.
Experimental Design
Experimental Design
We have designed a blocked, two-step randomization procedure to identify both direct treatment and spillover effects of the price-information intervention. This intervention will be carried out by the firm over 9 weeks.

The platform receives approximately 100,000 valid listings per month. Our sample is every seller that creates a post on the platform during the intervention period, except if we do not have sufficient data to provide a Price Calculator estimate for a given make-models/make-model-model-year. We define these exclusion criteria to be models that have self-reported transaction prices (which we use to provide the Machine-Learning price forecast) from at fewer than 10 listings per month, and to make-model-years with fewer than 3.33 reported transaction prices per month (which we use to provide information on the bounds). These criteria restrict our sample to approximately 88% of total posts, consisting of 73 distinct make-models.

Our two-step randomization process is as follows. In step 1, we will block-randomize the “cluster” of vehicles, defined as the make-model (e.g. Toyota Corolla), into 9 treatment groups, corresponding to the week in which treatment is given to some in the cluster. In step 2, we randomize posts into treatment based on the last digit of the user ID on PakWheels. In order to ensure that treatment and control groups are comparable over the primary outcome variables, we test for balance using listings data from a pre-treatment period with the same sample inclusion criteria as the experiment. We iterate the randomization procedure over 500 seeds, and randomly choose one seeds among the subgroup of assignments that failed to reject statistically significant differences in all primary outcome variables, adjusted for the false discover rate.

In step 1, we block the make-model clusters over standardized cluster-level means of primary outcome variables. Blocking is done with R’s blockTools package (Moore, 2012), which uses the optimal-greedy algorithm over the Mahalambois distance. We weight the five main outcome variables (log of price difference, occurrence of transaction, transaction price, advertising use, and demand index) twice as heavily as the cluster size. Our choice of weights is admittedly arbitrary, but we rationalize this as our objective is to balance over main outcome variables. Using these groupings, we will identify the first-stage assignment, i.e. the week in which some listings in the cluster is assigned treatment. In “Treat” clusters, we will randomly select 50% of new listings to receive the Price Calculator estimates, while none of the new listings in the “Control” clusters will receive them. On the 9th week, all new listings will receive the Price Calculator.

In step 2, listing-level randomization is done based on the last digit of their user-ID on PakWheels. The choice of 5 digits is fixed across cluster and time, so as to limit the extent of potential interference and for logistical simplicity. The second-stage treatment digits are chosen based on a random number generator.

The intervention is designed in a way that limits the extent of treatment non-compliance as much as possible; for those randomly assigned treatment, the Price Calculator estimate is given automatically on the user interface during the post-making process. One exception is
that the sellers that use PakWheels’ mobile app can only receive Price Calculator estimates once they update their mobile app after the start of the intervention period. On the other hand, there is no need for an update if the user accesses PakWheels.com via the web (including internet browsers on mobile phones), so anyone assigned to treatment would receive it automatically. This may generate selection into treatment conditional on assignment, based on a) users’ preference for PakWheels’ app and b) their propensity to update the app. To this end, we will plan on identifying both intend-to-treat as well as treatment-on-treated effects. We will use the assignment variable as an instrument for treatment.
Experimental Design Details
Not available
Randomization Method
As described in the Experimental Design section, the first-stage is cluster-randomized based on outcome means at the make-model level, and the second stage randomization is done by a random number generator in R.
Randomization Unit
The first stage is clustered at the make-model level, and second stage is randomized based on the last digit of the user ID.
Was the treatment clustered?
Yes
Experiment Characteristics
Sample size: planned number of clusters
We expect to have 73 clusters (i.e. the make-models), but this is dependent on for how many make-models we would be able to offer Price Calculator estimates.
Sample size: planned number of observations
We expect to have a sample of 90,000 to 140,000, depending on the number of new posts over the treatment period that meet the sample inclusion criteria.
Sample size (or number of clusters) by treatment arms
Approximately 25% (22,500-35,000) of posts will be directly treated, and another 25% will be in the spillover group. Remaining 50% will be the control group.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Through simulations using real data from PakWheels, we find that we are able to detect treatment and spillover effects of 0.05 times the standard deviation for most primary outcome groups, both direct and spillover, with over 80% probability. The exception is the spillover effect of the binary transaction outcome, where we have sufficiently high power for an effect size just above 0.1 SD. These effect sizes translate roughly into 4,900 PKR (32 USD) in absolute difference between listing price and Price Calculator estimate (level mean: 138,000 PKR), 5 percentage-points in transaction probability (mean: 0.42), and 49,000 PKR (320 USD) in transaction price (level mean: 1,735,707 PKR).
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
UC Davis IRB Administration
IRB Approval Date
2020-10-08
IRB Approval Number
1647279-1
IRB Name
Lahore University of Management Sciences IRB
IRB Approval Date
2020-09-07
IRB Approval Number
LUMS-IRB/08092020AT
IRB Name
UC Davis IRB Administration
IRB Approval Date
2021-06-01
IRB Approval Number
1647279-2
IRB Name
Lahore University of Management Sciences IRB
IRB Approval Date
2021-06-05
IRB Approval Number
LUMS-IRB/06042021/AT-FWA-00019408
Analysis Plan

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information