Testing incentive schemes in a retail setting

Last registered on March 13, 2023

Pre-Trial

Trial Information

General Information

Title
Testing incentive schemes in a retail setting
RCT ID
AEARCTR-0011044
Initial registration date
March 11, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
March 13, 2023, 3:26 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
The Hotel School, Cornell SC Johnson College of Business, Cornell University

Other Primary Investigator(s)

PI Affiliation
Arizona State University

Additional Trial Information

Status
In development
Start date
2023-03-15
End date
2023-06-30
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
The proposed objective the research team has converged on is to evaluate how much the incentive effect of a relative performance measurement system (such as a sales contest) is affected by the contestant’s knowledge of its opponent in the competition: their ability, their race, and their gender.
External Link(s)

Registration Citation

Citation
Casas-Arce, Pablo and Francisco de Asis Martinez Jerez. 2023. "Testing incentive schemes in a retail setting." AEA RCT Registry. March 13. https://doi.org/10.1257/rct.11044-1.0
Experimental Details

Interventions

Intervention(s)
We plan to organize a sales contest among stores during the NCAA basketball playoffs and championship games. We will have one-on-one matches of stores every week of the March basketball competition. The winner will be the store that achieves a higher growth in number of transactions over the last year.
In each of the three contests, stores will compete in one-on-one matches similar to those in the March Madness bracket. However, there will be no eliminations. Stores will be matched to a different store each week.
Intervention Start Date
2023-03-15
Intervention End Date
2023-04-04

Primary Outcomes

Primary Outcomes (end points)
Our main variables of interest are sales and number of transactions. Number of transactions is the performance metric of the tournament but the ultimate objective of the firm is to increase sales.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We will assign regions to each of the two experimental groups (full information and restricted information) and within each region we will pair stores randomizing the pairs using a two group blocking. The dimension of the blocking is product offering (fashion versus sport).
The contests will last for 3-4 days coinciding with the NCAA basketball games. We will test our hypotheses using dollar sales and number of transactions in a Diff-in-Diffs framework.
Experimental Design Details
We propose to have a series of three short contests to coincide with and be named after the First Round, Sweet Sixteen, and Final Four. To make them even more like the NCAA championship, where the rounds are held over weekends, we plan limiting these contests to 3 or 4 days, not the whole week.

a. Contests dates

First Round (or First Round-Second Round): Thursday, March 16 to Sunday, March 19.
Sweet Sixteen (or Sweet Sixteen-Elite Eight): Thursday, March 23 to Sunday, March 26.
Final Four: Saturday April 1 to Monday, April 3.

b. Tournament rules

In each of the three contests, stores will compete in one-on-one matches similar to those in the March Madness bracket. However, there will be no eliminations. Stores will be matched to a different store each week.

There will be two “treatments” or types of contests. Half of the stores in the tournament will know the identity of their opponents. The other half will not know who their opponents are. In this way we can gauge the impact that knowing the identity of their opponents has on the effort of the stores.

Different from the NCAA March Madness, the results of one round of contests will have no impact on the following round nor on the prizes to be collected by the winners. Each week the winners of all matches get a monetary prize per sales associate.

c. Performance metric

The performance metric we are suggesting is percentage increase in number of transactions (tickets) with respect to the same period of the prior year. In the case of a tie, we will use the percentage growth in number of pair of shoes sold as a tie breaker. If the tie subsists there will be a coin toss to decide the winner, or a second tiebreaker measure.

The use of a growth metric has the advantage of facilitating comparisons across stores. Thus, a larger store could be competing with a smaller store, but it will still have to increase its effort by a amount similar to the small store to achieve the same performance. This helps to equalize the stores’ abilities across different sizes.

The number of transactions is related to and should be correlated with dollar sales and should focus the associates’ attention on closing the transaction, not so much on the number of items or the value of the items in the transaction. This could help with customer satisfaction if customers feel pressured to buy more or more expensive items. However, it has the disadvantage of being manipulable by the store associates who could, for example, transform a single sale of two items in two sales of one item. We could turn this disadvantage into a benefit because it will allow us to understand under what circumstances the store associates are more likely to manipulate this metric.

For the base period to calculate the percentage growth in number of sales transactions we will use the exact same tournament period in 2022. This means that for the First Round we will use Thursday, March 17 to Sunday, March 20, 2022; for the Sweet Sixteen: Thursday, March 24 to Sunday, March 27, 2022; and for the Final Four: Saturday, April 2 to Monday, April 4, 2022.
Randomization Method
We will have the regions assigned to treatments using the synthetic control method described in Abadie, A., & Zhao, J. (2021). Synthetic Controls for Experimental Design (Unpublished Working Paper).
Then we will pair stores randomizing within two blocks in each region. The blocks will be formed classifying the stores by the type of product offering (fashion vs. sport). The randomization will be done in the office by a computer.
Randomization Unit
We will assign regions to treatments (full information and restricted information) using the synthetic control method.
We will pair stores in tournaments using a randomized routine that blocks stores by product
offering.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
About 800 stores.
Sample size: planned number of observations
About 10,000 employees.
Sample size (or number of clusters) by treatment arms
About 300 stores in the limited information treatment and 500 stores in the full information treatment.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Testing incentive schemes in a retail setting
IRB Approval Date
2023-03-13
IRB Approval Number
IRB0147397
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials