Impact of Alignment between Leadership Style and Incentive Systems

Each region will be assigned to one of three treatments: sales contest with monetary prize, sales contest with non-monetary prize, and control group without any contest.

2022-01-17

2022-01-31

Sales per employee
Growth in the average number of items per transaction.
Average number of items per transaction

Number of items per transaction: number of items in each sale ticket. For instance, if the ticket (transaction) only includes one shirt, then the number of items is 1. If the ticket includes one shirt, a pair of socks, a cap and a pair of gloves, then the number of items in that transaction is 4.

We instructed the partner company to measure the leadership style of the store managers. The measurement took place in the first two weeks of December.
We will assign regions to each of the three experimental groups (monetary incentive, non-monetary incentive, and control) and within each region we will pair stores randomizing the pairs using a two-by-two blocking. The two dimensions of the blocking are product offering (fashion versus sport) and store sales volume (sales above and below the median of the region).
The contests will last for two weeks. We will test our hypotheses using sales and points earned during the contest in a Diff-in-Diffs framework.

Not available

We will have the regions assigned to treatments using the synthetic control method described in Abadie, A., & Zhao, J. (2021). Synthetic Controls for Experimental Design (Unpublished Working Paper).
Then we will pair stores randomizing within four blocks in each region. The blocks will be formed using a 2-by-2 classification of the stores: one dimension being type of product offering (fashion vs. sport) and the other by sales volume of the store. The randomization will be done in the office by a computer.

We will assign regions to treatments (monetary incentive, non-monetary incentive, and control) using the synthetic control method.
We will pair stores in tournaments using a randomized routine that blocks stores by product offering and sales volume.

Yes

About 800 stores

About 10,000 employees.

About 275 stores in each of the groups (monetary incentives, non-monetary incentives, control)

Using the sales data from past contests, we conducted power analyses to determine whether our planned intervention could identify effect sizes similar in magnitude to past changes in sales activity due to contests. The results may be found in more detail in the analysis plan. We utilized the multiple regression partial correlation framework per Cohen (1988), which identifies the minimum identifiable change in regression model R2 and minimum identifiable squared partial correlation between a hypothetical independent variable and an outcome variable as a function of sample size, desired power level, statistical significance level, the number of independent variables included in the regression model, and the number of control variables included in the regression model. As a baseline multiple regression model for the power analysis, we utilize a regression of an outcome variable (for brevity, we utilize sales revenue only) measured at the store-day or employee-day level on a vector of store, day of week, and month fixed effects. For store-level analyses, we calculate the number of control variables in the baseline regression model as 1,061 (the total number of store, day of week, and month fixed effects); for employee-level analyses, we calculate the number of control variables in the baseline regression model as 1,065 (the above plus four potential employee-level controls). We calculate minimum detectable effect size and partial squared correlation in models designed to test the hypotheses of the project, for several conventional levels of power (0.80, 0.85, and 0.90) and sample sizes corresponding to a two-, three-, and four-week contest in that order. For comparative purposes, we calculate partial squared correlations obtained from a multiple regression of sales revenue on a contest indicator and store, day of week, and month fixed effects at the store-day and employee-day levels. In all cases, the squared partial correlations of the contest indicator presented in the latter, of 0.0058 and 0.00034 (corresponding to partial correlations of 0.076 and 0.018, respectively), at the store-day and employee day levels, respectively, are well in excess of the minimum detectable squared partial correlations. This indicates that our study has sufficient power to detect a change in sales similar in magnitude to that observed in historical contests run by the company.