Impact of Alignment between Leadership Style and Incentive Systems

Last registered on January 18, 2022

Pre-Trial

Trial Information

General Information

Title
Impact of Alignment between Leadership Style and Incentive Systems
RCT ID
AEARCTR-0008801
Initial registration date
January 14, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
January 18, 2022, 4:04 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
The Hotel School, Cornell SC Johnson College of Business, Cornell University

Other Primary Investigator(s)

PI Affiliation
Arizona State University
PI Affiliation
The Wharton School, University of Pennsylvania

Additional Trial Information

Status
In development
Start date
2022-01-12
End date
2022-12-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
In partnership with a US apparel retailer with over 10,000 employees and 800 stores, we plan a randomized control trial to study the role of leadership style in mediating the effectiveness of different types of incentives. The proposed study involves the collection of detailed socio-demographic and performance information about the sales associates and store managers, including the administration of a survey instrument to capture the leadership style of the latter. In addition, we will implement a field experiment that will vary the prize that stores can win in a series of tournaments we plan to organize. The prizes will be either monetary or non-monetary. We expect to show that servant leaders, that instill a team spirit among their subordinates, respond more to non-monetary incentives, while non-servant leaders respond more to monetary ones.
External Link(s)

Registration Citation

Citation
Casas-Arce, Pablo, Francisco de Asis Martinez Jerez and Joseph Moran. 2022. "Impact of Alignment between Leadership Style and Incentive Systems ." AEA RCT Registry. January 18. https://doi.org/10.1257/rct.8801-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
Each region will be assigned to one of three treatments: sales contest with monetary prize, sales contest with non-monetary prize, and control group without any contest.
Intervention Start Date
2022-01-17
Intervention End Date
2022-01-31

Primary Outcomes

Primary Outcomes (end points)
Sales per employee
Growth in the average number of items per transaction.
Average number of items per transaction
Primary Outcomes (explanation)
Number of items per transaction: number of items in each sale ticket. For instance, if the ticket (transaction) only includes one shirt, then the number of items is 1. If the ticket includes one shirt, a pair of socks, a cap and a pair of gloves, then the number of items in that transaction is 4.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We instructed the partner company to measure the leadership style of the store managers. The measurement took place in the first two weeks of December.
We will assign regions to each of the three experimental groups (monetary incentive, non-monetary incentive, and control) and within each region we will pair stores randomizing the pairs using a two-by-two blocking. The two dimensions of the blocking are product offering (fashion versus sport) and store sales volume (sales above and below the median of the region).
The contests will last for two weeks. We will test our hypotheses using sales and points earned during the contest in a Diff-in-Diffs framework.
Experimental Design Details
III.1. Measure of Leadership Style
Our objective in this study is to measure the impact of alignment between the store manager’s management style and control systems. In the choice between monetary and non-monetary rewards the differential element is that the latter appeal more to the intrinsic motivation of individuals. Intrinsic motivation pertains to the domain of values, team spirit, and higher order goals, in contrast with extrinsic motivation that leverages the employees’ drive for economic compensation and career advancement (Benabou & Tirole, 2003; Deci, 1975). In the management literature there are different constructs that capture the orientation to team, values, and higher goals. The one that better fits our objective is servant leadership (Greenleaf, 1970), which has become more prominent in the last two decades since the formalization by Ehrhart (2004), who defines a servant leader as one who “recognizes his or her moral responsibility not only to the success of the organization but also to his or her subordinates, the organization’s customers, and other organizational stakeholders” (Ehrhart, 2004, p. 68).
Our plan is to use the Ehrhart (2004) (see appendix A) instrument to measure servant leadership. This instrument has been widely adopted in the management literature and it is available for other researchers to use and duplicate results. We considered other instruments like SL-7 (Liden et al., 2015), SLBS-6 (Sendjaya et al., 2019), and SLS (van Dierendonck et al., 2017), but finally chose this one as it has recently achieved some preeminence in the field, although we are receptive to other alternatives.
A common criticism of the leadership literature is that studies overstate the effects of the leadership styles analyzed because they do not include differing leadership theories in their research design (Antonakis et al., 2010; Eva et al., 2019). To address this concern, we also plan to measure behavioral integrity, defined as “the perceived pattern of alignment between a manager’s words and deeds, with special attention to promise keeping, and espoused and enacted values,” (T. Simons, 2002) as this construct directly relates to the store managers’ ability to credibly appeal to the team spirit of the sales associates (Appendix A).
In the management literature leadership style is measured either through the responses of subordinates or the managers themselves. Each approach has advantages as well as disadvantages. Asking subordinates avoids the potentially biased responses of managers that may unconsciously try to project onto themselves the ideal image of the manager, but runs the risk of subordinates responding according to what they think their manager would like to see in their responses. We favor measuring the leadership style of the store manager by surveying sales associates as they may be more receptive to incentive programs that they perceive to be aligned with the style of their manager, and we want to understand the impact of this alignment on employee performance. However, logistically, it is easier to survey managers, as they have a corporate email account, facilitating the administration of the instrument, and they may be more inclined to respond. Thus, because of logistic risks and because one can argue that what is relevant is the manager’s perception of the alignment between her style and the incentive system, we plan to measure leadership style by surveying both sales associates and store managers. We expect both measures to be correlated but we expect to obtain a higher response rate for store managers because we can contact them directly through their corporate emails while we will need to physically distribute personalized links to the employee instrument at each store. We intend to use the responses of the store associates in our main analyses should adequate response levels be achieved, with managers’ self-assessments being used in supplemental analyses if necessary.
To the extent that leadership style could be learned and nurtured it is possible that the leadership style of the store managers has been modeled through a trickle-down-approach throughout the organization (Mayer et al., 2009, 2012). Regional managers who are servant leaders create more servant leaders in the district manager ranks who, in turn, create servant leaders in the store managers via a social learning process. To take this possibility into account we plan to expand our measurement of leadership style to regional and district managers. Thus, the survey that we plan to administer to store and district managers will include questions to assess their own leadership style and that of their immediate supervisor, while the survey of the sales associates will only ask questions about the style of the immediate supervisor, and the survey of the regional managers will only include self-evaluative questions.
Our objective is to conduct the measurement shortly before—one month—the launch of the experiment. In this way we will maximize the overlap between the surveyed population and the employees active during the experiment, a challenge given the high turnover that characterizes the industry.

III.2. Assignment of Stores to Treatments and Control Groups
In assigning the stores to treatments we want to achieve three objectives. First, we want to minimize the likelihood that the stores realize that they are the subject of an experiment, so we avoid strategic behaviors which lead the company to believe that the most effective incentive scheme is the one that the stores prefer. Second, we want to have a representative set of stores in each of the treatments so we can interpret the difference between the performance of the treated and the performance of the control set as the average treatment effect. Finally, we want to create the maximum incentive for stores to exert effort.
To minimize the likelihood of store employees realizing that they are the subject of an experiment, sales associates should perceive the contest as similar to any other contest they have participated before at the company and they should not realize that the rules governing the contest are different for other stores. These objectives require that the assignment to treatment groups or the control be performed at the regional level. Usually, the regional managers gather the district managers to communicate the rules of any contest and communicate the matchings for the contests. Then, regional managers issue electronic notifications of the rules of contest to the store managers in their region. Keeping these customary procedures in the planned treatment contest implies that the assignment to treatments and control groups needs to be conducted at the regional level.
We want stores in each of the treatments and the control groups to be as similar as possible to the population so we can interpret the difference in performance between treatment and control stores as the average treatment effect. If we were to perform the assignment at the store level, we could randomize the assignment to treatments in the expectation that characteristics that affect the treatment effect are evenly distributed across groups. This is a reasonable expectation when the number of experimental units is sufficiently large. However, if we assign treatments at the regional level, we only have six experimental units and randomization may induce large estimation biases. For that reason we propose to use the synthetic control approach described in Abadie & Zhao (2021) in which the large aggregate experimental entities (regions) are assigned to treatments in such a way that minimizes the quadratic difference between the average characteristics of the population and the average characteristics of each of the groups. The procedure is described in more detail in Section VI of this proposal.
Finally, to maximize the incentive to exert effort, within each treatment or control group, we plan to match stores by their ability. In this way we expect that performance throughout the contest remain similar between the store pairs and they do not become complacent or give up on their chances to win the contest (Casas-Arce & Martínez-Jerez, 2009). We plan to use size—measured in sales—and performance in past contests as indicators of ability and match stores according to these criteria, grouping the stores in three buckets (top, middle, and bottom performers based on last three months of sales), and then randomly assign them to contests within each bucket.
III.3. Description of the Treatments
Each region will be assigned to one of three treatments: contest with monetary prize, contest with non-monetary prize, and control group.
In the contests conditions each store will be matched with a store of the same performance category to compete for a prize. The contest will last for two weeks. The store with a higher number of points during the contest period will win each matchup. Points will be earned by adding complements to the sale of items in the basic apparel category. In the monetary prize condition, all sales associates of the winning stores will receive $25 in their paychecks immediately following the end of the contest, representing roughly twice the mean hourly pay rate or 2% of mean monthly take-home pay of the population of store employees. In the non-monetary condition, all sales associates of the winning stores will receive a team-oriented award that is not saleable such as a bowling outing, a movie outing, or a pizza party for all the members of the store.
The rules of the contest and the list of match ups will be communicated to the district and store managers the day before the start of the contest. Stores will receive an update on their performance in the contest three times a week. Winning stores will be announced the day after the end of the contest.
Randomization Method
We will have the regions assigned to treatments using the synthetic control method described in Abadie, A., & Zhao, J. (2021). Synthetic Controls for Experimental Design (Unpublished Working Paper).
Then we will pair stores randomizing within four blocks in each region. The blocks will be formed using a 2-by-2 classification of the stores: one dimension being type of product offering (fashion vs. sport) and the other by sales volume of the store. The randomization will be done in the office by a computer.
Randomization Unit
We will assign regions to treatments (monetary incentive, non-monetary incentive, and control) using the synthetic control method.
We will pair stores in tournaments using a randomized routine that blocks stores by product offering and sales volume.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
About 800 stores
Sample size: planned number of observations
About 10,000 employees.
Sample size (or number of clusters) by treatment arms
About 275 stores in each of the groups (monetary incentives, non-monetary incentives, control)
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Using the sales data from past contests, we conducted power analyses to determine whether our planned intervention could identify effect sizes similar in magnitude to past changes in sales activity due to contests. The results may be found in more detail in the analysis plan. We utilized the multiple regression partial correlation framework per Cohen (1988), which identifies the minimum identifiable change in regression model R2 and minimum identifiable squared partial correlation between a hypothetical independent variable and an outcome variable as a function of sample size, desired power level, statistical significance level, the number of independent variables included in the regression model, and the number of control variables included in the regression model. As a baseline multiple regression model for the power analysis, we utilize a regression of an outcome variable (for brevity, we utilize sales revenue only) measured at the store-day or employee-day level on a vector of store, day of week, and month fixed effects. For store-level analyses, we calculate the number of control variables in the baseline regression model as 1,061 (the total number of store, day of week, and month fixed effects); for employee-level analyses, we calculate the number of control variables in the baseline regression model as 1,065 (the above plus four potential employee-level controls). We calculate minimum detectable effect size and partial squared correlation in models designed to test the hypotheses of the project, for several conventional levels of power (0.80, 0.85, and 0.90) and sample sizes corresponding to a two-, three-, and four-week contest in that order. For comparative purposes, we calculate partial squared correlations obtained from a multiple regression of sales revenue on a contest indicator and store, day of week, and month fixed effects at the store-day and employee-day levels. In all cases, the squared partial correlations of the contest indicator presented in the latter, of 0.0058 and 0.00034 (corresponding to partial correlations of 0.076 and 0.018, respectively), at the store-day and employee day levels, respectively, are well in excess of the minimum detectable squared partial correlations. This indicates that our study has sufficient power to detect a change in sales similar in magnitude to that observed in historical contests run by the company.
IRB

Institutional Review Boards (IRBs)

IRB Name
Cornell University Institutional Review Board
IRB Approval Date
2022-01-14
IRB Approval Number
2112010745
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials