Experimental Design Details
III.1. Measure of Leadership Style
Our objective in this study is to measure the impact of alignment between the store manager’s management style and control systems. In the choice between monetary and non-monetary rewards the differential element is that the latter appeal more to the intrinsic motivation of individuals. Intrinsic motivation pertains to the domain of values, team spirit, and higher order goals, in contrast with extrinsic motivation that leverages the employees’ drive for economic compensation and career advancement (Benabou & Tirole, 2003; Deci, 1975). In the management literature there are different constructs that capture the orientation to team, values, and higher goals. The one that better fits our objective is servant leadership (Greenleaf, 1970), which has become more prominent in the last two decades since the formalization by Ehrhart (2004), who defines a servant leader as one who “recognizes his or her moral responsibility not only to the success of the organization but also to his or her subordinates, the organization’s customers, and other organizational stakeholders” (Ehrhart, 2004, p. 68).
Our plan is to use the Ehrhart (2004) (see appendix A) instrument to measure servant leadership. This instrument has been widely adopted in the management literature and it is available for other researchers to use and duplicate results. We considered other instruments like SL-7 (Liden et al., 2015), SLBS-6 (Sendjaya et al., 2019), and SLS (van Dierendonck et al., 2017), but finally chose this one as it has recently achieved some preeminence in the field, although we are receptive to other alternatives.
A common criticism of the leadership literature is that studies overstate the effects of the leadership styles analyzed because they do not include differing leadership theories in their research design (Antonakis et al., 2010; Eva et al., 2019). To address this concern, we also plan to measure behavioral integrity, defined as “the perceived pattern of alignment between a manager’s words and deeds, with special attention to promise keeping, and espoused and enacted values,” (T. Simons, 2002) as this construct directly relates to the store managers’ ability to credibly appeal to the team spirit of the sales associates (Appendix A).
In the management literature leadership style is measured either through the responses of subordinates or the managers themselves. Each approach has advantages as well as disadvantages. Asking subordinates avoids the potentially biased responses of managers that may unconsciously try to project onto themselves the ideal image of the manager, but runs the risk of subordinates responding according to what they think their manager would like to see in their responses. We favor measuring the leadership style of the store manager by surveying sales associates as they may be more receptive to incentive programs that they perceive to be aligned with the style of their manager, and we want to understand the impact of this alignment on employee performance. However, logistically, it is easier to survey managers, as they have a corporate email account, facilitating the administration of the instrument, and they may be more inclined to respond. Thus, because of logistic risks and because one can argue that what is relevant is the manager’s perception of the alignment between her style and the incentive system, we plan to measure leadership style by surveying both sales associates and store managers. We expect both measures to be correlated but we expect to obtain a higher response rate for store managers because we can contact them directly through their corporate emails while we will need to physically distribute personalized links to the employee instrument at each store. We intend to use the responses of the store associates in our main analyses should adequate response levels be achieved, with managers’ self-assessments being used in supplemental analyses if necessary.
To the extent that leadership style could be learned and nurtured it is possible that the leadership style of the store managers has been modeled through a trickle-down-approach throughout the organization (Mayer et al., 2009, 2012). Regional managers who are servant leaders create more servant leaders in the district manager ranks who, in turn, create servant leaders in the store managers via a social learning process. To take this possibility into account we plan to expand our measurement of leadership style to regional and district managers. Thus, the survey that we plan to administer to store and district managers will include questions to assess their own leadership style and that of their immediate supervisor, while the survey of the sales associates will only ask questions about the style of the immediate supervisor, and the survey of the regional managers will only include self-evaluative questions.
Our objective is to conduct the measurement shortly before—one month—the launch of the experiment. In this way we will maximize the overlap between the surveyed population and the employees active during the experiment, a challenge given the high turnover that characterizes the industry.
III.2. Assignment of Stores to Treatments and Control Groups
In assigning the stores to treatments we want to achieve three objectives. First, we want to minimize the likelihood that the stores realize that they are the subject of an experiment, so we avoid strategic behaviors which lead the company to believe that the most effective incentive scheme is the one that the stores prefer. Second, we want to have a representative set of stores in each of the treatments so we can interpret the difference between the performance of the treated and the performance of the control set as the average treatment effect. Finally, we want to create the maximum incentive for stores to exert effort.
To minimize the likelihood of store employees realizing that they are the subject of an experiment, sales associates should perceive the contest as similar to any other contest they have participated before at the company and they should not realize that the rules governing the contest are different for other stores. These objectives require that the assignment to treatment groups or the control be performed at the regional level. Usually, the regional managers gather the district managers to communicate the rules of any contest and communicate the matchings for the contests. Then, regional managers issue electronic notifications of the rules of contest to the store managers in their region. Keeping these customary procedures in the planned treatment contest implies that the assignment to treatments and control groups needs to be conducted at the regional level.
We want stores in each of the treatments and the control groups to be as similar as possible to the population so we can interpret the difference in performance between treatment and control stores as the average treatment effect. If we were to perform the assignment at the store level, we could randomize the assignment to treatments in the expectation that characteristics that affect the treatment effect are evenly distributed across groups. This is a reasonable expectation when the number of experimental units is sufficiently large. However, if we assign treatments at the regional level, we only have six experimental units and randomization may induce large estimation biases. For that reason we propose to use the synthetic control approach described in Abadie & Zhao (2021) in which the large aggregate experimental entities (regions) are assigned to treatments in such a way that minimizes the quadratic difference between the average characteristics of the population and the average characteristics of each of the groups. The procedure is described in more detail in Section VI of this proposal.
Finally, to maximize the incentive to exert effort, within each treatment or control group, we plan to match stores by their ability. In this way we expect that performance throughout the contest remain similar between the store pairs and they do not become complacent or give up on their chances to win the contest (Casas-Arce & Martínez-Jerez, 2009). We plan to use size—measured in sales—and performance in past contests as indicators of ability and match stores according to these criteria, grouping the stores in three buckets (top, middle, and bottom performers based on last three months of sales), and then randomly assign them to contests within each bucket.
III.3. Description of the Treatments
Each region will be assigned to one of three treatments: contest with monetary prize, contest with non-monetary prize, and control group.
In the contests conditions each store will be matched with a store of the same performance category to compete for a prize. The contest will last for two weeks. The store with a higher number of points during the contest period will win each matchup. Points will be earned by adding complements to the sale of items in the basic apparel category. In the monetary prize condition, all sales associates of the winning stores will receive $25 in their paychecks immediately following the end of the contest, representing roughly twice the mean hourly pay rate or 2% of mean monthly take-home pay of the population of store employees. In the non-monetary condition, all sales associates of the winning stores will receive a team-oriented award that is not saleable such as a bowling outing, a movie outing, or a pizza party for all the members of the store.
The rules of the contest and the list of match ups will be communicated to the district and store managers the day before the start of the contest. Stores will receive an update on their performance in the contest three times a week. Winning stores will be announced the day after the end of the contest.