Fair Play in a Male Dominated Diversity Setting

We examine the fairness of team manager's decisions based on employee's gender identity, and examine two policies (more information and personal reward) to see whether it reduces the expected unfairness in decision-making.

Each team is composed of three members: one male team leader and two predictors, one male and one female. However, the gender or any other demographic information of any team member is not explicitly disclosed to anyone within the team. Instead, participants are assigned random numbers and avatars. These avatars are randomly assigned as either male or female to the two predictors in each team. Consequently, one-fourth of the time, predictors are allocated avatars corresponding to their actual gender, while in two-fourth of cases, one predictor is assigned an avatar of the incorrect gender. In the remaining one-fourth of instances, both predictors receive avatars representing the incorrect gender. These avatars are visible solely to the team leader and remain concealed from the predictors, who can only see the random number assigned to the team members and to the team. The team leader, however, is consistently assigned a male icon, aligning with their actual gender. This approach suggests to the leader that the assigned avatar gender of the predictors corresponds to their true gender. Predictors are, however, informed explicitly that their team comprises of a male leader, a female predictor and a male predictor.

The random allocation of gendered avatars serves as our primary experimental intervention, allowing us to assess whether leaders evaluate individuals based on these assigned genders inferred through the avatars. We refer to this as our Baseline treatment (Treatment T1), which helps us understand whether team leaders exhibit bias in their replacement decisions against female predictors when individual predictor performances are not visible. This experimental manipulation is similar, in spirit, to several other studies that examine the presence of gender bias by varying names on resumes or in emails, and has emerged as a response to challenges in causally identifying discrimination using naturally occurring data. In our case, we vary avatars that appear male or female but do not use any names. Use of avatars to signal gender has been used in several prior studies to induce a particular gender. This intervention helps us understand how a random assignment of gender, controlling for actual performance and team performance, influences team leaders' decisions, providing clearer evidence of biases based on stereotypes and personal tastes.

The experiment will include two additional treatments: Performance Information Treatment (T2) and Reward for Retention Treatment (T3). Treatment T2 aims to determine if providing team leaders with partial visibility into individual predictor performance reduces bias in their replacement decisions, helping to identify whether the discrimination is statistical or taste-based. Leaders will receive performance data for 60% of of the predicted outcomes on a match day, with the remaining 40% performance still ambiguous. Treatment T3 investigates whether incentivizing team leaders to retain their team members promotes fairer treatment of employees and reduces biased decision-making against female avatars, even when individual predictor performance is not visible.

Across all treatments, the primary outcome measured is the team leader's decision to replace or retain existing predictors. The scoring rules will remain consistent across all treatments and teams; however, the visibility of individual performance varies between T1 and T2, and the personal reward for the leader to retain the existing team varies between T1 and T3. Team leaders (or teams) will be randomly assigned to the three treatments (T1, T2, and T3) in a between-subject design, where each team will participate in only one of the three treatments across their four rounds (match days) of surveys. This setup ensures that each team leader is assigned to a single treatment. Due to our primary focus on leaders' decision, we assign each predictor to more than one teams (which can increase decrease depending on replace decision or if a predictor from another team became unresponsive to our study). Each predictor will overwhelmingly be a part of one treatment, but may belong to more than one if such a requirement arises. Male predictors are lesser in number than female predictors, so we assign male predictors to many more teams than female predictors to ensure that each team comprises of one male and one female predictors. Predictors will not predict match outcomes separately for each of their team leaders. They will not be informed about the different treatments, as the treatments only affect the decision-making of the team leaders. Each predictor will be informed about one treatment and will predict the outcomes for the leader assigned to that treatment, unaware that the leader in the other team they are part of may be assigned to a different treatment.

2024-06-17

2024-06-30

Leader's assessment of employees across different treatment conditions
Employees' expectations of how leaders will assess them across different treatment conditions

Leader's assessment will be captured in their decision - retain or replace
Employees will predict their performance relative to other employee (we hypothesize underestimation by females); how leader will assess them (we hypothesize optimism bias by females); and whether they'd be interested in moving to (away from) T2 if they are in T1 or T3 (if they are in T2) (we hypothesize females choosing greater ambiguity work environment).

Leader's evaluation of individual performance captured retrospectively through an end survey (retrospective and not prospectively so as to not force the leaders to think that there is no evidence that one predictor performed better than other, thus manipulating their natural decision making process).

Predictors' evaluation of how Leaders would have behaved in other treatment arms, specifically, we will ask those in T1 and T3 (ambiguity and reward treatments) to assess whether replacement decision will be greater in T2 (info treatment); and will ask those in T2 to assess whether replacement decision will be greater in T1 and T3.

Participants assigned to a team play role of leader and predictors in the ongoing T20 cricket world cup, specifically during the Super 8 stages, starting June 19. They will predict several match related outcomes and earn points. Top 3 ranked teams will win monetary rewards, in addition to a fixed payment that every participant who finishes the study will recieve. The predication exercise will not be conducted during India matches.

In our experimental setup, we randomly assign around 450+ participants to form around 220+ teams, each with three members (these numbers may vary from round to round depending on the unresponsiveness of the existing leaders and availability of waitlisted leaders). These teams are randomly assigned to one of three treatments: T1, T2, and T3. In each treatment, participants designated as predictors will forecast match outcomes before each match day. Predictions will include the winning team (225 points), toss winners (75 points), batting or bowling preferences (65 points), top run-scorers (120 x 2 points), top wicket-taker (110 x 2 points), and man of the match (175 points). Predictors can score a maximum of 1000 points individually and 2000 points collectively as a team. Before each match day, predictors will receive an email with a survey link to predict the match outcomes and answer other survey questions, taking about 3-5 minutes to complete.

After each match day, teams will be scored and ranked based on their performance, with the top three teams receiving monetary rewards. In case of ties within the top three ranks, the reward will be shared equally among all tied teams. For example, if two teams tie for 1st place, they will share the combined reward for the 1st and 2nd ranks equally. Similarly, the distribution will follow suit for other rank ties. The tournament rewards the top-performing teams after each match day, with 1st place receiving Rs. 7000, 2nd place Rs. 5000, and 3rd place Rs. 3000. If there's a tie for 1st place, the combined Rs. 10000 (1st + 2nd place) will be divided between the two teams, awarding Rs. 5000 each. This logic applies to all ties within the top three ranks. For example, with three teams tied for 1st place, the total Rs. 15000 (1st, 2nd, and 3rd place) gets split three ways, giving each team Rs. 5000. The same principle applies for ties in 2nd and 3rd place, ensuring a fair distribution of rewards for exceptional performance, even in the case of a tie.

Team leaders, although not participating in outcome predictions, are entitled to a 50\% share of the reward if their team ranks in the top three, with the remaining 50\% split equally between the two predictors. After each match day, team leaders must decide whether to retain or replace predictors. Replacing a predictor costs the team 100 points, deducted from their next match day score. The team leader has no control over the new predictor, who will be assigned by the experimenter and will be of the same actual gender (unknown to the team leader) but with a randomly assigned avatar visible to the team leader. Each predictor is part of two (or more) teams (depending on the need - due to replacement or unresponsiveness); if replaced by one team leader, the predictor continues to predict for the other team until replaced by that leader as well. The new predictor starts with one team until another leader in a different team replaces a predictor of the same actual gender as the new predictor. Each male predictor is going to be a part of many more teams than a female predictor, due to lesser absolute number of male predictors than female predictors available. Predictors are informed that they may be a part of more than one teams, and we share with them all team numbers that they become a part of, in each round.

After each match day, team leaders receive an email with a survey link detailing their team's scores (and in T2, additional information on individual performance for some prediction questions; and in T3, information that leaders are rewarded to retain their leaders), rank, and are asked to decide whether to replace or retain each predictor. This survey takes about 2-3 minutes to complete. Once team leaders have made their decisions, predictors are emailed their team's score, rank, and the leader's decisions. If retained, they receive a prediction survey for the next match day. If replaced, their participation in that team ends, although they may continue in the other team they were assigned to if the other team's leader retains them. Once both (or all) leaders have replaced a predictor, their participation ends, and they receive their experimental earnings.

In case of a predictor's non-response on any match day during the experiment, the team will be reassigned different predictors from waitlist or existing teams, about which team leader with unresponsive predictors will be informed. Their score for that round will then be based on the remaining or newly assigned participant's responses. This is to ensure that leaders do not get discouraged and dropout from our study in case they score low due to predictors becoming unresponsive. We will maintain the gender match of the original predictor while reassigning predictors, but with randomly assigned avatars. If a team leader fails to respond on a match day, they are replaced by another male player from the leaders' waitlist. In such instances, if the team ranks in the top three, the reward is evenly distributed among the predictors. All participants will receive detailed game rules and provide informed consent before participation. Additionally, demographic information will be collected to ensure balanced groups across treatments.

An end of the experiment survey will be conducted with Leaders and Predictors.

With leaders, this survey will ask leaders to evaluate their two predictors' performance in round 4 of predictions only (while we initially planned to do it retrospectively for all 4 rounds, Qualtrics does not allow us to individually pick avatars assigned to a leader in four different rounds). It will be an incentivized exercise, with correct prediction getting them a monetary reward. This will help us understand whether they believe women avatars would perform differently than male avatars assigned to their team, and whether that is a function of the assigned treatment. We also ask their general perception of women predictors performance relative to men predictors in an incentivized manner based on last round's outcomes (telling them the exact number of male and female predictors in round 4, and asking the percentage of female predictors who will be there in top 15 of all predictors in the last round). Additionally, we make them predict match outcomes for the next cricket match (one of the semi-finals) after round 4, an unincentivized exercise, to gauge their own level of skill in the game, which may moderate how they may judge their predictors in the game rounds.

With predictors, this survey will ask general perception of women predictors performance relative to men predictors in an incentivized manner based on last round's outcomes (telling them the exact number of male and female predictors, and asking the percentage of male predictors who will be there in top 15 of all predictors in the 4th round). This will help us understand whether they believe women are (or are not) going to be good at this exercise, which may explain their confidence (or lack of it) reflected in each round's performance evaluation question. We will also ask them to predict the percent of replace decisions taken for predictors in the partially-ambiguous treatment arm (T2) relative to T1 based on the findings of Round 4. This will help us understand the reason behind their preference (or not of) to switch to T2 if in T1 (or to T1 if in T2) which was asked at the end of Round 2. We will also ask them to predict the replacement decision for predictors in T3 compared to T1, which will help us understand the reasons behind expected greater optimism bias of female in T3 treatment. Lastly, we will ask them if they could go back in time and choose to be with leaders in one of the three treatment arms, which one would they prefer? This will help us see whether female employees self select into organizations based on performance measurability aspect and rewards to the managers for retaining diversity.

Randomization done using Stata

Individual

No

Planned is 900, but we have 450 registrations so far. We will conduct the study with whatever final number of participants we manage to register for the study.

450+

220+ in each of the three conditions (T1= Ambiguous, T2= Info, and T3 = Reward)