Signals We Give: Gender, Feedback, and Competition

2021-02-01

2022-12-31

Our primary outcome variables are:
1. Managers’ choice of feedback to be sent to their Worker [FeedbackSent]
2. Worker’s choice of payment scheme [ChoiceTournament]

Manager feedback is ordinal (precise, moderate, vague). For part of our analysis we will consider binary indicators for whether feedback is precise, as well as whether feedback is vague.

Our secondary outcome variables are:
1. Manager’s prior and posterior beliefs about the Worker’s performance [BeliefWorkerPrior; BeliefWorkerPosterior]
2. Worker’s prior and posterior beliefs about their own performance [BeliefPrior; BeliefPosterior]

During the experiment, beliefs will be elicited over a distribution, where subjects will allocate 10 tokens across four possible quartile ranks. Hence, two possible measures of beliefs are used: (i) number of tokens assigned to each quartile rank, separately; and (ii) the expected rank (i.e., mean of the distribution of tokens).

Participants will be divided into groups of two. One participant is assigned the role of the Manager while the other is assigned the role of the Worker. Managers will receive a signal about their matched Worker’s task performance and choose how they would like to provide feedback to their Worker. The Worker then chooses how they would like to be compensated for their task performance.

NOTE: This project is a follow-up experiment from a previously registered trial (https://www.socialscienceregistry.org/trials/5543), which was designed as a laboratory experiment. That experiment was abandoned due to the COVID-19 pandemic. In this study, the experiment has been redesigned to be conducted online and to address a revised set of research questions.

Participants will be divided into groups of two. One participant is assigned the role of the Manager while the other is assigned the role of the Worker.

The experiment is conducted asynchronously over Prolific in three parts. Workers first complete Part 1. Then Managers in Part 2 will receive information about their matched Worker’s performance in Part 1. Then, the same group of Workers will be invited to Part 3 where they will receive the feedback from their matched Managers in Part 2.

Part 1 (Workers):
Workers first complete a pre-experimental questionnaire to allow us to elicit their demographic characteristics. Some of their responses will then be used to create a short CV, which includes their gender, that will be presented to their matched Managers (in Part 2).

Workers will then complete a set of 20 questions of Progressive Raven’s Matrices (framed as an IQ task). Each Worker will be ranked against 19 other participants who have completed the same task, and they will be paid based on their quartile rank. However, they do not receive any feedback about their performance at this stage. We will also elicit Workers’ prior beliefs about their relative performance.

Part 2 (Manager):
Managers complete the same set of tasks as in Part 1 (i.e., pre-experimental questionnaire, IQ task, and elicitation of prior beliefs).

Managers are then shown the CV of their matched Worker, containing information about their quarter of birth, favorite hot beverage, favorite color, and gender. Depending on the treatment, gender-specific pronouns (e.g., he/ she) will be used whenever the instructions refer to the Worker.
Note: This is our first treatment dimension (Worker’s gender is either Male or Female).

Following which, each Manager’s prior belief about their Worker’s performance is elicited.

Next, Managers will receive a signal of their Worker’s quartile rank in Part 1. This signal could either be precise (Quartile 1/2/3/4) – 65% of all Managers will receive this, moderate (Top/Bottom half) – 30% of all Managers will receive this, or vague (Rank is between 1 and 4) – 5% of all Managers will receive this. Managers will then be asked to choose how they would like to convey this signal to their Worker. Specifically, they can choose to send either send precise, moderate, or vague feedback. Each Manager’s options for feedback are limited based on the nature of the signal they received (i.e., they cannot send a precise feedback if they only observed a vague signal), and feedback is restricted to be true.

Note: This is our second treatment dimension (signal is either Precise, Moderate, or Vague). The probability distribution of signal types {Precise, Moderate, Vague} is given by {0.65, 0.30, 0.05}. Both Managers and Workers will know the exact probability distribution of each signal type.

Our third treatment dimension is whether the Manager’s feedback is instrumental or non-instrumental to the Worker. Specifically, Managers are informed that in Part 3, their Workers will be reevaluated for their performance against a new group of 20 participants. The Worker will therefore receive a new quartile rank based on their performance in the new group.

This instrumentality of the Manager’s feedback is determined by the timing of the feedback relative to the Worker’s decision. Each Manager’s matched Worker will receive their feedback either before or after they make a decision on how they would like to be paid based on their new rank. Specifically, Worker’s will be asked to choose whether to receive piece rate or a tournament payment based on their new rank. However, they will either receive the Manager’s feedback before they make their payment choice (i.e., feedback will be instrumental) or after they have made this decision (i.e., feedback will not be instrumental).

Finally, we will elicit each Manager’s posterior beliefs about their Worker’s new rank given the signal they have received about their Worker’s rank in Part 1.

At the end of the experiment, Managers will be asked to complete a survey asking them questions about their decisions in the experiment, as well as an Implicit Association Test on gender bias.

Part 3 (Workers):
The Workers from Part 1 of the experiment will be invited to complete this part of the experiment. They are first provided a summary of the instructions from Part 1 as a reminder.

Workers are informed that, in Part 3, they will be reevaluated for their performance against a new group of 20 participants. The Worker will therefore receive a new quartile rank based on their performance in the new group, and they can choose whether to receive a piece rate or tournament payment for this new rank.

Workers in both the Instrumental and Non-Instrumental treatments will first be asked to make a choice between the two payment schemes.

Then, they are informed of the decisions faced by the Managers in Part 2 (i.e., the possible signals Managers have received, as well as the feedback decisions they were asked to make). Each Worker is then shown the feedback sent by their Manager, and we elicit their posterior beliefs about their new rank.

Following which, only in the Instrumental treatment, Workers are asked again to make a choice between the two payment schemes for their new rank. They are informed that this decision will overwrite their previous decision and be used to determine their payment in Part 2. Workers in the Non-Instrumental treatment do not make any decisions in this stage.

At the end of the experiment, Workers will be asked to complete a survey asking them questions about their decisions in the experiment.

Our experiment follows a 2 × 2 × 2 between-subject treatment design and conducted entirely using participants recruited from Prolific. We will recruit a gender-balanced sample for both Workers and Managers.

All treatment assignment is determined randomly using the following protocols:
1. Worker’s gender: Managers will be matched with either a male Worker or a female Worker. This is randomly determined at the individual level by the computer software, such that there are an equal number of male and female Workers to be matched to Managers.
2. Instrumentality of Manager’s feedback (i.e., timing of feedback): Each Manager-Worker pair will be randomly assigned to either the Instrumental treatment or the Non-Instrumental treatment. This will again be randomly determined at the individual level by the computer software, such that there are an equal number of Managers in each treatment.
3. Precision of signal received by Managers: Each Manager will either receive a precise, moderate, or vague signal about their Worker’s performance. This is randomly determined at the individual level using the computer software. The probability distribution of signal types {Precise, Moderate, Vague} is fixed at {0.65, 0.30, 0.05}. Note that managers receiving a vague signal will be excluded from the analysis.

The unit of randomization is at each Manager-Worker pair level.

Yes

1,200 (1,200 Workers and 1,200 Managers).

2,400 (1,200 Workers and 1,200 Managers).

We plan to recruit 1,200 Managers that will be pre-assigned to one of these four treatment cells: (i) female Workers × instrumental feedback; (ii) female Workers × non-instrumental feedback; (iii) male Workers × instrumental feedback; and (iv) male Workers × non-instrumental feedback. Participants will be recruited via Prolific so that there is a gender-balanced sample of Workers to be matched to Managers. The experiment software will ensure that the Manger-Worker pairs are uniformly assigned to the instrumental and non-instrumental treatments.

Hence, within each treatment cell, we expect to have 300 Manager-Worker pairs (i.e., 600 participants). Given the distribution of signal precision, this implies that about 200 Managers within each cell will receive precise signals, while about 100 Managers will receive moderate signals. The probability of receiving a vague signal is negligible, and these observations will be excluded from the analysis.

The primary outcome variable is the Manager’s choice of feedback. Specifically, we consider whether Managers choose to increase the noise of the signal by sending a less precise message to the Worker, or to keep the signal precision as it is. Hence, the Manager’s feedback choice will be transformed into a binary outcome variable. As a main test of our theoretical predictions, we consider pairwise comparisons of the Manager’s feedback choice between the non-instrumental and instrumental treatments for a given signal that they have received. With 6 possible signal types (ranks between 1 and 4, top half, and bottom half), 2 treatments, and 1,200 Managers, this implies that we have 100 Managers in each cell. Our power calculation is based on: (i) baseline proportions of 0.01, 0.25, 0.50, or 0.75; (ii) one-tailed z-tests of differences between two independent proportions; (iii) Type I error rate of 0.05 and power of 0.80. Given these parameters, the minimum detectable effect size is an increase in proportion of between 0.074 and 0.172 in the treated group. The analysis of gender differences in feedback provision will primarily be conducted using parametric regressions. Using simple linear probability models, we first consider a baseline model of Manager’s feedback choice against the Worker’s gender, the Manager’s gender, the signal type received by the Manager, the treatment variable (instrumental vs. non-instrumental), as well as the Manager’s posterior and second-order beliefs about the Worker’s ability. This provides a total of 10 predictors with 1 predictor to be tested. Considering a F-test of an increase in R2 with 1,200 Managers, a Type I error rate of 0.05, and a power of 0.80, the minimum detectable effect size is 0.0066. To allow for heterogeneity in treatment effects, we also consider interaction terms between the Worker’s gender and Manager’s signals in the regression model. This gives a total of 15 predictors of which 6 are to be tested. Using the same parameters as above, the minimum detectable effect size is 0.0114. On the Workers’ end, the main outcome variable is a binary decision of incentive choice on their Part 3 rank (competition versus piece-rate). Our treatment comparisons will be similar to that for Managers, except that we will now consider the feedback sent by the Managers instead of the signal that they have received. Hence, the power calculations and minimum detectable effect sizes follow that of the preceding paragraphs.

Pre-analysis plan (Version 2021, January 27)