Managerial implicit stereotypes and where to find them: Evidence from Incentivized Resume Rating

Last registered on May 16, 2022

Pre-Trial

Trial Information

General Information

Title
Managerial implicit stereotypes and where to find them: Evidence from Incentivized Resume Rating
RCT ID
AEARCTR-0008308
Initial registration date
October 26, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
October 27, 2021, 10:19 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
May 16, 2022, 6:56 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
Bocconi University

Other Primary Investigator(s)

PI Affiliation
Bocconi University
PI Affiliation
Bocconi University

Additional Trial Information

Status
Completed
Start date
2021-11-15
End date
2022-03-30
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Do implicit gender bias affect managers' discriminating behaviors? Does becoming aware of own implicit bias attenuate discrimination?

The project consist of a two-step lab-in-the-field experiment. As a general summary: we will evaluate gender bias of managers through the IAT test, reveal to the respondents of the treatment group their own score in the IAT, deliver a second survey in which we will be asking managers from treatment and control group to evaluate explicitly hypothetical students profiles according to the methods of the Incentivized Resume Rating.

The experiment is done on two different population:
1) the population of Italian managers making part of the largest managers' association in Italy
2) the population of all employees from a High Tech firm in Italy: in this second case, we will be able to identify employees, middle managers and managers. To the bulk of employees, we will provide two different treatments: the one aforementioned and another treatment containing general information about the gender gap in Italy. We will be able to identify the effect of both disclosing own implicit stereotypes or to provide information about the general status of the labor market in terms of gender gap.
External Link(s)

Registration Citation

Citation
Spadavecchia, Lorenzo, Paola Antonia Profeta and Maddalena Ronchi. 2022. "Managerial implicit stereotypes and where to find them: Evidence from Incentivized Resume Rating." AEA RCT Registry. May 16. https://doi.org/10.1257/rct.8308
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2022-01-15
Intervention End Date
2022-01-30

Primary Outcomes

Primary Outcomes (end points)
The primary outcomes will be the score provided by respondents to the Incentivized Resume Rating. Through the Incentivized Resume Rating we will ask managers to evaluate 10 explicitly fake CVs, that contain a bunch of information such as gender, GPA, previous education, work experience, experience abroad, languages.
Please find attached the second survey containing the IRR. You will see only one CV, as characteristics inside of it are randomized for 10 iterations, while keeping the shape and presentation of the CV fixed. At the end of each CV, a slider will be used to give an evaluation from 1 to 10.


Managers will be asked to provide an evaluation of each profile on a Likert Scale from 1 to 10. We are interested in understanding how the gender of the profile and manager’s own implicit bias affect the score provided to the profile. Other variables included in the profile and manager’s characteristics will be used as control variables. Our preferred specification will be as follows:

〖Profile Score〗_im= α+βX_i+ρ〖Gender〗_i+ γ〖Treatment〗_m×〖Gender〗_i+δ_m+ε_im

Where X_i are profile characteristics, δ_m are manager’s m fixed effects and γ is our coefficient of interest, identifying the effect of IAT score disclosure on on outcome variable 〖Profile Score〗_im.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
The other outcomes will be given by the IAT score itself, which will be regressed on manager’s individual characteristics.
We will also use firm’s performance variables and the share of women in the firm and in managerial positions as outcome variables, and study whether manager’s bias affect these figures.
Evetually, we also use explicit attitudes and belilefs of managers on the IAT score, showing whether correlation arises between explicit and implicit attitudes. This is ex ante ambiguous and the prior is that the explanatory power of the IAT score in explaining explicit bias is low. This might be due both to Social Desirability Bias, for which managers provide answers that are more acceptable, but also to the fact that managers are trained in answering these type of questions in a non discriminatory way (thanks for example to diversity training they have undertaken during their career).
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The treated grous receive the treatment before the Incentivized Resume Rating (IRR). Managers in the control group will receive the control straight after the IRR.

Please refer to the "hidden" experimental design for more information: excessive details in these section might alter results.
Experimental Design Details
The timing of the Incentivized Resume Rating will be randomized across managers: in particular, Federmanager will be able to tell us whether two or more identifying numbers as described in the “Partner and survey delivery” section belong to the same firm (this is however implausible and very rare as stated by Federmanager). So, for ease the explanation, let us think as the unit of our analysis the firm, which might have one (or more managers) taking part to the survey. Again, we don’t know neither the identity of the manager nor the identity of the firm. So for firm with only one manager responding to the survey, the manager will correspond to the firm itself.

To be noted is that the randomization will be carried out at manager level if we have no firm with multiple answers (more plausible event). Our prior is that we will not have answers from managers belonging to the same firm, as Federmanager has never seen such a case in previous surveys delivered to its subscribers. But if this is not the case for our experiment, we will choose to randomize at the firm level rather than at the manager’s level in order to avoid contamination between managers who receive the early feedback and those who receive the feedback after the IRR.

The treated grous receive the feedback (IAT or general information treatment) before the Incentivized Resume Rating. Managers in the control group will receive the IAT feedback straight after the IRR. We are obliged to reveal to everyone his/her IAT, as the Bocconi Ethics Committee requested us to do so, in order not to annoy the managers in the control group.

There is no decepetion, managers are aware that CVs are fake.

Collaborating with Federmanager, the biggest association of Italian managers, we aim to deliver a first survey in which we will collect managers' Implicit Gender Bias (IAT Gender). The survey will be distributed directly from Federmanager, hence Bocconi researchers will not have access to mail addresses of respondents (but just to anonymized identification numbers), nor any kind of question that would allow any sort of identification of the respondent will be asked (we will ask demographics questions, such as age, gender, education, but will not ask any information about the name of firm they work for).

The data collection of the first survey will last two weeks, starting from end of October.
After ten days (during which the researchers will divide respondents into treatment and control group), the treatment group will receive the communication about own IAT score. After one or three weeks, we will be sending a second (experimental) survey to both treatment and control groups, in which they will be asked to evaluate fake profiles (CVs) of MBA students and will ask to give an evaluation on a Likert Scale for the extent to which they think that profile is "promising to become a good manager".

Outcomes (dependent variables) will be the score (Likert from 1 to 10) of each profile evaluated by respondents. Independent variables will range to the treatment variable (IAT score revealed or not), IAT score, managers' demographics, profile's characteristics (gender, age, field of specialization, ..).

In order to construct realistic CVs of Master's student, we will be collaborating together with Bocconi SDA. MBA students will be asked whether they are interested in participating to this project and, in case, to provide us their CVs. We would scrape their CV and construct fake profiles on basis of true CVs. Characteristics in fake profiles are randomized, as done in Factorial Survey Analysis.

There will be no participant remuneration. The incentive we provide is twofold:
1. we ask managers to contribute to identify the "most promising managers of the future" for a presentation of the Association to Master students. We will contact those students that mostly resemble the preferred fake profiles and ask them whether they would be interested in participating in an event in which Federmanager presents itself.
2. Upon student consent, we will ask managers if they are willing to receive Master students' CVs that mostly resemble their own preferred characteristics.


Randomization Method
The randomization will be done in office by a computer. We will stratify managers by their demographic characteristics and firm’s characteristics. In particular, the characteristics we will take into consideration when stratifying the managers into treatment and control group will be:
- Sector: it is in fact well known in the literature that more discriminatory behaviour might affect some sectors more than others
- Gender of the manager
- Age of the manager
- IAT score

We will divide the sample of respondents in 6 groups, 4 treatment and 2 control groups. This will be done to study whether the revelation of the IAT score (the treatment) has persistent effects over time. The first treatments and control group will hence receive the Incentivized Resume Rating after one week from the revelation (of the IAT or of the general information about the gender gap). The second groups will receive the IRR after three months.
Randomization Unit
The randomization unit will be at the manager's level. As the Association of managers we are collaborating with ensured that respondents never belong to the same firm. Anyway, we will know whether two or more respondents belong to the same company.

If such an event happened, our unit of randomization will be the firm instead of the single manager.

Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
The sample size has a planned number of cluster equal to 3000.
We will deliver the survey to about 30’000 managers, but previous survey run by the Company we are collaborating with (we do not display the name here publicly for experimental reasons) show that usual take up rate if of about 10%.
Each of the six treatment and control goups is estimated to contain about 500 managers.
Sample size: planned number of observations
The planned number of observations is 3000
Sample size (or number of clusters) by treatment arms
500 managers: first treatment (revealing IAT and receive IRR after one week: IAT & Short Run effect)
500 managers: second treatment (informative treatment on gender gap and receive IRR after one week: Information & Short Run effect)
500 manager: first control (no treatment and receive IRR together with first and second treatment groups: Control & Short Run effect)

500 managers: third treatment (revealing IAT and receive IRR after three months: IAT & Long Run effect)
500 managers: fourth treatment (informative treatment on gender gap and receive IRR after three months: Information & Long Run effect)
500 manager: second control (no treatment and receive IRR together with third and fourth treatment groups: Control & Long Run effect)
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
The IRR will contain abut 20 profiles that the respondents will have to evaluate. If all the respondents in the first survey will respond to the second survey containing the IRR, we will receive about 12'000 evaluations of CV. Groups of 500 for treatment and control outnumber the ones usually used in the literature.
Supporting Documents and Materials

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
Bocconi Research Ethics Committee
IRB Approval Date
2021-10-20
IRB Approval Number
SA000360

Post-Trial

Post Trial Information

Study Withdrawal

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials