| Field | Before | After |
|---|---|---|
| Field Abstract | Before The aim of affirmative action (AA) policies is to increase the representation of minorities in candidate pools for hiring and/or promotions. In this study, we plan to use the controlled setting of a lab experiment to find evidence and understand the true size and nature of the spillover effects of a soft AA policy on employer discrimination. It allows us to determine 1) whether this effect is predominately positive or negative, and 2) whether it is primarily driven by behavioural preferences (taste-based discrimination) or rational choice (statistical discrimination). We do this by separating hiring decisions from output estimation decisions, and by comparing AA policies for an ethnic minority group with for a random “priority” group that has no distinct characteristics. Our findings aim to provide evidence and insights into the mechanisms of spillover effects of soft AA policies in the labour market. | After Affirmative action (AA) policies are used to increase the representation of minorities in candidate pools for hiring and/or promotions. In this study, we plan to use the controlled setting of a lab experiment to understand the true size and nature of the spillover effect of a soft AA (SAA) policy on employer’s discrimination. It allows us to determine 1) whether this effect is predominately positive or negative, and 2) if it is primarily driven by behavioural preferences (i.e., taste-based discrimination) or rational choices (i.e., statistical discrimination). We do this by comparing a soft AA policy based on minority ethnicity status with one for a random “priority” group that has no distinct characteristics, and also by separating hiring decisions from performance estimations. Our findings would provide insights into the mechanisms of the spillover effects of soft AA policies in the labour market. |
| Field Trial Start Date | Before January 17, 2022 | After September 26, 2022 |
| Field Trial End Date | Before December 31, 2022 | After October 31, 2022 |
| Field Last Published | Before May 11, 2022 01:02 AM | After September 25, 2022 07:54 PM |
| Field Intervention Start Date | Before January 18, 2022 | After September 30, 2022 |
| Field Intervention End Date | Before December 01, 2022 | After October 31, 2022 |
| Field Primary Outcomes (End Points) | Before Overall effect: The main variable of interest is how a soft affirmative action (soft AA) policy impacts the percentage of candidates being hired of the type that is the target of the policy. | After Overall Effect: The main outcome of interest is how a soft affirmative action (SAA) policy impacts the percentage of candidates being hired of the type that is the target of the policy. |
| Field Primary Outcomes (Explanation) | Before We will compare the percentage (%) of hired candidates that are minorities in the Baseline Type (2) and Soft AA minority (3) interventions. Similarly, we will compare the percentage of hired candidates that are “lucky” in the Baseline (1) and Soft AA Lucky (4) treatments. The difference in these treatment effects (the percent of hired that are minority in (3) vs. (2) vs. the percentage of the hired that are “lucky” in (4) vs (1)) will identify the role that minority status plays in the impact of a Soft AA policy. | After We will compare the percent of hired candidates that are minorities in Treatment 2 (Baseline_Minor) and Treatment 4 (SAA_Minor). Similarly, we will compare the percent of hired candidates that are “lucky” in Treatment 1 (Baseline_Colour) and Treatment 3 (SAA_Colour). The difference in treatment effects (the difference between the percent of hired candidates that are minorities in Treatment 4 vs. Treatment 2 and the percent of hired that are “lucky” in Treatment 3 vs Treatment 1) will identify the role that minority status plays in the impact of a SAA policy. |
| Field Experimental Design (Public) | Before In this experiment, we have two phases: 1) preliminary phase, in which we aim to recruit 100 participants to complete a series of tasks. 2) The secondary phase, in which we aim to recruit 100 participants for every four treatments to complete a hiring game with a hiring decision and 4 estimating decisions. The preliminary phase is designed to generate actual profiles of candidates for use in the second phase of the hiring game. The benefit of using actual profiles is to introduce actual costs for discriminatory behaviour and therefore capture the actual level of employer discrimination (Hedegaard & Tyran, 2018). During this phase, we are going to ask participants to finish an individual experiment, including five individual tasks with 2 minutes each. The second phase is a hiring game, in which we will introduce four different treatments, a soft AA policy for an ethnic minority group, a soft AA policy for a randomly selected group, and baselines both with and without information about ethnicity. In the second part of a hiring game, we will only recruit the majority as our participants. Prior to making a hiring decision, all profiles will go through a “pre-screen process" in which the computer will randomly select one of the five tasks completed by the individuals of profiles during the preliminary phase and rank all 12 profiles. Only 4 profiles will be selected as candidates during the hiring decision. During the hiring decision, participants will receive the profiles of four candidates, including scores of another drawn task (different from task used in “pre-screen process"), and age. The pre-screen process and the given information in the profiles vary treatments by treatment (see interventions). | After This experiment consists of two phases: 1) the preliminary phase, in which we aim to recruit 150 participants to complete a series of anagram tasks; and 2) the main phase, in which we aim to recruit 100 participants for each of the four treatments to complete either hiring or performance estimation tasks (i.e., 800 participants in total). In the preliminary phase, participants will be asked to complete five 2-minute anagram tasks individually and paid by piece-rate performance. This phase is designed to generate actual profiles of candidates to be used in the main phase of the experiment. The benefit of using actual profiles is to introduce real consequences for discriminatory behaviour and therefore capture the actual level of employer discrimination (Hedegaard & Tyran, 2018). To construct a balanced candidates pool for the second phase, 75 participants will be recruited from an ethnic minority group (i.e., East Asians) and the remaining 75 will be recruited from the ethnic majority group (i.e., Whites). Out of the five performances, we drop the lowest and the highest scores to form the final candidate profiles. For each candidate profile, we randomly name each of the three scores remaining as the pre-screening score, interview score and employment score. In the main phase of the experiment, participants will be asked to either complete some hiring decisions or make performance estimations, given sets of pre-screened candidate profiles drawn from the data collected in the preliminary phase. Each participant in this phase will be assigned to one of the following four experimental treatments: a baseline_colour treatment with randomly assigned colours (red or green) to each candidate, but with neither the soft AA policy nor the information about candidates’ ethnicities (Treatment 1), a baseline-minority treatment without soft AA policy but with revealed information on candidates’ ethnicities (Treatment 2), a baseline_colour intervention treatment with soft AA policy for a randomly selected “lucky” group, namely the candidates assigned to red colour (Treatment 3), and a baseline-minority intervention Treatment with soft AA policy for the ethnic minority group (Treatment 4). Note that we will only recruit participants from the ethnic majority group as “employers/managers”. Each set of the four candidates’ profiles is pre-screened from 12 randomly selected participants (with similar average performances from the preliminary phase) through a treatment-specific “pre-screen process” based on the candidates’ pre-screening performances. The employer/manager is given the interview performances of all four candidates and is asked to either select one candidate to be hired (in the hiring task condition) or to give estimates of the actual employment performances for all four candidates (in the estimation task condition). On top of interview performances, the randomly assigned group colour (in Treatment 1 and 3) or ethnicity information (reflected through the surnames in Treatment 2 and 4), the potential employer/manager will also be given candidates’ age, prolific id, priority status (in Treatment 3 and 4). |
| Field Planned Number of Clusters | Before 100 individual participants per treatment (500 in total, 400 for the main experiment) | After 150 participants in the preliminary phase and 100 participants per Treatment per task in the second phase (950 in total, 800 for the main experiment in phase 2) |
| Field Planned Number of Observations | Before 12 profiles per session. 100 sessions per treatment. Total observations are 12*100*4 = 4800. | After 4*100*4 = 1600 performance estimation decisions and 100*4=400 hiring decisions. |
| Field Sample size (or number of clusters) by treatment arms | Before 100 sessions per treatment (total are 400 sessions) | After 200 participants in the preliminary phase, and 100 participants per Treatment per task (total are 800 clusters) in the main experiment. |
| Field Keyword(s) | Before Behavior, Electoral, Firms And Productivity, Lab, Labor | After Behavior, Firms And Productivity, Lab, Labor |
| Field Intervention (Hidden) | Before - Baseline (1): The profiles of the top 4 best performers will enter the candidate pool for all participants to choose from. During the selection, the ethnic type will not be explicitly informed. - Baseline Type (2): The profiles of the top 4 best performers will enter the candidate pool for all participants to choose from. During the selection, the ethnic type will be explicitly informed. - Soft AA minority (3): The profiles of the top 2 best-performing minorities and the top 2 best-performing non-selected remainders will enter the candidate pool. During the selection, the ethnic type will be explicitly informed. - Soft AA Lucky (4): Half of the profiles will be assigned as “lucky", and half will be assigned as “unlucky". The profile of the top 2 best performing individuals from the “lucky" group and the top 2 best-performing non-selected remainders will enter the candidate pool. During the selection, the ethnic type will not be explicitly informed. | After |
| Field Secondary Outcomes (End Points) | Before 1) Exposure Effect: How the soft AA policy increases the number of candidates of the targeted type in the candidate pool. 2) Signal Effect: How the soft AA policy impacts the estimation of scores of the targeted type as compared to the untargeted type. 3) Fairness Effect: How does the soft AA policy impact the willingness to hire targeted groups vs. non-targeted groups controlling for expected scores. 4) Token Effect: How the soft AA policy impacts the likelihood that a member of the targeted group is hired given they are selected from the candidate pool to be interviewed. Hypothesis (estimated outcomes): 1) Hypothesis 1: Behaviour story (Large unfairness impact – hired less but estimated same). - In the hiring decision: The negative spillover effect is much larger in the soft AA policy minority(3) than in the soft AA policy lucky(4). - In the estimation decision: Expect no difference in soft AA policy minority(3) and in the soft AA policy lucky(4). 2) Hypothesis 2: Rational story (Small unfairness impact – hired more or indifferent but estimated less) within Soft AA minority(3). - In the hiring decision: Exposure and frequency effects (positive spillover) dominate the signal effects (negative spillover). - In the estimation decision: Signal effects (negative spillover) dominate the exposure effects (positive spillover). | After The primary outcome (overall effect) can be decomposed to a Frequency Effect (which is mechanically generated by the SAA policy) and a Token effect (which is potentially confounded by several factors). The Frequency Effect measures how the SAA policy increases the number of candidates of the targeted type in the candidate pool, whereas the Token Effect1 measures how the SAA policy impacts the likelihood that a member of the targeted group is hired given that they are selected by the pre-screening process. Our design aims to further disentangle the confounding factors of the Token Effect. |
| Field Secondary Outcomes (Explanation) | Before 1) Exposure effects: - Hiring: The percent of advantaged candidates in each treatment. We expect the percent of minority candidates is higher in the AA minority treatments than in the Baseline type treatments while the percent of lucky candidates is higher in the AA lucky treatments than in the Baseline treatments. 2) Singal effects: - The difference in average scores of Task A between advantaged groups and disadvantaged groups. We expect the difference should be positive and significant in AA minority treatments (majority-minority) and in AA lucky treatments (unlucky - lucky) while the difference should be close to zero in Baseline treatments and Baseline type treatments. - The difference in estimated scores between the disadvantaged group and advantaged group should be significantly greater in the AA minority and AA lucky treatments. - The contribution of Task B's scores to the estimated scores should be smaller in AA minority and AA lucky treatments. This is because TaskB's score is less effective in the context of Affirmative action policy. 3) Fairness effects: - The answer to the first question in the post-experimental survey. We expect participants will perceive the pre-screen is less fair in AA minority treatments and in AA lucky treatments than in Baseline treatments and Baseline type treatments. - We will compare the percent of candidates hired that are minorities in the Baseline Type (2) and Soft AA minority (3) interventions. Similarly, we will compare the percent of candidates hired that are “lucky” in the Baseline (1) and Soft AA Lucky (4) treatments. We expect the percent of minority hired is higher in the AA minority treatments than in the Baseline type treatments while the percent of lucky hired is higher in the AA lucky treatments than in the Baseline treatments, if fairness effects is significant. 4) Token Effects: We will compare the percentage of minority candidates who are selected to be interviewed by the pre-screen process that are hired in the Baseline Type (2) and Soft AA minority (3) interventions. Similarly, we will compare the percentage of “lucky” candidates who are selected to be interviewed by the pre-screen process that are hired in the Baseline (1) and Soft AA lucky (4) interventions. | After We measure both the base-level belief bias (i.e., Stereotype Effect) and preference bias (i.e., Taste-base Discrimination Effect). We also measure the two effects that impact the Token Effect: the Backfire Effect and the Perception Effect. Our definitions of these effects are the following: (1) Stereotype Effect (or Base-level belief bias): Whether and how much the managers perceive the minority group having lower expected employment performance due to existing stereotypical beliefs. (2)Taste-base Discrimination Effect (or Base-level preference bias): Whether and how much the managers (who are majorities) are less willing to hire minority candidates given the same expected employment performance. (3) Perception Effect (Policy induced belief bias): How the SAA policy impacts the managers’ beliefs about the expected performance of the targeted group vs. non-targeted group through changes in the pre-selection process and through additional exposure to candidates’ performances from the targeted group. (4) Backfire Effect (Policy induced preference bias): How the SAA policy impacts the managers’ willingness to hire targeted groups vs. non-targeted groups controlling for expected (employment performance) scores. |
| Field | Before | After |
|---|---|---|
| Field Document | Before |
After
Analysis plan for pre-registration.docx
MD5:
16b187a562b66142582bbd118776df8b
SHA1:
d33d0adb8ab5d53279aee4d4529655698882a550
|
| Field Title | Before | After Analysis plan |
| Field | Value |
|---|---|
| Field Document |
Value
Analysis+plan+.docx
MD5:
105468d440fe0e02aff4b0946aa0eab7
SHA1:
2be53f3459776823edda61b1806ecac79d95c37b
|
| Field Title | Value Analysis plan |