Attitudes towards hiring decisions

Last registered on December 11, 2023

Pre-Trial

Trial Information

General Information

Title
Attitudes towards hiring decisions
RCT ID
AEARCTR-0005064
Initial registration date
May 11, 2020

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
May 13, 2020, 3:46 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
December 11, 2023, 4:02 PM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
Victoria University of Wellington

Other Primary Investigator(s)

PI Affiliation
Victoria University of Wellington
PI Affiliation
Victoria University of Wellington

Additional Trial Information

Status
Completed
Start date
2020-05-18
End date
2020-12-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This paper investigates attitudes towards hiring decisions.
External Link(s)

Registration Citation

Citation
Feess, Eberhard, Jan Feld and Shakked Noy. 2023. "Attitudes towards hiring decisions." AEA RCT Registry. December 11. https://doi.org/10.1257/rct.5064-4.0
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2020-05-18
Intervention End Date
2020-06-30

Primary Outcomes

Primary Outcomes (end points)
1. “pro-women attitude” in base scenario.
2. “pro-woman attitude” after reflection.
3. Effect of holding discrimination constant on pro-women attitudes (separately for “pro-women”, “neutral” and “pro-men” respondents)
4. Effect of holding effort constant on pro-women attitudes (separately for “pro-women”, “neutral” and “pro-men” respondents)
5. Effect of holding suffering constant on pro-women attitudes (separately for “pro-women”, “neutral” and “pro-men” respondents)
6. Effect of holding discrimination, effort and suffering constant on pro-women attitudes. (separately for “pro-women”, “neutral” and “pro-men” respondents)
Primary Outcomes (explanation)
Terminology: We measure “pro-women attitude” as “attitude towards discriminating against a man” minus “the attitude towards discrimination against a woman”. For example, a respondent who finds discrimination against a man neutral (50, where 0 is very morally wrong and 100 is very morally right) and discrimination against a woman is somewhat morally bad (40) has a pro-women attitude of 10. Negative pro-women attitudes reflect that respondents find discrimination against men more morally objectionable than discrimination against women. As shorthand, we will refer to respondents’ as “pro-women” if they have a pro-women attitude >1, “neutral” if they have a pro-women attitude of -1, 0 or 1, and “pro-men” if pro-women attitude <-1.

Primary outcome 1: Average pro-women attitude in the sample.

Primary outcome 2: We measure explicit attitudes toward gender discrimination as respondents’ answers to questions following up on answering these two scenarios. In this question, respondents can indicate whether they find 1) discrimination against women more morally objectionable, 2) discrimination against men more morally objectionable, or 3) both equally morally objectionable. A random subset of 50% of respondents are shown this question. Our second primary outcome is the percentage of respondents who choose each option.

Background primary outcomes 3-6 are based on an experiment embedded in the survey. In this experiment, all respondents see four additional pairs of scenarios. A scenario pair shows two scenarios that are identical expect for that in one a women is discriminated against and in the other a man is discriminated against. In the control group, respondents see four scenario pairs that are similar to the base scenarios, except that they are describing the geographical location of the job (urban, suburban, rural, major city). In the treatment groups, respondents see scenarios in the same geographical locations with four additional information treatments, that is, texts that hold potential reasons for judging discrimination differently constant. The information treatment for primary outcomes 3-5 state that “ The job is in an industry where there is no gender discrimination”, “The man and the woman have worked equally hard in their career”, and “The man and the woman would suffer equally much from not getting the job.” The information treatment for primary outcome 6 combines the previous three information treatments.

To illustrate how this experiment leads to causal estimates, consider the following example. Respondents in the control group see a first pair of scenarios in which they are asked to judge discrimination against men and women for jobs in urban areas. Respondents in one of the treatment groups are asked to judge discrimination against men and women for jobs in urban areas (same as control group) in an industry without gender discrimination. This design allows us to compare evaluations for scenarios that are identical except for the information treatment. To estimate the effect of the information treatment “no discrimination”, we can take the average pro-women bias in the control group minus the average pro-women bias in this treatment group.

Primary outcome 3: Differences in average pro-women attitude in control scenarios and average pro-women attitude in “no-discrimination” information scenarios. We will show these differences separately for respondents we classified as pro-women, neutral, and pro-men; based on their answers in the base scenario.

Primary outcome 4: Differences in average pro-women attitude in control scenarios and average pro-women attitude in “same effort” information scenarios. We will show these differences separately for respondents we classified as pro-women, neutral, and pro-men; based on their answers in the base scenario.

Primary outcome 5: Differences in average pro-women attitude in control scenarios and average pro-women attitude in “same suffering” information scenarios. We will show these differences separately for respondents we classified as pro-women, neutral, and pro-men; based on their answers in the base scenario.

Primary outcome 6: Differences in average pro-women attitude in control scenarios and average pro-women attitude in “no-discrimination, same effort, and same suffering” information scenarios. We will show these differences separately for respondents we classified as pro-women, neutral, and pro-men; based on their answers in the base scenario.

Secondary Outcomes

Secondary Outcomes (end points)
1. Average pro-women attitude (between sample)
2. Average pro-women attitude by gender
3. Average pro-women attitude by education
4. Average pro-women attitude by income
5. Average pro-women attitude by political leanings
6. Relationship between pro-women attitude in base and belief that women suffered more in hypothetical scenarios.
7. Relationship between pro-women attitude in base and belief that women worked harder in their career in hypothetical scenarios.
8. Relationship between pro-women attitude in base and belief that women worked harder in general in hypothetical scenarios.
9. Relationship between pro-women attitude in base and belief that women are more discriminated against in hypothetical scenarios.
Secondary Outcomes (explanation)
Secondary outcome 1. One concern is that respondents might evaluate the first scenario following their intuition and then answer the second scenario with the goal of wanting to be consistent with their first answer. Randomly selected 50% respondents first see a scenario in which a women is discriminated against whereas the remaining 50% of respondents first see a scenario in which a man is discriminated against. Secondary outcome 2 is the average pro-women attitude based on respondents’ answers to the first scenario. This between-person design allows us to estimate pro-women attitude which is not driven by concerns about wanting to be consistent.

Secondary outcomes 2-5 are pro-women attitudes, separated by women and men (2), the following five categories of education: high school, some college, 2-year college, 4-year college, professional degree or postgrad (3), below median income and above median income (4), and democrats, independents and republicans (5).

Secondary outcomes 6-9 are based on questions asked to the control group after the survey experiment. The questions are about the hypothetical scenarios.

For secondary outcome 6, respondents are asked: “In the previous scenarios, which of the candidates do you think would have suffered more from not getting the job?” The answer options range from 0 (“The women would have suffered much more”) over 50 (“Both would suffered equally much”) to 100 (“The men would have suffered much more”). We transform each respondent’s answer into a “women suffered more” score where positive values reflect that women would have suffered more, 0 reflects both would have suffered equally and negative values reflect that men would have suffered more. Secondary outcome 6 shows the average women suffered more score for respondents we classified as pro-women, neutral pro-men based on their answers in the base scenario.

The procedure for estimating secondary outcomes 7-9 is analogous to the procedure outlined for secondary outcome 6.

Secondary outcome 7 is based on this question: Who do you think would have worked harder to get where they are in their career?

Secondary outcome 8 is based on this question: Who do you think is generally more hard-working (in their career and other aspects of their life)?

Secondary outcome 9 is based on this question: Who do you think would be more discriminated against in the labor market?

Experimental Design

Experimental Design
Nothing to report yet.
Experimental Design Details
The questionnaire is structured as follows:
1. Demographic questions
2. Base scenarios.
3. Reflection question (only for randomly selected 50% of respondents).
4. Survey experiment (see description above)
5. Questions about beliefs about hypothetical scenarios (control group of survey experiment only)
6. Questions about beliefs about US labor market in general
7. Demographic and attitude questions.

See the full questionnaire in Appendix A.

***Update July 29, 2020***

We have decided to collect around 1,300 additional observations to get more precise estimates primary outcomes 3-6 (those from the survey experiment). To that end, we will post a shortened version of the questionnaire on Amazon Mechanical Turk.
Randomization Method
Randomization will be carried out in the Qualtrics Survey by a computer.
Randomization Unit
Respondents.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
450 respondents

***Update July 29, 2020***
additional 1,300 respondents only for the survey experiment
Sample size: planned number of observations
For primary outcomes 1-2: 450 observations (1 observation per respondent) For primary outcomes 3-6: 1,800 observations (1 per respondent-scenario combination - each respondent sees 4 scenarios) ***Update July 29, 2020*** For primary outcomes 3-6, we will collect an additional 5,200 observations (1 per respondent-scenario combination)
Sample size (or number of clusters) by treatment arms
For primary outcomes 1 and 2:
Treatment arm 1: approx. 225 respondents (50%) will first see the scenario in which a woman is discriminated against
Treatment arm 2: approx. 225 respondents (50%) will first see the scenario in which a man is discriminated against
For primary outcomes 3-6:
• Control group: approx. 225 respondents (50%)
For ease of exposition, we will number the information treatments as same suffering (treatment 1), worked equally hard (2), no discrimination (3), same suffering, worked equally hard and no discrimination (4).
• Treatment group 1: order of information treatments (1,2,3,4), approx. 37.5 respondents (8.3%)
• Treatment group 2: order (1,3,2,4), approx. 37.5 respondents (8.3%)
• Treatment group 3: order (2,1,3,4), approx. 37.5 respondents (8.3%)
• Treatment group 4: order (2,3,1,4), approx. 37.5 respondents (8.3%)
• Treatment group 5: order (3,1,2,4), approx. 37.5 respondents (8.3%)
• Treatment group 6: order (3,2,1,4), approx. 37.5 respondents (8.3%)

***Update July 29, 2020***
For primary outcomes 3-6, the additional respondents will be assigned to treatment groups in the same proportions as stated above.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
Victoria University Human Ethics Committee
IRB Approval Date
2020-05-10
IRB Approval Number
0000028526

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
August 22, 2020, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
August 22, 2020, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
Was attrition correlated with treatment status?
Final Sample Size: Total Number of Observations
Final Sample Size (or Number of Clusters) by Treatment Arms
Data Publication

Data Publication

Is public data available?
Yes
Public Data URL

Program Files

Program Files
Yes
Program Files URL
Reports, Papers & Other Materials

Relevant Paper(s)

Abstract
Previous research has shown that people care less about men than about women who are left behind. We show that this finding extends to the domain of labor market discrimination: In identical scenarios, people judge discrimination against women more morally bad than discrimination against men. This result holds in a representative sample of the US population and in a larger but not representative sample of Amazon Mechanical Turk (Mturk) respondents. We test if this gender gap is driven by statistical fairness discrimination, a process in which people use the gender of the victim to draw inferences about other characteristics which matter for their fairness judgments. We test this explanation with a survey experiment in which we explicitly hold information about the victim of discrimination constant. Our results provide only mixed support for the statistical fairness discrimination explanation. In our representative sample, we see no meaningful or significant effect of the information treatments. By contrast, in our Mturk sample, we see that providing additional information partly reduces the effect of the victim’s gender on judgment of the discriminator. While people may engage in statistical fairness discrimination, this process is unlikely to be an exhaustive explanation for why discrimination against women is judged as worse
Citation
Feess, Eberhard, Jan Feld, and Shakked Noy. "People Judge Discrimination Against Women More Harshly Than Discrimination Against Men–Does Statistical Fairness Discrimination Explain Why?." Frontiers in psychology 12 (2021): 675776.

Reports & Other Materials