Mitigating Gender Bias

Last registered on November 22, 2015

Pre-Trial

Trial Information

General Information

Title
Mitigating Gender Bias
RCT ID
AEARCTR-0000950
Initial registration date
November 22, 2015
Last updated
November 22, 2015, 9:55 PM EST

Locations

Primary Investigator

Affiliation
Carnegie Mellon University

Other Primary Investigator(s)

PI Affiliation
University of Pennsylvania

Additional Trial Information

Status
In development
Start date
2015-11-23
End date
2016-11-15
Secondary IDs
Abstract
We aim to study factors that exacerbate or mitigate gender bias.
External Link(s)

Registration Citation

Citation
, and Alex Imas. 2015. "Mitigating Gender Bias." AEA RCT Registry. November 22. https://doi.org/10.1257/rct.950-1.0
Former Citation
, , Alex Imas and Alex Imas. 2015. "Mitigating Gender Bias." AEA RCT Registry. November 22. https://www.socialscienceregistry.org/trials/950/history/6129
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2015-11-23
Intervention End Date
2016-05-31

Primary Outcomes

Primary Outcomes (end points)
User ratings: number of up votes, reputation points earned.
Peer engagement: number of answers and comments from other users.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Examining factors that mitigate gender bias.
Experimental Design Details
We will generate mathematics questions at an advanced undergraduate level, and generate answers to posts by users on the Mathematics StackExchange site (undergraduate research assistants with a background in mathematics will assist with these steps). We will create accounts on StackExchange with male and female usernames, and randomly assign these questions or answers to each type of account. These questions and answers we generate will be evaluated by the pool of users in the StackExchange community. Other users can vote posts up or down, which earns the posting user reputation points, and answer and comment on posts.

Does gender bias inhibit women from entering and succeeding in STEM careers? We will experimentally test for the presence of gender bias in an online mathematics community, MathStackExchange, and determine whether the magnitude of this bias depends on the reputation of a
user. On MathStackExchange, users post mathematics questions or answer others' questions. A user's reputation score rises when others vote up her question/answer or accept her answer; the reputation score decreases when others vote down the posting. We will run a randomized control trial by creating male and female usernames and randomly assigning mathematics questions and answers to each username. We will test for gender bias in evaluating users' posts by comparing the peer evaluation of posts (number of up or down votes received, reputation points earned) and peer engagement with posts (number of answers and comments), by gender. We will first conduct an experiment where gendered accounts with no reputation (new accounts) post questions or answers and compare the resulting reputation scores. After this, we will build up the reputation of the same number of single-gender accounts, switch the username gender on a random subset, and post new questions and answers for each account. Comparison of ratings on the new question/answer at higher reputation levels will determine whether building up a reputation is a successful way to mitigate gender bias. Understanding the dynamics of gender bias and the types of evaluations that are more prone to gender bias are important for designing fair evaluation procedures in the workplace and effective policy initiatives to increase female representation in STEM and other underrepresented fields.

We will implement several treatments.
1) We will generate mathematics questions/answers and randomly assign each question to a new account with a male or female username. The target difficulty of questions will be an advanced undergraduate level.
2) We will build up the reputation score of single gender accounts by randomly answering questions. We will then change the
username of half of the accounts to the opposite gender and post a question/answer.

Posts on MathStackExchange will be made between 7PM- 10PM on weekdays, except Friday. We will set a posting frequency that does not inundate the site, approximately 3-4 posts per day. We will not collect any identifiable private information from the subjects.

We will collect the following data:
1) Up and Down votes on each posted question or answer 24, 72 and 168 hours after the question or answer is posted.
2) Number of comments to the question
3) Number of answers to the question
4) Number of comments to each answer
5) Whether an answer is accepted
Data 2-5 will be collected 168 hours after the question is posted.

Randomization Method
Randomization will be done by a computer.
Randomization Unit
Individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
1
Sample size: planned number of observations
280 questions
Sample size (or number of clusters) by treatment arms
70 questions or answers per condition.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
University of Pennsylvania Institutional Review Board
IRB Approval Date
2015-05-06
IRB Approval Number
822049

Post-Trial

Post Trial Information

Study Withdrawal

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials