Mitigating Gender Bias

2015-11-23

2016-05-31

User ratings: number of up votes, reputation points earned.
Peer engagement: number of answers and comments from other users.

Examining factors that mitigate gender bias.

We will generate mathematics questions at an advanced undergraduate level, and generate answers to posts by users on the Mathematics StackExchange site (undergraduate research assistants with a background in mathematics will assist with these steps). We will create accounts on StackExchange with male and female usernames, and randomly assign these questions or answers to each type of account. These questions and answers we generate will be evaluated by the pool of users in the StackExchange community. Other users can vote posts up or down, which earns the posting user reputation points, and answer and comment on posts.

Does gender bias inhibit women from entering and succeeding in STEM careers? We will experimentally test for the presence of gender bias in an online mathematics community, MathStackExchange, and determine whether the magnitude of this bias depends on the reputation of a
user. On MathStackExchange, users post mathematics questions or answer others' questions. A user's reputation score rises when others vote up her question/answer or accept her answer; the reputation score decreases when others vote down the posting. We will run a randomized control trial by creating male and female usernames and randomly assigning mathematics questions and answers to each username. We will test for gender bias in evaluating users' posts by comparing the peer evaluation of posts (number of up or down votes received, reputation points earned) and peer engagement with posts (number of answers and comments), by gender. We will first conduct an experiment where gendered accounts with no reputation (new accounts) post questions or answers and compare the resulting reputation scores. After this, we will build up the reputation of the same number of single-gender accounts, switch the username gender on a random subset, and post new questions and answers for each account. Comparison of ratings on the new question/answer at higher reputation levels will determine whether building up a reputation is a successful way to mitigate gender bias. Understanding the dynamics of gender bias and the types of evaluations that are more prone to gender bias are important for designing fair evaluation procedures in the workplace and effective policy initiatives to increase female representation in STEM and other underrepresented fields.

We will implement several treatments.
1) We will generate mathematics questions/answers and randomly assign each question to a new account with a male or female username. The target difficulty of questions will be an advanced undergraduate level.
2) We will build up the reputation score of single gender accounts by randomly answering questions. We will then change the
username of half of the accounts to the opposite gender and post a question/answer.

Posts on MathStackExchange will be made between 7PM- 10PM on weekdays, except Friday. We will set a posting frequency that does not inundate the site, approximately 3-4 posts per day. We will not collect any identifiable private information from the subjects.

We will collect the following data:
1) Up and Down votes on each posted question or answer 24, 72 and 168 hours after the question or answer is posted.
2) Number of comments to the question
3) Number of answers to the question
4) Number of comments to each answer
5) Whether an answer is accepted
Data 2-5 will be collected 168 hours after the question is posted.

Randomization will be done by a computer.

Individual

No

1

280 questions

70 questions or answers per condition.