Do Students Gender-discriminate against Professors? Evidence from a Natural Field Experiment

Last registered on April 22, 2024

Pre-Trial

Trial Information

General Information

Title
Do Students Gender-discriminate against Professors? Evidence from a Natural Field Experiment
RCT ID
AEARCTR-0010208
Initial registration date
October 14, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
October 17, 2022, 5:32 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
April 22, 2024, 7:44 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
Management Development Institute Gurgaon

Other Primary Investigator(s)

PI Affiliation
Ahmedabad University

Additional Trial Information

Status
Completed
Start date
2022-10-17
End date
2023-05-01
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Student evaluation of teaching (SET) scores is one of the most common criterion used by universities to judge instructor quality. Such evaluations are often biased negatively for female instructors. It is more pronounced for junior female instructors, and for the female instructors teaching a technical course, for instance, the ones involving more math. Such discrimination stems mostly from the evaluations by male students and have been tested causally in several developed countries.

Our study proposes to test this hypothesis in a developing country context where the evidence on gender-bias against women instructors in academia is limited. Using a natural field experiment conducted with over 500 students of Principles of Microeconomics class at a large private university in India, we also test a behavioral intervention to mitigate the gender-bias, if any.
External Link(s)

Registration Citation

Citation
Arora, Puneet and Moumita Roy. 2024. "Do Students Gender-discriminate against Professors? Evidence from a Natural Field Experiment." AEA RCT Registry. April 22. https://doi.org/10.1257/rct.10208-1.1
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
We conduct our experiment at the beginning of the bi-semester 2022-2023 (end of October) with students enrolled in the 7 sections of Principles of Microeconomics course. The experiment will be conducted in the first class of the course where students will be taught the same topic by male and female. The experimental design is such that it ensures minimal confounds in estimating the gender-bias.

We create two additional treatment groups testing our intervention to mitigate the gender-bias.
Intervention Start Date
2022-10-21
Intervention End Date
2022-10-26

Primary Outcomes

Primary Outcomes (end points)
1. Test scores
2. Student Evaluation of Teaching (SET) Scores

We stratify the randomization by student gender, and are interested in studying the heterogeneity of our findings with respect to the gender of student
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The experiment will be conducted in the first lecture of the Principles of Microeconomics course, where students will be randomly assigned to four treatment groups, with each treatment sitting in separate classrooms. Due to over 500 enrolled students, we will conduct the experiment in two time slots - with students divided into four groups in each slot. Students will be assigned to attend only one of these two slots. They will attend the lecture, write a short quiz, fill in the SET form and their demographics anonymously.
Experimental Design Details
We propose to conduct our experiment at the beginning of the bi-semester 2022-2023 with students enrolled in the 7 sections of Principles of Microeconomics course. The experiment will be conducted in the first class of the course. We will randomly divide all students into four equal groups, and each of these groups will sit together in their assigned classroom only for that one class. This first class, which will incorporate our intervention, will not be conducted by their regular instructor. The session will be for 1 hour and 15 minutes, where students will attend a recorded video lecture which will introduce them to the early concepts in economics for 20 minutes, students will then take an 8 question multiple choice question quiz for 10 minutes, and will then fill the teaching evaluation and demographics form (information on age, gender, programme of study, major, parental education, household income-bracket, etc.). Quiz, teaching evaluation and demographics collection will happen anonymously over Google Form after the lecture, without asking for subject's name or university id. Students will be articulately informed that as instructors, we are interested in their average understanding of the topic and their average feedback of the instructor, and therefore, all the information being collected is anonymous.

The four groups created in the experiment will include Control- Male-No Info group, where a male instructor will start teaching but will not introduce himself. The treatment groups include T1: Male-With Info group, where a male instructor will introduce himself for two minutes, before starting with the lecture; T2: Female-No info, where a female instructor will start the lecture without introducing herself; and T3: Female-With Info, where a female instructor will introduce herself for two minutes, before starting with the lecture. In order to estimate the causal effect of information provision, we record the lecture by a course instructor, and modulate the voice using a software to create a male and a female version of the same lecture. We do not use the lecture recorded in the original voice. This design aspect ensures that teaching style and effectiveness is same in all four groups.

This implies that the differences between Control and T1 groups' SET scores are attributed to provision of information intervention for males; and between T2 and T3 groups' SET scores are attributed to provision of information intervention for female instructors- and a comparison of these two differences will inform whether information provision is more effective for male instructors or female instructors. Most importantly, the difference in SET scores between Control and T2 groups will indicate whether students discriminate between male and female instructors; and a comparison with difference in SET scores between T1 and T3 groups will indicate whether such discrimination goes down in response to the intervention.

In groups T1 and T3, the information introducing the instructor will include instructor's PhD granting institution (a US university name used to reflect the typical internationally education faculty at this institution), teaching experience, research interests, and professional service engagement. We will use hypothetical names for the instructors (Amit Agarwal for male and Sunita Sharma for female), and the information curated will be such that it reflects the profile of a typical young faculty teaching such courses at the university. The experiment, however, will be conducted like a natural field experiment and students will not know the hypothetical nature of such information. This is to prevent any bias that provision of such information may cause with their regular instructor for the course. Students will, however, be sent a debriefing email at the end of the experiment informing them of the intent of the study, and to request their consent allowing us to use their data in the study.
Randomization Method
Randomization conducted using Stata
Randomization Unit
Individual level randomization, with stratification at gender and section level
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
500+
Sample size: planned number of observations
500+
Sample size (or number of clusters) by treatment arms
125+ observations in each of the four treatment arms
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
With 500 observations and 125 in each group randomized at individual level, we should be able to detect a 0.14 sd effect with 80% statistical power. Prior research (Boring and Philippe, 2021) shows that the bias and treatment effects are expected to be larger than 0.14sd.
IRB

Institutional Review Boards (IRBs)

IRB Name
Ahmedabad University IRB
IRB Approval Date
2021-12-01
IRB Approval Number
N/A

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials