Improving Women's Career Progression in Academia: The Impact of an Anti-Bias Intervention on Student Teaching Evaluations

Last registered on May 13, 2024

Pre-Trial

Trial Information

General Information

Title
Improving Women's Career Progression in Academia: The Impact of an Anti-Bias Intervention on Student Teaching Evaluations
RCT ID
AEARCTR-0013491
Initial registration date
April 26, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
May 13, 2024, 12:51 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
University of Girona

Other Primary Investigator(s)

PI Affiliation
Universitat Autònoma de Barcelona

Additional Trial Information

Status
In development
Start date
2024-04-29
End date
2024-07-03
Secondary IDs
CEBRU0002-24
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Women continue to be underrepresented in academia, as well as in the majority of STEM fields, despite ongoing efforts to narrow the gender gap across various industries, professions, and occupations. According to SheFigures (European Commission, 2021), women constitute only 33% of all researchers in Europe. This disparity is further highlighted by an AAUW report (American Association of University Women, 2023), which reveals that women predominantly occupy non-tenure-track lecturer and instructor roles across institutions, while also remaining underrepresented in top academic positions.

The gender gap in academia can be partly attributed to the gender bias in teaching evaluations: there is ample evidence in the literature that Student Evaluation of Teaching (SET) results tend to favour male professors over female professors (Boring, 2016; Genetin et al., 2021; Ayllón, 2022). Given the detrimental effect that negative teaching evaluations can have on the progression of women in academia, there has been a growing body of research exploring the effectiveness of field interventions that can mitigate gender biases. Such interventions, often using randomization techniques, have effectively raised awareness among students regarding the existence of gender biases and have partly corrected them. Previous treatments have primarily involved emails or short scripts (Peterson, 2019; Boring and Philippe, 2020; Genetin et.al., 2021; Kogan et al., 2022).

This study conducts a field experiment at the University of Girona (UdG) with the objective of preventing gender-biased behaviours that can arise when students evaluate their professors. To define the treatment and control groups, we conduct a randomized controlled trial at the individual level stratified by gender. Our experimental design consists of sending treated students an email with a link to a video that they shall watch before answering the teaching questionnaire. We produced two different videos and therefore the design considers two treatment arms (Treatment 1 and Treatment 2). The first video contains an informative message about implicit biases and highlights their unintentional and unconscious nature. It ends by asking students to avoid prejudices or discriminatory behaviour. The second video contains a similar message but refers more directly to gender biases and also informs that the related literature has documented that male students have been found to be more biased against females professors than female students. The control group does not receive any treatment video. By relying on videos, we take advantage of the effectiveness of audio-visual tools.

Our research is expected to contribute to discussions surrounding gender bias in academia and to deliver valuable information to administrators in order to implement changes that improve the situation of women in academia. Specifically, we aim to promote fairness and equity in academic evaluations, enhance diversity and inclusion in academia, increase awareness and advocacy for gender equality, and generate a positive impact on career advancement opportunities for women. Additionally, we aim to propose novel strategies for developing more effective anti-bias interventions. Furthermore, we expect to demonstrate that anti-bias interventions can enhance students' awareness and willingness to change biased behaviour. This, in turn, will enable professors to receive more equitable evaluations based on the quality of their performance, without account of other factors.
External Link(s)

Registration Citation

Citation
Ayllon, Sara and Camila Zamora. 2024. "Improving Women's Career Progression in Academia: The Impact of an Anti-Bias Intervention on Student Teaching Evaluations." AEA RCT Registry. May 13. https://doi.org/10.1257/rct.13491-1.0
Experimental Details

Interventions

Intervention(s)
This study conducts a field experiment at the University of Girona (UdG) with the objective of preventing gender-biased behaviours that can arise when students evaluate their professors. To define the treatment and control groups, we conduct a randomized controlled trial at the individual level stratified by gender. Our treatment consists of sending by email a link to a video showing them anti-bias messages before they complete the teaching questionnaire. The intervention has two treatment arms: (1) video 1 contains an informative message about implicit biases and highlights their unintentional and unconscious nature. It ends by asking students to avoid prejudices or discriminatory behaviour; (2) video 2 contains a similar message but refers more directly to gender biases and also informs that the related literature has documented that male students have been found to be more biased against females professors than female students. The control group does not receive any treatment video. By relying on videos, we take advantage of the effectiveness of audio-visual tools.
Intervention Start Date
2024-04-29
Intervention End Date
2024-07-03

Primary Outcomes

Primary Outcomes (end points)
The main outcome variable is the answer to the teaching evaluation questionnaire that provides an overall assessment of each lecturer for every course taught. That is, from 1 to 5, students need to state the extent to which they mostly disagree (value 1) or mostly agree (value 5) with the statement: "Overall, I evaluate positively the teaching of this professor". Two other questions in the questionnaire will be considered: (1) one that refers to the teaching method of the lecturer; and (2) one that evaluates the support activities planned by the lecturer and his/her office hours.
Primary Outcomes (explanation)
Outcomes will not be constructed. They will be obtained directly from the students answers of the teaching questionnaire.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Our experimental design consists of sending University students an email with a link to a video that they shall watch before answering the teaching questionnaire. We produced two different videos and therefore the design considers two different treatment arms.
Treatment 1: The first video contains an informative message about implicit biases and highlights their potential unintentional and unconscious nature.
Treatment 2: The second video delivers a more direct message, introducing more directly that previous literature has found gender biases against female professors and how such biases often have been found to proceed from male students.
Control: The control group does not receive any treatment video.
By relying on videos, we take advantage of the effectiveness of audio-visual tools in the target group of this study (young University students).
Experimental Design Details
Randomization Method
Randomization at the student individual level will be done in office by a computer using the software package STATA 17.0.
Randomization Unit
Individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
0
Sample size: planned number of observations
Note that answering the questionnaire is voluntary at the University of Girona and therefore the final number of observations in the study (and in each treatment arm) is at the time of RCT registry unknown. However, if the response rate is similar than in other semesters, the total number of observations shall be around 20,000 responses from about 5,000 to 6,000 students.
Sample size (or number of clusters) by treatment arms
All students registered at the University of Girona in the second semester of the academic year 2023/24 who have taken a regular course and therefore are invited to answer the teaching questionnaire will be assigned to Treatment 1, Treatment 2 or the Control group. Each group will be one third of the total. Note that answering the questionnaire is voluntary at the University of Girona and therefore the final number of observations in the study (and in each treatment arm) is at the time of RCT registry unknown.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Comitè d'Ètica i Bioseguretat de la Recerca de la Universitat de Girona
IRB Approval Date
2024-03-18
IRB Approval Number
CEBRU0002-24
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials