Bias in peer evaluations

Last registered on November 02, 2020
View Trial History

Pre-Trial

Trial Information

General Information

Title
Bias in peer evaluations
RCT ID
AEARCTR-0006697
Initial registration date
November 02, 2020

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
November 02, 2020, 8:46 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
November 02, 2020, 9:35 AM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Country
South Africa
Region
Gauteng

Primary Investigator

Name
Nicky Nicholls
Affiliation
University of Pretoria
Contact Primary Investigator

Other Primary Investigator(s)

Additional Trial Information

Status
On going
Start date
2020-08-01
End date
2021-12-01
Keywords
Behavior, Education, Gender
Additional Keywords
Bias, peer evaluation, discrimination
JEL code(s)
I23, J15, J16
Secondary IDs
Abstract
We study race and gender bias in peer evaluations of partners' relative contributions to a series of university assignments, where partners are randomly assigned.
External Link(s)

Registration Citation

Citation
Nicholls, Nicky. 2020. "Bias in peer evaluations." AEA RCT Registry. November 02. https://doi.org/10.1257/rct.6697-1.1
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
We randomly assign students in an undergraduate university course to partners to complete a series of paired assignments. Partners either share or do not share race and/or gender, and partners are different for each assignment, giving individual level variation in match/mismatch of own and partner race and gender. Following the assignment we ask students to report their own percentage contribution to an assignment (where the residual indicates the implied partner contribution). We compare the reported contributions for randomly assigned same and different race and gender partners to investigate possible race and/or gender bias that might arise in peer evaluations.
Intervention Start Date
2020-09-01
Intervention End Date
2020-11-15

Primary Outcomes

Primary Outcomes (end points)
Student reports of own (versus partner) contribution to assignments
Primary Outcomes (explanation)
(This is gathered as a percentage)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Students complete a series of assignments with different randomly assigned partners, where match of race and gender between the 2 students in the partnership varies both between individuals and within individuals (by assigning different partners for each assignment). We investigate whether student ratings of own contribution varies by race and gender, and by whether the reporters' race and/or gender is the same as or different from that of the partner. We control for the perceived difficulty of the assignment, as well as student grades, preference for working alone or in pairs and aspects of internet access (since the study was conducted in an unusual online semester in 2020 during COVID-19 lockdown).
Experimental Design Details
Randomization Method
Partners were assigned randomly using stata.
Randomization Unit
individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
350-400 individuals
Sample size: planned number of observations
350-400 individuals * 3 assignments = 1050-1200 observations
Sample size (or number of clusters) by treatment arms
number of same and other race; and same and other gender pairs is determined by the random assignment
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
University of Pretoria Economic and Management Sciences Ethics Committee
IRB Approval Date
2019-09-09
IRB Approval Number
EMS157/19 (the original 2019 approval was extended to 2020 by the IRB)riginal approval to 2020)
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials