Bias in peer evaluations

Last registered on November 02, 2020


Trial Information

General Information

Bias in peer evaluations
Initial registration date
November 02, 2020

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
November 02, 2020, 8:46 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
November 02, 2020, 9:35 AM EST

Last updated is the most recent time when changes to the trial's registration were published.



Primary Investigator

University of Pretoria

Other Primary Investigator(s)

Additional Trial Information

On going
Start date
End date
Secondary IDs
We study race and gender bias in peer evaluations of partners' relative contributions to a series of university assignments, where partners are randomly assigned.
External Link(s)

Registration Citation

Nicholls, Nicky. 2020. "Bias in peer evaluations." AEA RCT Registry. November 02.
Experimental Details


We randomly assign students in an undergraduate university course to partners to complete a series of paired assignments. Partners either share or do not share race and/or gender, and partners are different for each assignment, giving individual level variation in match/mismatch of own and partner race and gender. Following the assignment we ask students to report their own percentage contribution to an assignment (where the residual indicates the implied partner contribution). We compare the reported contributions for randomly assigned same and different race and gender partners to investigate possible race and/or gender bias that might arise in peer evaluations.
Intervention Start Date
Intervention End Date

Primary Outcomes

Primary Outcomes (end points)
Student reports of own (versus partner) contribution to assignments
Primary Outcomes (explanation)
(This is gathered as a percentage)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Students complete a series of assignments with different randomly assigned partners, where match of race and gender between the 2 students in the partnership varies both between individuals and within individuals (by assigning different partners for each assignment). We investigate whether student ratings of own contribution varies by race and gender, and by whether the reporters' race and/or gender is the same as or different from that of the partner. We control for the perceived difficulty of the assignment, as well as student grades, preference for working alone or in pairs and aspects of internet access (since the study was conducted in an unusual online semester in 2020 during COVID-19 lockdown).
Experimental Design Details
Randomization Method
Partners were assigned randomly using stata.
Randomization Unit
Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters
350-400 individuals
Sample size: planned number of observations
350-400 individuals * 3 assignments = 1050-1200 observations
Sample size (or number of clusters) by treatment arms
number of same and other race; and same and other gender pairs is determined by the random assignment
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

Institutional Review Boards (IRBs)

IRB Name
University of Pretoria Economic and Management Sciences Ethics Committee
IRB Approval Date
IRB Approval Number
EMS157/19 (the original 2019 approval was extended to 2020 by the IRB)riginal approval to 2020)


Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information


Is the intervention completed?
Data Collection Complete
Data Publication

Data Publication

Is public data available?

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials