Back to History Current Version

Are the Referees and Editors in Economics Gender Neutral?

Last registered on October 26, 2018

Pre-Trial

Trial Information

General Information

Title
Are the Referees and Editors in Economics Gender Neutral?
RCT ID
AEARCTR-0003048
Initial registration date
June 07, 2018

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
June 11, 2018, 10:59 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
October 26, 2018, 8:51 PM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Primary Investigator

Affiliation
UC Berkeley

Other Primary Investigator(s)

PI Affiliation
UC Berkeley
PI Affiliation
Università della Svizzera Italiana
PI Affiliation
University of the Basque Country

Additional Trial Information

Status
Completed
Start date
2018-06-11
End date
2018-09-11
Secondary IDs
Abstract
Do men under-value the scientific contributions of women? We study the effects of gender on the evaluation of economic research using data on submissions to four leading journals, matched to referee recommendations, editorial decisions, and subsequent citations. A customized name-matching algorithm allows us to classify the genders of 97% of all authors and referees with an error rate of under 1%. About one-fifth of submitting authors are female, though rates vary widely across subfields. The fractions of female referees are similar. We begin by examining whether editors are more likely to match a female-authored paper with a female referee, suggesting an awareness of possible gender differences in recommendations. We then address four main sets of questions. First, do male and female referees assess papers differently, and does the gender composition of authors matter for how different referees rate a paper? Second, how reliable are the assessments of male and female referees in predicting future citations, and does this vary with the gender of the authors? Third, how do editors weigh the recommendations of different referees against the information contained in prior publications and other author characteristics, including gender? Fourth, are there gender-related differences in the time that referees take to make a recommendation, or that editors take to reach an initial decision? We compare our findings to the results from a survey of economists, and use the survey results to help interpret the gender gaps in referees' and editors' decisions.
External Link(s)

Registration Citation

Citation
Card, David et al. 2018. "Are the Referees and Editors in Economics Gender Neutral?." AEA RCT Registry. October 26. https://doi.org/10.1257/rct.3048-2.1
Former Citation
Card, David et al. 2018. "Are the Referees and Editors in Economics Gender Neutral?." AEA RCT Registry. October 26. https://www.socialscienceregistry.org/trials/3048/history/197335
Experimental Details

Interventions

Intervention(s)
This registration is for an observational study analyzing whether editors and referees at 4 leading economics journals are gender-neutral in their recommendations and decisions. We detail how we will analyze the data in a pre-analysis plan, written before we have access to the main data set.
Intervention (Hidden)
Intervention Start Date
2018-06-11
Intervention End Date
2018-09-11

Primary Outcomes

Primary Outcomes (end points)
Rates of desk rejection and revise and resubmit among submitted papers; length (in days and number of rounds) of the editorial process; rate of positive referee recommendations
Primary Outcomes (explanation)
Details are in the pre-analysis plan

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The trial is about an observational study, analyzing the role of gender of authors and referees in editorial decisions. The design, which is specified in detail in the analysis plan, builds on Card and DellaVigna (2017). As part of the project, we will also survey economists about how they see the role of gender in editorial decisions; the full test of the survey is in the analysis plan.
Experimental Design Details
Randomization Method
N/A
Randomization Unit
N/A
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
We will analyze about 30,000 papers, of which a bit more than half are not desk rejected.
Sample size: planned number of observations
29,872 papers total, including 15,177 non desk rejected papers.
Sample size (or number of clusters) by treatment arms
There are not treatment arms in this observational study
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
UC Berkeley Committee for Protection of Human Subjects
IRB Approval Date
2018-06-04
IRB Approval Number
2018-04-10955
Analysis Plan

Analysis Plan Documents

EditorGenderAnalysisPlan

MD5: 5f4d8c8ff7dbbed13ae21784f6082fdf

SHA1: c2ac217b3582e33d6bf65e4775b1cc53d99e44d7

Uploaded At: June 07, 2018

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Yes
Data Collection Completion Date
Final Sample Size: Number of Clusters (Unit of Randomization)
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
Final Sample Size (or Number of Clusters) by Treatment Arms
Reports, Papers & Other Materials

Relevant Paper(s)

Abstract
We study the role of gender in the evaluation of economic research using submissions to four leading journals. We find that referee gender has no effect on the relative assessment of female- versus male-authored papers, suggesting that any differential biases of male referees are negligible. To determine whether referees as a whole impose different standards for female authors, we compare citations for female- and male-authored papers, holding constant referee evaluations and other characteristics. We find that female-authored papers receive about 25% more citations than observably similar male-authored papers. Editors largely follow the referees, resulting in a 1.7 percentage point lower probability of a revise and resubmit verdict for papers with female authors relative to a citation-maximizing benchmark. In their desk rejection decisions, editors treat female authors more favorably, though they still impose a higher bar than would be implied by citation maximization. We find no differences in the informativeness of female versus male referees or in the weight that editors place on the recommendations of female versus male referees. We also find no differences in editorial delays for female- versus male-authored papers.
Citation
"Are Referees and Editors in Economics Gender Neutral?" (with David Card, Patricia Funk, Nagore Irriberri), Quarterly Journal of Economics, February 2020, Vol. 135, 269-327

Reports & Other Materials