Are the Referees and Editors in Economics Gender Neutral?
Last registered on June 11, 2018

Pre-Trial

Trial Information
General Information
Title
Are the Referees and Editors in Economics Gender Neutral?
RCT ID
AEARCTR-0003048
Initial registration date
June 07, 2018
Last updated
June 11, 2018 10:59 PM EDT
Location(s)
Primary Investigator
Affiliation
UC Berkeley
Other Primary Investigator(s)
PI Affiliation
University of the Basque Country
PI Affiliation
Università della Svizzera Italiana
PI Affiliation
UC Berkeley
Additional Trial Information
Status
In development
Start date
2018-06-11
End date
2018-09-11
Secondary IDs
Abstract
Do men under-value the scientific contributions of women? We study the effects of gender on the evaluation of economic research using data on submissions to four leading journals, matched to referee recommendations, editorial decisions, and subsequent citations. A customized name-matching algorithm allows us to classify the genders of 97% of all authors and referees with an error rate of under 1%. About one-fifth of submitting authors are female, though rates vary widely across subfields. The fractions of female referees are similar. We begin by examining whether editors are more likely to match a female-authored paper with a female referee, suggesting an awareness of possible gender differences in recommendations. We then address four main sets of questions. First, do male and female referees assess papers differently, and does the gender composition of authors matter for how different referees rate a paper? Second, how reliable are the assessments of male and female referees in predicting future citations, and does this vary with the gender of the authors? Third, how do editors weigh the recommendations of different referees against the information contained in prior publications and other author characteristics, including gender? Fourth, are there gender-related differences in the time that referees take to make a recommendation, or that editors take to reach an initial decision? We compare our findings to the results from a survey of economists, and use the survey results to help interpret the gender gaps in referees' and editors' decisions.
External Link(s)
Registration Citation
Citation
Card, David et al. 2018. "Are the Referees and Editors in Economics Gender Neutral?." AEA RCT Registry. June 11. https://www.socialscienceregistry.org/trials/3048/history/30641
Experimental Details
Interventions
Intervention(s)
This registration is for an observational study analyzing whether editors and referees at 4 leading economics journals are gender-neutral in their recommendations and decisions. We detail how we will analyze the data in a pre-analysis plan, written before we have access to the main data set.
Intervention Start Date
2018-06-11
Intervention End Date
2018-09-11
Primary Outcomes
Primary Outcomes (end points)
Rates of desk rejection and revise and resubmit among submitted papers; length (in days and number of rounds) of the editorial process; rate of positive referee recommendations
Primary Outcomes (explanation)
Details are in the pre-analysis plan
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
The trial is about an observational study, analyzing the role of gender of authors and referees in editorial decisions. The design, which is specified in detail in the analysis plan, builds on Card and DellaVigna (2017). As part of the project, we will also survey economists about how they see the role of gender in editorial decisions; the full test of the survey is in the analysis plan.
Experimental Design Details
Randomization Method
N/A
Randomization Unit
N/A
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
We will analyze about 30,000 papers, of which a bit more than half are not desk rejected.
Sample size: planned number of observations
29,872 papers total, including 15,177 non desk rejected papers.
Sample size (or number of clusters) by treatment arms
There are not treatment arms in this observational study
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
UC Berkeley Committee for Protection of Human Subjects
IRB Approval Date
2018-06-04
IRB Approval Number
2018-04-10955
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
No
Is data collection complete?
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports and Papers
Preliminary Reports
Relevant Papers