Are the Referees and Editors in Economics Gender Neutral?

Last registered on October 26, 2018

Pre-Trial

Trial Information

General Information

Title
Are the Referees and Editors in Economics Gender Neutral?
RCT ID
AEARCTR-0003048
Initial registration date
June 07, 2018
Last updated
October 26, 2018, 8:51 PM EDT

Locations

Primary Investigator

Affiliation
UC Berkeley

Other Primary Investigator(s)

PI Affiliation
University of the Basque Country
PI Affiliation
Università della Svizzera Italiana
PI Affiliation
UC Berkeley

Additional Trial Information

Status
Completed
Start date
2018-06-11
End date
2018-09-11
Secondary IDs
Abstract
Do men under-value the scientific contributions of women? We study the effects of gender on the evaluation of economic research using data on submissions to four leading journals, matched to referee recommendations, editorial decisions, and subsequent citations. A customized name-matching algorithm allows us to classify the genders of 97% of all authors and referees with an error rate of under 1%. About one-fifth of submitting authors are female, though rates vary widely across subfields. The fractions of female referees are similar. We begin by examining whether editors are more likely to match a female-authored paper with a female referee, suggesting an awareness of possible gender differences in recommendations. We then address four main sets of questions. First, do male and female referees assess papers differently, and does the gender composition of authors matter for how different referees rate a paper? Second, how reliable are the assessments of male and female referees in predicting future citations, and does this vary with the gender of the authors? Third, how do editors weigh the recommendations of different referees against the information contained in prior publications and other author characteristics, including gender? Fourth, are there gender-related differences in the time that referees take to make a recommendation, or that editors take to reach an initial decision? We compare our findings to the results from a survey of economists, and use the survey results to help interpret the gender gaps in referees' and editors' decisions.
External Link(s)

Registration Citation

Citation
, et al. 2018. "Are the Referees and Editors in Economics Gender Neutral?." AEA RCT Registry. October 26. https://doi.org/10.1257/rct.3048-2.0
Former Citation
, et al. 2018. "Are the Referees and Editors in Economics Gender Neutral?." AEA RCT Registry. October 26. https://www.socialscienceregistry.org/trials/3048/history/36293
Experimental Details

Interventions

Intervention(s)
This registration is for an observational study analyzing whether editors and referees at 4 leading economics journals are gender-neutral in their recommendations and decisions. We detail how we will analyze the data in a pre-analysis plan, written before we have access to the main data set.
Intervention Start Date
2018-06-11
Intervention End Date
2018-09-11

Primary Outcomes

Primary Outcomes (end points)
Rates of desk rejection and revise and resubmit among submitted papers; length (in days and number of rounds) of the editorial process; rate of positive referee recommendations
Primary Outcomes (explanation)
Details are in the pre-analysis plan

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The trial is about an observational study, analyzing the role of gender of authors and referees in editorial decisions. The design, which is specified in detail in the analysis plan, builds on Card and DellaVigna (2017). As part of the project, we will also survey economists about how they see the role of gender in editorial decisions; the full test of the survey is in the analysis plan.
Experimental Design Details
Randomization Method
N/A
Randomization Unit
N/A
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
We will analyze about 30,000 papers, of which a bit more than half are not desk rejected.
Sample size: planned number of observations
29,872 papers total, including 15,177 non desk rejected papers.
Sample size (or number of clusters) by treatment arms
There are not treatment arms in this observational study
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
UC Berkeley Committee for Protection of Human Subjects
IRB Approval Date
2018-06-04
IRB Approval Number
2018-04-10955
Analysis Plan

Analysis Plan Documents

EditorGenderAnalysisPlan

MD5: 5f4d8c8ff7dbbed13ae21784f6082fdf

SHA1: c2ac217b3582e33d6bf65e4775b1cc53d99e44d7

Uploaded At: June 07, 2018

Post-Trial

Post Trial Information

Study Withdrawal

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials