Potential Biases in the Publication Process

Last registered on April 12, 2023
View Trial History

Pre-Trial

Trial Information

General Information

Title
Potential Biases in the Publication Process
RCT ID
AEARCTR-0005729
Initial registration date
April 16, 2020

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
April 17, 2020, 12:56 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
April 12, 2023, 2:08 PM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Country
United States of America
Region

Primary Investigator

Name
Fulya Ersoy
Affiliation
University of Chicago
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Jennifer Pate
PI Affiliation
Loyola Marymount University
Contact Investigator

Additional Trial Information

Status
Completed
Start date
2020-05-01
End date
2020-08-31
Keywords
Gender
Additional Keywords
JEL code(s)
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This project explores the existence and the degree of the different biases economists face in the publication process with the help of an experiment.
External Link(s)

Registration Citation

Citation
Ersoy, Fulya and Jennifer Pate. 2023. "Potential Biases in the Publication Process." AEA RCT Registry. April 12. https://doi.org/10.1257/rct.5729-4.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
Intervention (Hidden)
Intervention Start Date
2020-05-01
Intervention End Date
2020-08-31

Primary Outcomes

Primary Outcomes (end points)
1. Guesses about whether a paper has been published and if yes, in which journal
2. Guesses about the number of citations a paper has
3. Guesses about the overall quality of the paper
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Subjects will receive 8 abstracts that belong to economics papers. These abstracts will vary along some dimensions. Subjects will be randomly assigned into one of the four treatments. Depending on the treatment, subjects will receive no information, partial information or full information about these abstracts. Subjects will answer questions regarding the quality of the abstracts. There will be a demographics questionnaire at the end.
Experimental Design Details
Subjects will receive 8 abstracts that belong to economics papers. These abstracts will vary along two dimensions: gender of the author and institution of the author. Subjects will be randomly assigned into one of the four conditions: no information, name information, institution information, name and institution information. In the no information treatment, subjects will not receive any information about the author of the paper. In the name information, subjects will know the name of the author. In the institution information, subjects will know the institution of the author. In the name and institution information, subjects will know both the name and the institution of the author. Subjects will answer questions regarding the quality of the abstract per each abstract. At the end of the survey, we ask our respondents a few demographic questions (gender, race, age, title, research area, institutional affiliation, journal affiliation) based on which we will conduct our heterogeneity analysis.
Randomization Method
Randomization is done using random.org.
Randomization Unit
Randomization unit is individual. Individuals are first stratified into 4 groups based on their journal affiliations, then randomly assigned to one of the 4 treatment groups.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
200 subjects
Sample size: planned number of observations
200 subjects
Sample size (or number of clusters) by treatment arms
50 subjects in each treatment arm
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
Loyola Marymount University
IRB Approval Date
2020-03-17
IRB Approval Number
2020 SP 26 R
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
August 01, 2020, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
August 01, 2020, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
Was attrition correlated with treatment status?
Final Sample Size: Total Number of Observations
Final Sample Size (or Number of Clusters) by Treatment Arms
Data Publication

Data Publication

Is public data available?
Yes
Public Data URL
https://doi.org/10.3886/E185342V4

Program Files

Program Files
Yes
Program Files URL
https://doi.org/10.3886/E185342V4
Reports, Papers & Other Materials

Relevant Paper(s)

Abstract
How might the visibility of an author's name and/or institutional affiliation allow bias to enter the evaluation of economics papers? We ask highly qualified journal editors to review abstracts of solo-authored papers which differ along the dimensions of gender and institution of the author. We exogenously vary whether editors observe the name and/or institution of the author. We identify positive name visibility effects for female economists and positive institution visibility effects for economists at the top institutions. Our results suggest that male economists at top institutions benefit the most from non-blind evaluations, followed by female economists (regardless of their institution).
Citation
Ersoy, Fulya and Pate, Jennifer, Invisible hurdles: Gender and institutional differences in the evaluation of economics papers, Economic Inquiry, 2023.
URL
https://doi.org/10.1111/ecin.13145

Reports & Other Materials