Editorial Decisions and Journal Submissions in Economics

Last registered on May 19, 2021

Pre-Trial

Trial Information

General Information

Title
Editorial Decisions and Journal Submissions in Economics
RCT ID
AEARCTR-0007448
Initial registration date
April 23, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
April 23, 2021, 11:17 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
May 19, 2021, 8:59 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
Wellesley College

Other Primary Investigator(s)

PI Affiliation
Wellesley College

Additional Trial Information

Status
In development
Start date
2021-04-26
End date
2022-04-26
Secondary IDs
Abstract
The aim of this research study is to shed light on the peer review process and gauge the impacts of editorial decisions on researchers. We hope that the survey results will spark fruitful discussions and potentially lead to improvements in the publication process. The survey elicits information from researchers on their submission history. After reading a fake decision letter on a hypothetical submission to a top-5 journal, respondents are asked to report their beliefs about whether the paper will eventually be published in that journal or any leading journal.

External Link(s)

Registration Citation

Citation
Shastry, Gauri Kartini and Olga Shurchkov. 2021. "Editorial Decisions and Journal Submissions in Economics." AEA RCT Registry. May 19. https://doi.org/10.1257/rct.7448
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2021-04-26
Intervention End Date
2021-07-26

Primary Outcomes

Primary Outcomes (end points)
1. Perceived probability the paper will be published in the top-5 journal
2. Perceived probability the paper will be published in any leading journal (i.e., top field or higher, including this specific one)
3. Whether or not the respondent would resubmit the paper to a top-5 general interest journal
4. Whether or not the respondent would submit their next paper of similar quality to a top-5 general interest journal
5. From how many people respondent would solicit advice on publishing this paper
6. Whether the respondent would extensively revise the paper before resubmitting it
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
This study employs a survey experiment, and the participants will be academic researchers from all over the globe who have submitted at least one paper to a peer-reviewed journal for publication and received at least one editor decision letter in the past five years in economics or a related field (finance, for example) and for whom we can find a publicly available email address. We collect the email addresses from publicly available sources including the articles, journal webpages, university webpages, or other public sources. In total, there are approximately 43,000 addresses on the list.

The survey has five sections: Consent, basic questions, encouragement screen, decision letter and follow-up questions, and demographic questions. We will use Qualtrics to create and distribute the survey. The duration is approximately 10 to 15 minutes.

Subjects who consent will proceed to the basic question section where they will be asked about their job and research activities. After these baseline questions, respondents are asked to carefully read a sample letter from the editor of a top-5 general interest journal who is writing with a decision on their hypothetical submission. Respondents will indicate what they would do next if they were to receive such a letter. They will also answer questions about their confidence in the success of this paper at this journal and more generally at other leading journals. The survey will end with standard demographic questions. This section also includes questions about the approximate rank of their PhD institution and current institution. We will offer them the opportunity to provide a link to their website for us to glean this information, as well as their publication record, but this will be completely optional. The final screen will ask respondents for their email address if they would like to enter the lottery for a chance to win a prize of $50 value.

Experimental Design Details
Please see analysis plan.
Randomization Method
Qualtrics randomization
Randomization Unit
Individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
We will send invitations to the survey to around 43,000 email addresses. Assuming a 5% take-up rate, we expect a sample of 2150 respondents.
Sample size: planned number of observations
We will send invitations to the survey to around 43,000 email addresses. Assuming a 5% take-up rate, we expect a sample of 2150 respondents.
Sample size (or number of clusters) by treatment arms
We expect around 350 respondents in each treatment arm.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Please see analysis plan.
IRB

Institutional Review Boards (IRBs)

IRB Name
Human Research Protection Program of Brandeis University
IRB Approval Date
2021-04-22
IRB Approval Number
21130R-E
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials