Evaluating an AI-Powered Research Development Tool for Academic Productivity and Well-being

Last registered on January 28, 2026
View Trial History

Pre-Trial

Trial Information

General Information

Title
Evaluating an AI-Powered Research Development Tool for Academic Productivity and Well-being
RCT ID
AEARCTR-0017749
Initial registration date
January 22, 2026

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
January 28, 2026, 7:07 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Name
Jaime Polanco-Jimenez
Affiliation
KU Leuven
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Almudena Sevilla
PI Affiliation
London School of Economics (LSE)
Contact Investigator

Additional Trial Information

Status
In development
Start date
2026-02-01
End date
2028-06-30
Keywords
Behavior, Education, Firms & Productivity, Gender, Labor, Welfare
Additional Keywords
AI, Research Development, Peer Review, Academic Skills, Work-Life Balance, Machine Learning, Tacit Knowledge, Equity, Productivity, Innovation, Digital Tools, Human Capital
JEL code(s)
A23, C93, J24, O31, L86, I31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This randomized controlled trial (RCT) evaluates the causal impact of an AI-powered Research Development Tool on the academic productivity and well-being of researchers. Participants, primarily PhD students and junior female economists, will be randomly assigned to one of two groups: a control group receiving feedback from a general-purpose AI, or a treatment group gaining access to a comprehensive AI-driven "Research Development Suite." This suite offers detailed, structured feedback on research papers and integrated workflow features. Over a 24-month intervention period, we will measure changes in objective productivity metrics (e.g., papers submitted/published, co-author networks) and subjective well-being (e.g., job satisfaction, work-life balance). The study aims to determine whether advanced AI tools can enhance career development, foster equity in the research community by making tacit knowledge more accessible, and ultimately improve the quality of academic output.

External Link(s)

Registration Citation

Citation
Polanco-Jimenez, Jaime and Almudena Sevilla. 2026. "Evaluating an AI-Powered Research Development Tool for Academic Productivity and Well-being." AEA RCT Registry. January 28. https://doi.org/10.1257/rct.17749-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
This is a two-arm, parallel-group, single-blind randomized controlled trial (RCT). Participants will be randomly allocated to either a control group or a treatment group in a 1:1 ratio. The intervention will last for 24-26 months. Recruitment will target PhD students and junior female economists via email campaigns and professional development workshops. Data will be collected through baseline and endline surveys and continuous behavioral logging from the AI platform. The study aims to evaluate the tool's impact on academic productivity and well-being.

Intervention Start Date
2026-02-01
Intervention End Date
2028-06-30

Primary Outcomes

Primary Outcomes (end points)
Change in the number of papers submitted for publication.
Change in the number of papers published (working papers, R&Rs, accepted/published journal articles).
Change in self-reported job satisfaction.
Change in self-reported work-life balance satisfaction.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
This is a two-arm, parallel-group, single-blind randomized controlled trial (RCT). Participants will be randomly allocated to either a control group or a treatment group in a 1:1 ratio. The intervention will last for 24-26 months. Recruitment will target PhD students and junior female economists via email campaigns and professional development workshops. Data will be collected through baseline and endline surveys and continuous behavioral logging from the AI platform. The study aims to evaluate the tool's impact on academic productivity and well-being.

Experimental Design Details
Not available
Randomization Method
Individual participants will be randomly assigned to one of the two arms using a computerized random assignment algorithm, ensuring a 1:1 allocation ratio. To mitigate potential contamination and spillovers, each participant will receive a unique, tokenized access link to their assigned tool. The platform will prevent simultaneous logins from different IP addresses using the same token. Furthermore, participants in the control group will be informed about a waitlist control design, where they will be granted full access to the treatment tool after the intervention period concludes, reducing any incentive to seek access to the treatment arm during the study. Heterogeneous treatment effects will be explored for PhD students versus junior professors.

Randomization Unit
Individual participant

Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
Not applicable as randomization is individual

Sample size: planned number of observations
The primary planned sample size is 128 participants. This number is derived to detect a medium effect size. Sample size will be sensitive to the observed effect size; a study targeting smaller effects may require up to 786 participants.
Sample size (or number of clusters) by treatment arms
Arm 1 (Control Group): 64 participants
Arm 2 (Treatment Group - RefereeAI Suite): 64 participants
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Accounting for a two-arm design with individual randomization, the power analysis for 80% power and a significance level of 0.05 indicates the following sample size requirements per arm: 1. Large Effect (Cohen's d = 0.8): 25 participants per arm (50 total). 2. Medium Effect (Cohen's d = 0.5): 64 participants per arm (128 total). This is our primary target sample size. 3. Small Effect (Cohen's d = 0.2): 393 participants per arm (786 total). Our planned sample size of 128 (64 per arm) is powered to detect a medium effect. A larger sample would be required to detect smaller effects with the same statistical power. If a sample of 2000 researchers (1000 per arm) were achieved, the study would be powered to detect a very small effect size of approximately Cohen's d = 0.18. This high statistical power implies that we would be able to detect even a minimal, subtle difference between the two groups. However, detecting a very small effect forces a critical interpretation of the results: while statistically significant, an effect of this magnitude (d=0.18) may lack practical significance in terms of meaningful real-world impact on researcher productivity or well-being.
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan