The Impact of Generative AI on Research Productivity

Recruitment: Over 400 participants (PhD, master’s, and undergraduate students) in economics-related fields, recruited online; the final sample size will depend on registrations.

Baseline Survey: Collect whether the participant has published in academic journals, hours and purposes of AI use, and WeChat ID; record covariates / benchmarks including prior publications, home institution, cohort/year, and results from an initial round of testing.

Round 1

Task: Each participant completes an English-language research proposal (topic assigned).
Requirements:
1) Literature review/summary
2) Statement of novelty/innovation
3) Data sources and sample size; clarify whether the data are obtainable
4) The econometric (or theoretical) model to be used
5) Complete within 0.5–2 hours
Restriction: No AI use (screen activity monitored via Tencent Meeting recording).
Compensation: ¥50 per person; the top 10% by score receive an additional ¥1,000 (announced after all rounds are completed).

Round 2
Control group: Same as Round 1 (new topic).
Treatment group: Identical except AI use is allowed (new topic).
Post-task: After the proposal is completed, provide a 10-minute AI training (watch a video and answer questions).
Other procedures: As in Round 1.

Round 3
The sample is divided into four groups:
1）a control group of 100 participants without training;
2）a control group of 100 participants with AI training;
3）a treatment group of 100 participants without training;
4）a treatment group of 100 participants receiving the same AI training.

This design allows us to disentangle whether any effects arise from the training itself and whether AI—through training and usage experience—attenuates humans’ intrinsic creativity. We will ensure strict uniformity of training content and avoid including task-specific solution techniques. (In my understanding, this effectively yields three intervention arms—T1: AI training plus permission to use AI; T2: AI training only; T3: permission to use AI only—and one pure control group.)

2025-10-20

2025-12-31

1. Grammatical errors
2. Logical coherence score
3. Expert rating
4. Task completion time
5. Output per unit time (productivity)

1. Methodological novelty score: Not determined yet
2. Innovativeness of research contribution score: Not determined yet

Not available.

1.Based on the registration records and baseline questionnaire, use stratified random sampling to assign participants to the treatment and control groups.
2.Train research assistants (RAs).
3.Randomly match treatment and control participants to RAs (avoiding pairings of classmates or the same major as far as possible).
4.RAs contact participants in advance to schedule a session and inform them that a 0.5–2 hour Tencent Meeting is required; each round offers ¥50–¥1,000 in compensation.

5. For each session, the RA contacts one randomly assigned participant, sends the Tencent Meeting link, ensures screen recording is enabled, provides the research topic and task requirements (to be completed within 0.5–2 hours), and clarifies AI rules: no AI in Round 1 (all participants); AI optional for the Round-2 treatment group.

6. At the end of the meeting, collect the proposal while keeping the recording; name both files as RAName-ParticipantName-Date.

7.Conduct anonymous human grading together with AI-based scoring.

8.Compile the writing submissions into an experimental-design manuscript suitable for submission.

Randomization done in office by a computer

individual

No

No

400 individuals

100 Pure control (Control and ban AI)
100 Treatment and allows AI
100 Treatment and ban AI
100 Control and allows AI