Algorithm Aversion Revisited: Examining the Role of Task Creativity, Incentives, and Issue Salience in Educational Writing Tasks

Last registered on November 19, 2025
View Trial History

Pre-Trial

Trial Information

General Information

Title
Algorithm Aversion Revisited: Examining the Role of Task Creativity, Incentives, and Issue Salience in Educational Writing Tasks
RCT ID
AEARCTR-0014603
Initial registration date
November 16, 2025

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
November 19, 2025, 2:12 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
China
Region
Wuhan

Primary Investigator

Name
Lian XUE
Affiliation
Wuhan University
Contact Primary Investigator

Other Primary Investigator(s)

Additional Trial Information

Status
Completed
Start date
2024-10-10
End date
2025-02-28
Keywords
Behavior, Education, Lab
Additional Keywords
Algorithm aversion; Issue salience
JEL code(s)
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This study revisits the concept of algorithm aversion, particularly in the context of educational writing tasks. While algorithm aversion— the preference for human judgment over algorithmic advice— has been observed in decision-making fields like business and healthcare, it remains underexplored in domains requiring creativity, such as education. We examine how task type (creative vs. non-creative), incentive structures (positive vs. negative), and issue salience influence individuals' reliance on AI-generated content. Through an experimental design, participants engaged in writing tasks that varied by these factors, allowing us to assess their willingness to integrate AI suggestions. Our findings aim to uncover the drivers of algorithm aversion and provide insights for educators, policymakers, and AI developers. The results have implications for designing AI tools that can complement rather than replace human creativity, fostering more effective human-AI collaboration in educational settings.
External Link(s)

Registration Citation

Citation
XUE, Lian. 2025. "Algorithm Aversion Revisited: Examining the Role of Task Creativity, Incentives, and Issue Salience in Educational Writing Tasks." AEA RCT Registry. November 19. https://doi.org/10.1257/rct.14603-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
The intervention in this study involves exposing participants to AI-generated content during educational writing tasks. Participants will complete both creative (commentary) and non-creative (translation) tasks.
Intervention (Hidden)
The study manipulates three key factors:

Task Type: Participants engage in both creative tasks, which require personal judgment, and non-creative tasks, which are more structured and rule-based.
Incentive Structure: Participants are randomly assigned to two conditions— a positive incentive condition, where they can earn additional rewards if they rank in the top 50%, or a negative incentive condition, where they face penalties if their performance ranks in the bottom 50%.
AI Suggestion Type: Participants are exposed to either neutral or non-neutral AI-generated examples during their tasks. Neutral examples are intended to be objective and unbiased, while non-neutral examples may contain suggestive language or opinions.
These interventions are designed to test how different contexts and motivations affect participants' reliance on algorithmic assistance in educational settings.

Intervention Start Date
2024-11-04
Intervention End Date
2024-12-14

Primary Outcomes

Primary Outcomes (end points)
Reliance on AI Suggestions and Task Performance
Primary Outcomes (explanation)
Reliance on AI Suggestions: Measured by the degree to which participants incorporate AI-generated content into their final written submissions. This will be quantified by assessing the similarity between the AI suggestions and the participants' outputs.
Performance: Evaluated based on task-specific criteria, where both creative and non-creative outputs are scored for quality, accuracy, and creativity. The scores will help determine if reliance on AI suggestions correlates with performance improvements.

Secondary Outcomes

Secondary Outcomes (end points)
Task completion time and Confidence
Secondary Outcomes (explanation)
Task completion time: The time taken by participants to complete each task, which will be used to assess if AI assistance reduces or increases task duration.
Confidence: Confidence in own ability and in AI ability might influence algorithm aversion. This will provide insights into the psychological aspects of algorithm aversion.

Experimental Design

Experimental Design
This study investigates the factors influencing algorithm aversion in educational writing tasks through a randomized experimental design. Participants will complete two types of writing tasks— creative and non-creative—while being exposed to different types of incentives and AI-generated suggestions. The design includes:
Task Type (Within-Subject Design): Participants engage in both creative (e.g., commentary) and non-creative (e.g., translation) writing tasks.
Incentive Structure (Between-Subject Design): Participants are randomly assigned to either a positive or negative incentive condition to evaluate how incentives influence reliance on AI suggestions.
AI Suggestion Type (Between-Subject Design): AI-generated examples vary, with participants exposed to either neutral or non-neutral suggestions to assess how suggestion nature affects acceptance.
We aim to explore how these factors interact and influence participants' willingness to rely on AI assistance in educational contexts.
Experimental Design Details
Randomization Method
The treatments are randomized at the session level using Weikeyan system -- a WeChat based experiment recruitment system to ensure randomization.
There are 4000 subjects in the Weikeyan subjects pool; the system will divide the subjects into random N cohorts and send advertisements based on the cohorts for each treatment. The advertisement is send by WeChat notifications, which is the mostly commonly used phone App in mainland China.
Randomization Unit
The unit observation is at the individual level
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
The treatments are clustered at the session level; we plan to run 4-10 sessions per treatment. Each session consists of around 10 -15 subjects.
Sample size: planned number of observations
240 to 400 participants in total: 60-100 participants/treatment * 4 treatments.
Sample size (or number of clusters) by treatment arms
We plan to recruit 60-100 participants for each treatment.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
Center of Behavioral and Experimental Research at Wuhan University
IRB Approval Date
2024-10-17
IRB Approval Number
EM240040
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials