Humans’ predictions based on text-based information

Last registered on May 21, 2024
View Trial History

Pre-Trial

Trial Information

General Information

Title
Humans’ predictions based on text-based information
RCT ID
AEARCTR-0013622
Initial registration date
May 15, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
May 21, 2024, 11:05 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
China
Region

Primary Investigator

Name
XIQIAN CAI
Affiliation
XIAMEN UNIVERSITY
Contact Primary Investigator

Other Primary Investigator(s)

Additional Trial Information

Status
In development
Start date
2024-05-22
End date
2024-06-10
Keywords
Behavior
Additional Keywords
JEL code(s)
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
In this study, we investigate how humans make predictions based on text-based information. Our objective is to use statements of loan purposes from a crowdfunding website as the text-based information. We examine how prediction efficiency changes when we include additional information such as gender, loan amount, and loan interest rate. Additionally, we explore if and how their decisions are affected when we provide experts’ opinions alongside the text-based information.

The rationale is that humans may use text-based information to predict whether a loan borrower is trustworthy and whether a loan may default. We hypothesize that when the text-based information is limited, the prediction is not significantly different from random guessing. We also hypothesize that humans may exhibit some gender bias in their predictions and that their accuracy may improve when additional screening information, such as loan amount and loan interest rate, is provided. Finally, we hypothesize that humans’ predictions may be influenced by both experts’ opinions and opinions generated by large language models (LLMs).
External Link(s)

Registration Citation

Citation
CAI, XIQIAN. 2024. "Humans’ predictions based on text-based information." AEA RCT Registry. May 21. https://doi.org/10.1257/rct.13622-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
The experiment will be divided into six sections, each containing ten questions. In the first section, participants will receive only text-based information. In the second section, they will receive text-based information and gender information. In the third section, they will receive text-based information and loan interest rate information. In the fourth section, they will receive text-based information and loan amount information. In the fifth section, they will receive text-based information and ratings from financial experts. In the sixth section, they will receive text-based information and ratings from large language models (LLMs).
Intervention Start Date
2024-05-22
Intervention End Date
2024-06-03

Primary Outcomes

Primary Outcomes (end points)
credibility rating for each statement on a scale from 0 to 100
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The experiment will be divided into six sections, each containing ten questions. In the first section, participants will receive only text-based information. In the second section, they will receive text-based information and gender information. In the third section, they will receive text-based information and loan interest rate information. In the fourth section, they will receive text-based information and loan amount information. In the fifth section, they will receive text-based information and ratings from financial experts. In the sixth section, they will receive text-based information and ratings from large language models (LLMs).
Experimental Design Details
All the text-based information, along with gender, loan interest rate, and loan amount, comes from real listings provided by a crowdfunding firm in China. We invited banking experts from major banks across various provinces and regions to rate these text-based listings. Additionally, we employed large language models to rate the same information.
The data form two pools for our experiment. The first pool (Pool 1) includes a larger number of listings with text-based information, gender, loan interest rate, and loan amount. Each listing is randomly drawn from Pool 1 for sections 1 through 4. The second pool (Pool 2) contains a smaller number of listings with text-based information and ratings from experts and LLMs. Each listing is randomly drawn from Pool 2 for sections 5 and 6.
Randomization Method
randomization done in platform by a computer
Randomization Unit
individual
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
1000 subjects
Sample size: planned number of observations
1000 subjects
Sample size (or number of clusters) by treatment arms
1000 subjects
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
The institutional Review Board of Finance and Economics Experimental Laboratory
IRB Approval Date
2024-05-15
IRB Approval Number
FEEL240501
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials