Can AI help people assess the trustworthiness of promises?

Last registered on June 24, 2024
View Trial History

Pre-Trial

Trial Information

General Information

Title
Can AI help people assess the trustworthiness of promises?
RCT ID
AEARCTR-0013158
Initial registration date
June 04, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
June 24, 2024, 12:09 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
Philippines
Region

Primary Investigator

Name
Renke Schmacker
Affiliation
WZB & DIW Berlin
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Lidingrong Huang
PI Affiliation
NYU Shanghai
Contact Investigator

Additional Trial Information

Status
In development
Start date
2023-12-06
End date
2024-10-06
Keywords
Behavior, Firms & Productivity, Labor
Additional Keywords
Promises, trustworthiness, Debt collection, human-AI interaction
JEL code(s)
C93, D83
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
We study the ability of AI to improve the assessment of trustworthiness using a natural field experiment. We collaborate with a nationally leading debt collection agency in the Philippines for this field experiment. After being contacted by collection agents, some debtors may offer their verbal promises to start repaying over the phone. Agents have to assess the trustworthiness of the promise and---if the promise is at risk of being broken---renegotiate the settlement. We test the effect of an AI-powered prediction model on the performance of debt collectors.
External Link(s)

Registration Citation

Citation
Huang, Lidingrong and Renke Schmacker. 2024. "Can AI help people assess the trustworthiness of promises?." AEA RCT Registry. June 24. https://doi.org/10.1257/rct.13158-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
Debt collectors are randomised into three experimental groups. One control group, one generic reminder treatment group that receives generic messages suggesting that not all repayment promises are trustworthy. One AI-generated feedback group that receives AI predictions on the trustworthiness of repayment promises. The only difference between the two treatment groups is in the message content.
Intervention Start Date
2024-06-06
Intervention End Date
2024-09-06

Primary Outcomes

Primary Outcomes (end points)
Performance measures:
• The percentage of PTPs that are honoured/fulfilled
• Average number of days for agents to receive repayment on their accounts
• Overall repayment amount
Effort measures
• Number of renegotiations done by agents
• Number of PTPs agents receive
• Agent-assessed trustworthiness of PTPs

Note that PTPs refer to debtors' promises to pay.
Primary Outcomes (explanation)
We will analyse these outcomes both on the agent level and on the PTP level. We will also look at dynamics to assess whether the treatment has spillovers on the behavior of agents in the treatment after receiving several reminders.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Subjects are existing debt collectors working for our industry partner in the Philippines. Subjects are randomized either into a control group, a generic reminder group, and an AI-powered reminder group.

For the control group: No manipulation.

For the generic-reminder group: Subjects receive written reminders online through the internal communication channel that many repayment promises are reneged upon.

For the AI-powered reminder group: Subjects receive written reminders online through the same internal communication channel that a specific promise they received has a high probability to be reneged upon based on the AI prediction model.
Experimental Design Details
Randomization Method
Randomisation by a computer
Randomization Unit
Individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
900-1000 debt collectors.
Sample size: planned number of observations
900-1000 debt collectors.
Sample size (or number of clusters) by treatment arms
900-1000 debt collectors.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
WZB Research Ethics Review
IRB Approval Date
2024-03-05
IRB Approval Number
2024/01/229
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials