Generative AI in Financial Services: A Field Experiment in Insurance Advisory

Last registered on October 16, 2026
View Trial History

Pre-Trial

Trial Information

General Information

Title
Generative AI in Financial Services: A Field Experiment in Insurance Advisory
RCT ID
AEARCTR-0019915
Initial registration date
October 09, 2026

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
October 16, 2026, 2:15 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
China
Region

Primary Investigator

Name
Yingju Ma
Affiliation
University of Cambridge
Contact Primary Investigator

Other Primary Investigator(s)

Additional Trial Information

Status
Completed
Start date
2025-03-01
End date
2025-07-31
Keywords
Finance & Microfinance, Firms & Productivity, Labor
Additional Keywords
Generative AI, financial advisory, human-AI collaboration
JEL code(s)
D83, G22, J24, L86, O33
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This study evaluates a production-grade Generative AI tool that offers real-time reply suggestions to licensed insurance brokers at Ant Insurance, China's largest insurance brokerage, during live customer advisory conversations. The tool was introduced in 2025. In the first stage, a subset of brokers received access to the tool ahead of the wider release. In the second stage, access was randomized at the customer level over a 20-day window: each customer who contacted the platform was assigned to a treatment group, in which the tool was active in the broker's workspace for that customer's conversations, or to a control group, in which the tool was disabled. The primary outcomes are realized premium revenue per customer and the conversion rate within 15 days of first contact. Secondary outcomes include dialogue rounds, and the broker's exposure to and adoption of AI suggestions. I estimate intent-to-treat effects of AI access, use random assignment as an instrument for actual AI use, and examine heterogeneity by broker seniority, tenure, and ability. The customer-level stage covers roughly 165,000 customers and about 1,000 brokers.
External Link(s)

Registration Citation

Citation
Ma, Yingju. 2026. "Generative AI in Financial Services: A Field Experiment in Insurance Advisory." AEA RCT Registry. October 16. https://doi.org/10.1257/rct.19915-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
During the experiment, customers on the insurance platform who sent inquiries were randomized: each customer who contacted the platform was assigned to a treatment group, in which the tool was active in the broker's workspace for that customer's conversations, or to a control group, in which the tool was disabled. The same broker typically served customers from both groups.
Intervention (Hidden)
Intervention Start Date
2025-05-20
Intervention End Date
2025-06-10

Primary Outcomes

Primary Outcomes (end points)
Realized premium revenue per customer and the conversion rate of brokers
Primary Outcomes (explanation)
The primary outcomes are realized premium revenue per customer and the conversion rate within 15 days of first contact.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary outcomes include dialogue rounds and the broker's exposure to and adoption of AI suggestions.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
During the experiment, customers on the insurance platform who sent inquiries were randomized: each customer who contacted the platform was assigned to a treatment group, in which the tool was active in the broker's workspace for that customer's conversations, or to a control group, in which the tool was disabled. The same broker typically served customers from both groups.
Experimental Design Details
Randomization Method
Randomization is done through Ant Group's internal A/B test platform, designed for managing all the tests for Alipay's user base.
Randomization Unit
The experiment includes two stages, first stage is rabdomized at broker level, the second stage is randomized at the customer level.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
1000 brokers
Sample size: planned number of observations
150,000 customers, and 1,000 brokers
Sample size (or number of clusters) by treatment arms
20% control, 80% treatment, among 150,000 customers
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials