A Randomized Evaluation of the Impact of GenAI Training in a Relational Job at a Global Bank

Last registered on August 10, 2026
View Trial History

Pre-Trial

Trial Information

General Information

Title
A Randomized Evaluation of the Impact of GenAI Training in a Relational Job at a Global Bank
RCT ID
AEARCTR-0019338
Initial registration date
August 09, 2026

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
August 10, 2026, 5:03 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
Spain
Region
Madrid

Primary Investigator

Name
Antonio Roldan Mones
Affiliation
ESADE and London School of Economics
Contact Primary Investigator

Other Primary Investigator(s)

Additional Trial Information

Status
In development
Start date
2026-08-06
End date
2027-06-28
Keywords
Education, Firms & Productivity, Labor
Additional Keywords
JEL code(s)
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This paper evaluates whether targeted training enables workers to translate access to generative artificial intelligence (GenAI) into improved performance in a relational job. We conduct a cluster-randomized controlled trial involving more than 700 relationship managers across approximately 300 branches of a large global bank in Mexico and Spain. All participants retain access to GenAI tools, while managers in treated branches receive a short, intensive training program designed around their day-to-day workflows. This design isolates the effect of targeted training relative to GenAI access alone. Using individual-level data on GenAI usage, administrative processes, client interactions, commercial opportunities, customer satisfaction, and incentive-based performance measures, we examine whether training increases the adoption and sophistication of GenAI use and whether it improves productivity. We investigate two principal mechanisms: the automation or streamlining of administrative tasks, which may free time for client-facing activities, and the augmentation of client interactions through better preparation and more tailored proposals. We also study whether effects differ according to workers’ prior performance, age, experience, previous GenAI use, and risk-taking profiles. The study contributes to the emerging literature on workplace GenAI by examining its deployment in a real-world relational occupation combining analytical, administrative, and interpersonal tasks. It also provides evidence on whether brief training can convert widespread access to GenAI into measurable organizational value and whether it narrows or widens performance differences across workers.
External Link(s)

Registration Citation

Citation
Roldan Mones, Antonio. 2026. "A Randomized Evaluation of the Impact of GenAI Training in a Relational Job at a Global Bank." AEA RCT Registry. August 10. https://doi.org/10.1257/rct.19338-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
The intervention consists of a short, intensive, in-person training program designed to improve relationship managers’ (RMs) ability to use generative artificial intelligence (GenAI) effectively in their daily work. The program lasts about three hours and focuses on practical applications of GenAI to the administrative, analytical, and client-facing tasks performed by relationship managers. The intervention will be implemented in two different countries (Spain and Mexico) in August, September and October in collaboration with a large Global Bank, BBVA.

Relationship managers assigned to the control group retain access to the bank’s existing GenAI tools but do not receive the additional training during the experimental period. The intervention therefore evaluates the effect of targeted GenAI training relative to GenAI access alone. The training will be offered to the control group after the experimental period /about six months later).


Intervention Start Date
2026-08-06
Intervention End Date
2026-12-31

Primary Outcomes

Primary Outcomes (end points)
1. GenAI adoption and use.
2. Commercial and client-facing activity.
3. Duration of selected internal administrative processes.
4. Conversion of commercial opportunities.
5. Customer satisfaction and client-experience indicators.
6. Relationship-manager productivity and commercial performance.

Primary Outcomes (explanation)
GenAI adoption and use will be measured using individual-level usage data for the GenAI tools available within the bank, including frequency and volume of use. Where permitted by the available data, we will also measure the sophistication of use through indicators capturing practices such as workflow creation, connections between tools, and the deployment of AI agents.

Commercial activity will be measured using the bank’s CRM records, including the number of client calls, meetings, visits, and other recorded interactions. Administrative-process efficiency will be measured by the time required to complete selected internal processes, including the preparation and revision of client risk profiles.

Opportunity conversion will be measured using CRM data covering the life cycle of commercial opportunities, including whether opportunities remain in progress or are closed and won or closed and lost. We will examine conversion rates and, where data permit, the time from the opening of an opportunity to its resolution.

Customer satisfaction and service quality will be measured using the raw "1-10" bank’s Net Promoter Score (NPS) and complementary Client Experience indicators linked to individual relationship managers. official NPS measure is a lossy transformation of the raw "on a scale of 1-10" question.

Productivity will be measured primarily using the bank’s internal incentive scorecards, which determine variable compensation and capture dimensions such as sales and value generation, customer acquisition, and risk-adjusted profitability. When outcomes are recorded on different scales, effects will also be reported in standardized units to facilitate comparison.

Secondary Outcomes

Secondary Outcomes (end points)
1. Allocation of commercial attention across the customer portfolio.
2. Quality and personalization of client interactions.
3. Customer retention and longer-term client attachment, where observable.
4. Communication-related and social-capital barriers.
5. Risk-taking and the preparation of client risk profiles.
6. Diffusion of GenAI practices and potential spillovers within treated branches.
Secondary Outcomes (explanation)
The allocation of attention across the customer portfolio will be examined using the number and distribution of interactions across clients. This will allow us to assess whether trained relationship managers engage with a broader set of customers or concentrate more attention on the most valuable clients.
Subject to data availability and privacy protections, the quality and personalization of client interactions will be studied through text-based indicators derived from recorded communications or related documentation. Customer retention will be measured using indicators of client turnover and continued attachment to the bank.
We will explore whether training changes communication-related barriers that may affect commercial performance and whether GenAI-assisted preparation changes risk-taking or produces convergence in client risk assessments. We will also examine patterns of diffusion within treated branches, including whether intensive GenAI users, colleagues, or branch managers influence the use of GenAI by other relationship managers.

Experimental Design

Experimental Design
The study is a cluster-randomized controlled trial conducted with relationship managers at a large global bank. Participating bank branches are assigned either to a treatment group that receives a short, intensive GenAI training program or to a control group that continues business as usual. Control group branches are expected to receive the training about six months later. Relationship managers in both groups retain access to the bank’s existing GenAI tools.

Randomization is conducted at the branch level because relationship managers within the same branch work closely together and frequently exchange information. This design reduces potential contamination between treatment and control participants. Outcomes will be measured using internal administrative, CRM, GenAI usage, customer-experience, and performance data collected before and during an experimental period of approximately four months.
Experimental Design Details
Not available
Randomization Method
Randomization will be conducted in an office by computer. A computer-generated random assignment will allocate participating bank branches to the treatment or control group.

Regarding stratification, in order to improve statistical power, in Spain we did it at the office-size level (by tertiles), while in Mexico we stratified by division and manager type - since the in-person trainings are being run at the division level (each of the 8 divisions comprises one or more offices).
Randomization Unit
Bank branch. All participating relationship managers working in the same branch will be assigned to the same experimental condition.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
Approximately 300 bank branches.
Sample size: planned number of observations
More than 700 relationship managers.
Sample size (or number of clusters) by treatment arms
Assuming equal allocation, approximately 150 branches and 350 relationship managers will be assigned to the GenAI-training group, and approximately 150 branches and 350 relationship managers will be assigned to the business-as-usual control group.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
A provisional calculation based on 700 relationship managers distributed across 300 branches, equal assignment between treatment and control, a two-sided significance level of 5 percent, statistical power of 80 percent, and an intracluster correlation of 0.10 yields a minimum detectable effect of approximately 0.23 standard deviations, equivalent to approximately 23 percent of the outcome’s standard deviation. This estimate does not incorporate precision gains from controlling for baseline outcomes. The final minimum detectable effect will be recalculated using the definitive roster of participating branches and relationship managers, the actual distribution of managers across branches, and pre-intervention estimates of the intracluster correlation, outcome variance, and correlation between baseline and follow-up measures.
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan