Using Natural Language Processing to Enhance Compliance Behavior
Last registered on November 20, 2019

Pre-Trial

Trial Information
General Information
Title
Using Natural Language Processing to Enhance Compliance Behavior
RCT ID
AEARCTR-0005049
Initial registration date
November 20, 2019
Last updated
November 20, 2019 2:54 PM EST
Location(s)
Region
Primary Investigator
Affiliation
University of Cologne
Other Primary Investigator(s)
PI Affiliation
Freie Universit├Ąt Berlin
PI Affiliation
Freie Universit├Ąt Berlin
Additional Trial Information
Status
In development
Start date
2019-11-26
End date
2019-12-20
Secondary IDs
Abstract
A lot of laboratory experiments in the field of behavioral economics require participants to chat with each other. Very often the chat is incentivized such that it is directly related to a more easily measurable variable, e.g., the amount paid to a public good or the reported number of a tossed die roll. If this relationship exists, the resulting data is gold-standard labeled data. Consequently, training a supervised machine learning classifier that learns the relationship between text and (numerical) output is a promising approach. This paper describes how we trained, based on chat texts obtained from a tax evasion experiment, a classifier to predict whether a group reported (taxable) income honestly or not. Before this classifier is leveraged for future studies, its generalisability needs to be assessed. Therefore, we designed an experiment, which alters the initial honesty framework with respect to three major dimensions: Firstly, the context is no longer a tax evasion setting, but participants are asked to report surplus hours. Secondly, the direction of the lie is switched. It is optimal to overreport in the surplus hour setting whereas it was optimal to underreport in the tax evasion setting. Thirdly, the group size is reduced from three to two. If the classifier achieves satisfying performance metrics based on out-of of sample predictions in a slightly different context, the technology can be leveraged in future experimental research.
External Link(s)
Registration Citation
Citation
Fochmann, Martin, Carina Ines Hausladen and Peter Mohr. 2019. "Using Natural Language Processing to Enhance Compliance Behavior." AEA RCT Registry. November 20. https://doi.org/10.1257/rct.5049-1.0.
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details
Interventions
Intervention(s)
Group members discuss which number of surplus hours they want to state.
Treatment-Group: The chat text is evaluated by a pre-trained classifier, labeling the group as either honest or dishonest. Based on the label, 2 out of 7 groups in each session are chosen to be controlled.
Control-Group: 2 out of 7 groups are randomly chosen to be controlled.
Intervention Start Date
2019-11-26
Intervention End Date
2019-12-20
Primary Outcomes
Primary Outcomes (end points)
The reported amount of surplus hours by each participant. The group chat between two members of a group.
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
Participants work for a fictive company in teams of two.
Both group members have to state their surplus hours.
Both members have to report the same amount of surplus hours.
The group being controlled depends either on the classifier's result (treatment) or is randomly chosen.
In both conditions, 2 out of 7 groups are chosen to be controlled.
If reports differ, the group is always controlled.
Experimental Design Details
Randomization Method
Participants draw a seat number.
Treatments are alternated each session, where the initial treatment to start with is chosen randomly.
Randomization Unit
Treatment and control conditions are alternated between sessions.
Was the treatment clustered?
Yes
Experiment Characteristics
Sample size: planned number of clusters
100 groups
Sample size: planned number of observations
200 participants
Sample size (or number of clusters) by treatment arms
100 participants (50 groups) in each treatment (2 treatments overall).
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
German Association for Experimental Economic Research e.V.
IRB Approval Date
2019-11-19
IRB Approval Number
EUFf7PP5
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
No
Is data collection complete?
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports and Papers
Preliminary Reports
Relevant Papers