Field
Trial Title
|
Before
Predicting (dis-)honesty: Leveraging text classification for behavioral experimental research
|
After
Predicting compliance: Leveraging chat data for supervised classification in experimental research
|
Field
Abstract
|
Before
A lot of laboratory experiments in the field of behavioral economics require participants to chat with each other. Very often the chat is incentivized such that it is directly related to a more easily measurable variable, e.g., the amount paid to a public good or the reported number of a tossed die roll. If this relationship exists, the resulting data is gold-standard labeled data. Consequently, training a supervised machine learning classifier that learns the relationship between text and (numerical) output is a promising approach. This paper describes how we trained, based on chat texts obtained from a tax evasion experiment, a classifier to predict whether a group reported (taxable) income honestly or not. Before this classifier is leveraged for future studies, its generalisability needs to be assessed. Therefore, we designed an experiment, which alters the initial honesty framework with respect to three major dimensions: Firstly, the context is no longer a tax evasion setting, but participants are asked to report surplus hours. Secondly, the direction of the lie is switched. It is optimal to overreport in the surplus hour setting whereas it was optimal to underreport in the tax evasion setting. Thirdly, the group size is reduced from three to two. If the classifier achieves satisfying performance metrics based on out-of of sample predictions in a slightly different context, the technology can be leveraged in future experimental research.
|
After
Behavioral and experimental economics have conventionally employed text data to facilitate the interpretation of decision-making processes. This paper introduces a novel methodology, leveraging text data for predictive analytics rather than mere explanation. We detail a supervised classification framework that interprets patterns in chat text to estimate the likelihood of associated numerical outcomes. Despite the unique advantages of experimental data in correlating textual and numerical information for predictive modeling, challenges such as limited sample sizes and potential data skewness persist. To address these, we propose a comprehensive methodological framework aimed at optimizing predictive modeling configurations, particularly in small experimental behavioral research datasets. We also present behavioral experimental data from a preregistered tax evasion game (n=324), demonstrating that chat behavior is not influenced by experimenter demand effects. This establishes chat text as an unbiased variable, enhancing its validity for prediction. Our findings further indicate that beliefs about others’ dishonesty, lying attitudes, and risk preferences significantly impact compliance decisions.
|
Field
JEL Code(s)
|
Before
C92, D90, H26, K42
|
After
C55,C92, D83
|
Field
Last Published
|
Before
June 24, 2020 04:01 AM
|
After
January 26, 2024 08:24 AM
|
Field
Final Sample Size: Number of Clusters (Unit of Randomization)
|
Before
175 groups
|
After
162 groups
|
Field
Final Sample Size: Total Number of Observations
|
Before
350 participants
|
After
324 participants
|
Field
Final Sample Size (or Number of Clusters) by Treatment Arms
|
Before
175 groups, 350 participants
|
After
162 groups, 324 participants
|
Field
Public Data URL
|
Before
|
After
https://github.com/carinahausladen/PredictingCompliance
|
Field
Restricted Data Contact
|
Before
[email protected]
|
After
[email protected]
|
Field
Program Files
|
Before
No
|
After
Yes
|
Field
Program Files URL
|
Before
|
After
https://github.com/carinahausladen/PredictingCompliance
|
Field
Is data available for public use?
|
Before
No
|
After
Yes
|
Field
Additional Keyword(s)
|
Before
machine learning, natural language processing, compliance, behavioral taxation
|
After
Chat data, Supervised classification, Experimental research, Tax evasion, Compliance
|
Field
Keyword(s)
|
Before
Crime Violence And Conflict, Firms And Productivity, Other
|
After
Crime Violence And Conflict, Firms And Productivity, Other
|
Field
Building on Existing Work
|
Before
|
After
Yes
|