|
Field
Trial Title
|
Before
Coarsening Signals in Human-AI Interaction - Experiment 1
|
After
Optimising Signals in Human-AI Interaction - Experiment 1
|
|
Field
Abstract
|
Before
Artificial intelligence (AI) has caught pace with — and in some contexts even surpassed — humans in the ability to make predictions from data, purporting to improve decision-making. However, in cases where humans are still responsible for the final decision, biases in probabilistic reasoning can render even informative AI predictions detrimental to decision-making outcomes. Using a randomised experiment with loan underwriters, we show that the provision of coarsened AI signals at the right thresholds can improve overall decision-making in spite of information loss.
|
After
Artificial intelligence (AI) has caught pace with — and in some contexts even surpassed — humans in the ability to make predictions from data, purporting to improve decision-making. However, in cases where humans are still responsible for the final decision, biases in probabilistic reasoning can render even informative AI predictions detrimental to decision-making outcomes. Using a randomised experiment with loan underwriters, we show that the provision of optimised AI signals can improve overall decision-making in spite of information loss.
|
|
Field
Last Published
|
Before
July 03, 2024 11:09 AM
|
After
September 07, 2024 11:13 PM
|
|
Field
Intervention End Date
|
Before
July 31, 2024
|
After
October 31, 2024
|
|
Field
Experimental Design (Public)
|
Before
We recruit 300 loan underwriters to make loan application decisions.
|
After
We recruit 300 loan underwriters/officers and other participants to make loan application decisions.
|
|
Field
Intervention (Hidden)
|
Before
In a cross-randomised design, we vary: (i) the type of AI signal that is shown {binary, probability, none}, with sub-treatments varying the thresholds at which the binary signals are shown, (ii) whether the AI signal or real loan application is shown first, and (iii) whether the human posterior is elicited after seeing just the loan application.
|
After
In a cross-randomised design, we vary: (i) the type of AI signal that is shown {binary, probability, none}, with sub-treatments varying the thresholds at which the binary signals are shown, and (ii) whether or not participants are put through "loan underwriter training". [we omit this latter treatment for those who are already experienced loan underwriters].
We also vary at a question-level (i) whether the AI signal or real loan application is shown first, and (ii) whether the human posterior is elicited after seeing just the loan application.
|