Fostering Trustworthy Human-AI Collaboration through Explainable AI and Uncertainty Quantification

We run four between-subjects treatments that can be derived from our two main treatment dimensions. Along the first treatment dimension, we vary whether participants receive AI explanations with AI recommendations in the credit score classification task. Along the second treatment dimension, we vary whether human participants receive AI certainty scores with AI recommendations.

Within subjects, the study consists of four phases. In the first phase, participants complete the task without any AI decision support. In the second phase, they receive AI support, which may include AI explanations and AI certainty scores depending on their assigned between-subjects treatment. In the third phase, participants complete the task again without any AI decision support. In the final phase, participants can choose whether to fully delegate the task to the AI system.

In addition, we vary between classification tasks of low and high AI uncertainty in each of the four phases.

In our study, we manipulate two treatment dimensions in a credit score classification task. The first treatment dimension determines whether participants receive AI explanations alongside AI recommendations (AI). Specifically, participants either (a) receive explanations with feature contribution visualizations using the well-known XAI technique SHAP (XAI) or (b) receive no explanations along with the AI recommendations.
The second treatment dimension concerns AI certainty scores. Participants either (i) receive AI certainty scores calculated using the Monte Carlo Dropout method (UQ) or (ii) do not receive any AI certainty scores along with the AI recommendations.

By combining these two treatment dimensions, we establish four distinct treatment conditions: 1) AI – participants receive AI recommendations without AI explanations or AI certainty scores, 2) XAI – participants receive AI recommendations with SHAP-based feature contribution explanations but no certainty scores, 3) UQ – participants receive AI recommendations with AI certainty scores but no explanations, and 4) XAI-UQ – participants receive AI recommendations with both SHAP-based explanations and AI certainty scores.

Before the experiment begins, all participants receive complete information about their assigned treatment condition through computerized instructions.

Within subjects, the study consists of four phases. In the first phase, participants complete the task without any AI decision support. In the second phase, they receive AI support, which may include AI explanations and AI certainty scores depending on their assigned between-subjects condition. In the third phase, participants complete the task again without any AI decision support. In the final phase, participants can choose whether to fully delegate the task to the AI system.

In addition, we vary between classification tasks of low and high AI uncertainty in each of the four phases.

2025-04-09

2025-05-31

Decision Performance, Perceived Human Certainty, Automation Decision

Decision performance is measured as the average accuracy of the credit score classification decisions with respect to the true label of the dataset. Perceived human certainty is measured on a 9-point Likert scale after each classification decision.
Automation Decision is measured as a single item question in the last part of the experiment, where participants are asked whether they want to delegate the final six credit score classification decisions towards the AI or continue classifying themselves with full AI support.

Trust, Perceived Usefulness

Trust is measured on a multi-item 7-point Likert scale at the end of the treatment phase, based on McKnight et al. (2002).
Perceived Usefulness is measured on a multi-item 7-point Likert scale at the end of the treatment phase, based on Davis et al. (1989).

References:
Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: A comparison of two theoretical models. Management Science, 35(8), 982–1003.
McKnight, D. H., Choudhury, V., & Kacmar, C. (2002). Developing and validating trust measures for e-commerce: An integrative typology. Information Systems Research, 13(3), 334–359.

Experiments are run on the online crowdworking platform Prolific. Treatments are randomized at the participant level. Each subject participates in only one between-subjects treatment. In all treatments, subjects are fully informed about the timeline of the experiment.

We conduct the randomized controlled online experiment to examine the causal effects of different AI decision-support settings—including AI explanations and AI uncertainty scores—on performance outcomes and users' perceived uncertainty. Participants will be recruited from the crowdworking platform Prolific. We will select a US-based sample for the experiment. Participants are randomly assigned to one of the four between-subjects treatment conditions. Consequently, participants either only received AI-generated recommendation, AI recommendations along with AI explanations, AI recommendations along with AI certainty scores, or AI recommendations along with AI explanations and AI certainty scores. This design aims to isolate and assess the impact of AI explanations and AI certainty scores on decision performance and user perceptions.

The online experiment employs a between-subjects design, i.e., each subject participates in exactly one of the four treatments. Subjects are aware of the treatment conditions they are receiving, but they do not know that it is a treatment, nor do they know about the other treatments. Treatments are randomized at the individual subject level.

Participants in the experiment have to complete a series of credit score classification tasks. Participants are given applicants' credit-related information, and their goal is to predict the correct credit score category as one of three classes: Poor, Standard, or Good. The experimental task is based on a credit scoring dataset from the online platform Kaggle. For the experiment, we randomly selected 36 instances from the dataset that are displayed to the participants, while ensuring a balanced set of takes in terms of credit scores (good, standard, poor) and difficulty as indicated by AI certainty scores (low and high certainty). In addition, the accuracy of the AI predictions is the same in all phases and reflects the overall accuracy of the AI system (about 71 %). The selected instances are the same for each treatment but are randomized in their order within each phase to avoid order effects.

To assist participants to make accurate classifications, they are presented with six key characteristics of each applicant. These include: Age, Annual Income, Number of Credit Cards, Interest Rate on Credit Card(s), Outstanding Debt, and Occupation. Their objective is to accurately predict the credit score of each applicant according to the ground truth in the original dataset. Participants receive a fixed participation fee of £3.00 and a bonus payment of £0.15 for each correctly classified applicant. Accordingly, total compensation ranges from a minimum of 3.00 to a maximum of £8.40. Based on findings from the pilot study, the experimental session is expected to last about 30 minutes.

To maintain the quality and validity of the data, iterative recruiting will be implemented based on participants’ performance on attention checks. If a participant fails an attention check, additional participants will be recruited to replace them. This iterative approach ensures that the final sample consists of high-quality data while adhering to the study's pre-defined inclusion criteria and timeline. Recruitment for replacements will be conducted promptly to minimize delays and maintain a consistent data collection process.

If a subject does not complete the experiment, it will be excluded from the analysis. Subjects with missing data will also be excluded, as this indicates a technical error occurring for them during the experiment.

Participants are assigned to the treatments in a randomized manner, aiming at an even distribution across tasks.

Individual randomization

No

In all treatments, the independent observation is at the individual participant level. We schedule data collection, aiming at 150 independent observation per treatment (excluding subjects that fail at least one attention check).

In all treatments, the independent observation is at the individual participant level. We schedule data collection, aiming at 150 independent observation per treatment. Thus, we aim for a total of 600 individual participants across the four treatments (excluding subjects that fail at least one attention check).

In each of the four treatments, we plan with 150 participants, which totals 600 subjects (excluding subjects that fail at least one attention check).

For our primary outcome, we aim to detect an effect size of at least d=0.25, with a power level of 80% and a significance level of 0.1 based on a pairwise t-test of treatment groups.