Value of AI Explanations

Subjects have to make a prediction about whether a specific loan applicant repaid a loan. AI prediction was offered. We elicit subjects' willingness to pay for explanations for AI predictions using BDM. We vary timing of when the explanation can be seen (before or after decision), with or without private signal, and with or without tutorial on contingent reasonsing.

(This is referred to as secondary experiment in AEARCTR-0015581, where we stated that we will separately pre-register this experiment. Similar bot-detecion techniques stated in AEARCTR-0015581 will apply here)

We use a between-subjects 2×2 design, crossed with a within-subjects manipulation:
1. Timing of the Explanation (between-subjects)
o Before: If a participant bids enough to win, the Explanation (a one-line “main feature” behind the High default risk label) is shown before the participant makes a final prediction. The Explanation can therefore influence accuracy and bonus.
o After: If a participant bids enough to win, the Explanation is shown after the participant’s prediction is locked. In this arm the Explanation cannot affect bonus odds; WTP reflects curiosity/consumption value.
2. Contingent Reasoning Aid / Information Provision (between-subjects)
o No-Aid: Participants receive no tutorial or guided walk-through of how the Explanation interacts with any private information.
o Aid: Participants see a brief, standardized two-case decision aid that makes the contingent logic transparent (i.e., when a private note about a feature is relevant depends on whether the Explanation names that feature as the driver).
3. Private Information (within-subjects, fixed order)
o No Private Note → Private Note: Each participant first completes the task without any extra information, then is shown a Private Note stating that the applicant’s 2-year employment gap is due to a benign accredited training program (not termination).
o Because the model was trained on data where gaps usually indicate termination, the Private Note creates conditions where the Explanation can be instrumentally valuable — if and only if the gap is the main feature driving the label.

2025-09-15

2025-12-31

Willingness to pay for explanation (in $$)

• WTP (No-Note) → baseline curiosity/control.
• WTP (Private-Note) → curiosity + instrumental.
• ΔWTP = WTP(Note) – WTP(No-Note) → incremental instrumental value.
• Between-subjects contrasts: Timing (Before vs After) identifies whether WTP reflects instrumental vs curiosity. Aid vs No-Aid tests whether contingent reasoning scaffolding increases recognition of instrumental value.

prediction about the loan status (Defaulted or Repaid)

Secondary: prediction accuracy (relative to true applicant outcome), purchase behavior, comprehension accuracy, time-on-page.

Between-subjects 2×2 design, crossed with a within-subjects manipulation

Same exclusion criteria for subjects as AEARCTR-0015581 (if they fail the bot tests or attention test)

Qualtrics in-build randomizer

individual

No

1000 individuals

1000 individuals

1000 individuals, 250 per arm