The Use of Algorithmic Support: Are Individuals Averse Towards Algorithms or Rather Overconfident Regarding Their Skills?

2022-10-31

2023-01-31

The primary outcomes comprise (i) the subjects’ (incentivized) monetary valuation of algorithmic support, i.e., their willingness-to-pay for advice from an algorithm, resp. delegation to an algorithm (ii) the use rate of algorithmic support, i.e. share of taken advice vs. share of delegated estimations.

To disentangle the effect of overconfidence and control over algorithms on algorithm aversion, I conduct an online experiment where participants perform an incentivized forecasting task. The experiment comprises a 2x2 full-factorial between-subject design. First, I systematically vary whether individuals receive feedback about their relative performance in the forecasting task, or not. Providing participants with feedback about their actual performance reduces overconfidence and, therefore, allows to study the effect of overconfidence on the use of algorithmic support. Second, I vary whether participants can either receive advice from an algorithm or can delegate the decision to an algorithm. This allows to investigate how the level of overconfidence at different control levels over the algorithm alters the use of algorithmic support. In all treatments, I measure the willingness-to-pay for the use of algorithmic support. This allows to analyze whether there exists a significant difference between the willingness-to-pay for advice and delegation, contingent on the performance feedback.

The experimental procedure is as follows: At the beginning of the experiment the participants receive the instructions and have to answer control questions. After successfully answering the control question, the participants execute the forecasting task where they estimate the normalized GDP per capita for different countries. They obtain some information about each country (i.e., life expectancy) but not the name of the country. Depending on the treatment participants obtain feedback about their relative performance. After completing the estimations, the participants answer a questionnaire. At the end of the experiment, they receive a payment which comprises a participation fee and the earnings of one randomly selected estimation.

Randomization by computer online

Randomization at the subject level: Randomized assignment of subjects to treatments.

No

Observations on the subject level are assumed to be independent, since subjects decide without interacting with other participants in the experimental session. Thus, the number of clusters equals the number of participants.

A minimum of 50 observations is planned for each treatment. A power analysis is conducted based on estimated effect sizes after 50 observations have been collected. Based on this power analysis, more observations may be acquired in order achieve a power level of at least 80%.

See above. Minimum of 50 per treatment.