When Good Advice is Ignored: The Role of Adviser Remuneration

Subjects undertake two tasks and are then offered the opportunity to replace their own decisions with those suggested by an advisor.

2017-04-27

2017-07-27

(good) advice followed or not; subject envy; stubbornness (in general and relating to the sunk-cost fallacy); whether the nature of the task matters (luck or ability-based) in determining if advice is followed; demographics

Whether good advice is followed is determined in the experiment: advice is good if following it increases expected payoff. Remuneration is determined by treatment (low or high, can be affected by the subject or cannot). We can also measure the role of the task itself (whether luck or ability-based) by looking at behaviour by the same individuals across tasks. Subject envy is a measure of envy drawn from the psychology literature. Stubbornness associated with the sunk-cost fallacy is measured using scenarios derived from the relevant social science literature converted to a Likert scale. Stubbornness is measured using a series of questions taken from the existing literature and again converted to a Likert scale.

MTurk study involving a series of tasks and demographic/psychometric measures.

WAVE 1
In wave 1, subjects are recruited using MTurk. They are asked to complete 2 tasks. In one they must guess the result of 10 coin tosses. In the second they are asked to complete a 10-questions Raven visual IQ test. We record their performance in the 2 tasks enabling us to allocate some of them as "advisors" for wave 2. We collect data on their experiences (for instance, on the perceived difficulty of the tasks). We record the distribution of answers so we can choose the advisors to be better than the bulk of other subjects in wave 2.

WAVE 2
In wave 2, subjects (different subjects, but again recruited through MTurk) take the 2 tasks (which are exactly the same as in wave 1). This time, after taking the tasks, they are offered the chance to switch their own answers for those of an "advisor" determined from wave 1. They are told that they can win a bonus if one of their answers (selected at random) proves to be correct. When making this choice, they are also told: (1) their own success rate; (2) the advisor's success rate; and (3) how the advisor was remunerated. Next the subjects are split into 3 treatments:
(A) In one treatment the advisor receives a bonus payment of the same level as the subjects.
(B) In another they received a larger bonus. Subjects are told that the advisor saw the same instructions as they did, hence the advisor did not know the size of the bonus before completing the tasks and so would not have influenced the advisor's behavior more than it did their own.
(C) A third involves a bonus that is only awarded if the subject explicitly follows the advice, allowing the subject to directly influence the remuneration of the advisor.
All subjects then complete a demographic survey and a psychological test designed to elicit an index of envy (using the Dispositional Envy Scale - Smith, R. H., Parrott, W. G., Diener, E. F., Hoyle, R. H., & Kim, S. H., 1999) and similar tests of stubbornness, both generally and in relation to the sunk-cost fallacy, drawn from various sources in social science.

The randomization is undertaken by Qualtrics (the software we are using to run the MTurk study).

Individual level only

No

No clustering

Wave 1: approx 75 MTurk users Wave 2: approx 1,500 MTurk users

approx 500 per condition for Wave 2.

Measure: The absolute difference in two proportions p & q. Sample size per condition: 500 Power: 80% Significance Level: 5% The minimum detectable effect size that is detectable regardless of the absolute values of p & q is: 0.0882 i.e., 8.82 percentage points (found where p=0.4559 and q=0.5441 i.e., where (p+q)/2=0.5). For different p & q values, smaller effect sizes can be detectable e.g., in an extreme example with p=0.0001 and q=0.0158, the difference of 0.0157 is detectable with sample sizes of 500 in each condition. Note that lower effect sizes are detectable in the case that the empirical values of p & q are such that (p+q)/2< or >1/2.