Experimental Design
In our experiment, there are two computers and two tasks. One computer is Good, the other is Bad, but the participants do not know which computer is Good. One task is Separating; the other is Pooling. Both computers perform equally well on the Pooling task. On the Separating task, the Good computer performs better than the Bad computer, which allows inferring the computers' quality from their output.
Participants face 10 rounds. Each round consists of two parts. Part 1 is a choice between the Pooling and Separating tasks. The participant's bonus from this part is the amount the two computers produce on the chosen task. Part 2 is a choice between hiring one of the computers to work on a Separating task or getting an outside option based on the amount the computers produce in part 1. In rounds 1 and 10, the parameters of the problem are such that it is optimal to choose the Separating task in part 1. In rounds 2-9, it is optimal to choose the Separating task in half of the rounds and the Pooling task in the other half of the rounds, in randomized order.
There are four treatments. In Baseline treatment, participants make the part 1 choice first. Then, they observe the output of both computers and make the part 2 choice. In the Automatic Inference treatment, participants face the interface except we them which computer is Good and which is Bad if they choose the Separating task and do not tell them if they choose the Pooling task. We add this information to the part 1 question. In the Strategy Method treatment, participants first answer part 2 questions conditional on possible task choices, with inference done for them in the case of Separating tasks. Then, they make their part 1 choice, where they see their possible payoffs from parts 1 and 2 for the two task choices, given their part 2 strategy. In the Planning treatment, participants make part 1 and part 2 choices on the same screen in reverse order: they first see the part 2 question and then the part 1 question.