Minimum detectable effect size for main outcomes (accounting for sample
design and clustering)
Experiment 1
Using data from Tayawa, Ko, and Mahmood (preprint), we calculate several hypothesized coefficients for the effect of the reward level and prior on the chosen level of informativeness. For the effect of a change in reward, the coefficients are 0.1140, 0.1254, 0.1368, and 0.1482 percentage points for priors 0.5, 0.55, 0.6, and 0.65, respectively. For the effect of a change in the prior when the subject is represented by a Prior-Invariant model, the coefficients are -0.0684, -0.1140, -0.1596, -0.2052 percentage points for reward levels 0.3, 0.5, 0.7, and 0.9, respectively.
Experiment 2
In addition to the analysis for identifying inattentive behavior as in Experiment 1, we also test the difference in behavior between parts 1 and 2 by running a joint significance test on the change in the slope and intercept parameters for the effect of the prior on the chosen level of informativeness.
We also provide additional analysis to identify weakly responsive levels of informativeness in relation to reward. Specifically, we calculate a region of practical equivalence for the estimated coefficient representing the effect of the reward level on the chosen level of informativeness. This interval is [-0.2439, 0.2439]. This interval is determined by imposing an upper bound on the expected utility loss from deviating from the optimal level of informativeness, assuming weak responsiveness is optimal. In particular, the percentage loss is capped at 5% if the participant deviates by at most 0.02439% from optimal informativeness for every 5 percentage-point increase in the reward. Given the parametrization of the payment rule, the expected earnings are at least $20.00, and therefore, the expected loss is capped at $1.00.
The same argument and calculations apply to determining the region of practical equivalence for the estimated coefficient representing the effect of the prior when classifying the participant to exhibit a behavior consistent with the Uniform Posterior Separable model. In other words, the interval [-0.2439, 0.2439] ensures that the percentage loss is capped at 5% if the participant deviates by at most 0.02439% from optimal informativeness for every 5 percentage-point increase in the prior.
For the analysis of parts 3 and 4, we investigate whether the participant’s choice aligns with consistency in a particular part or is entirely dependent on the frame. This consistency could manifest in either consistently leaning towards direct frame options (part 1) or natural frame options (part 2). We compare this model of consistency to a random choice model (choosing either option with a 50% chance). To provide bounds on how well this random choice model can predict, we simulated it for 102 decisions (34 rounds × 3 decisions per round) at different confidence levels: 90%, 95%, and 99%. For each confidence level, the random choice model can predict at most 59, 61, and 64 options that are consistent with a single frame. Therefore, we use these three threshold numbers of choices consistent with a frame (frame consistency) to reject the random choice model in favor of consistency, which can be weakly or strongly favored. Additionally, we consider the possibility that the participant’s choice aligns with a frame (either part 1 or 2), conditional on their current frame (frame dependence). We use the same threshold numbers of choices that align to a specific frame conditional on the current frame (in part 3 or 4) to reject the random choice model in favor of frame dependence.
Finally, we use our secondary measures—frame easiness, preference, and distinctiveness—to gain further insights into changes in information acquisition behavior between parts 1 and 2, as well as the behavior on frame consistency or dependence in parts 3 and 4.