|
Field
Last Published
|
Before
May 02, 2025 05:49 AM
|
After
April 19, 2026 01:29 AM
|
|
Field
Intervention (Public)
|
Before
The design varies several variables regarding the decision problem of the subject. Each subject is invited to participate in three sessions to do 168 round of a simple guessing task where the guess whether or not the randomly chosen urn for a given round contains a hexagon. Subjects are allowed to gather information about the chosen urn by taking out polygons from the chosen urn. We ask how much information they want to gather, and we vary the frame regarding this question between the three sessions. Within a session, we vary the reward for guessing the chosen urn across the rounds; and we also vary the probability the chosen urn has a hexagon across the rounds. Two rounds are randomly chosen for payment at the end of each session. Please refer to the succeeding research design and study procedures for the details and purpose of variations.
We use different frames and provide more information in succeeding treatments to observe changes in the intended posterior distributions.
|
After
Experiment 1
The design varies several variables regarding the decision problem of the subject. Each subject is invited to participate in three sessions to do 168 round of a simple guessing task where the guess whether or not the randomly chosen urn for a given round contains a hexagon. Subjects are allowed to gather information about the chosen urn by taking out polygons from the chosen urn. We ask how much information they want to gather, and we vary the frame regarding this question between the three sessions. Within a session, we vary the reward for guessing the chosen urn across the rounds; and we also vary the probability the chosen urn has a hexagon across the rounds. Two rounds are randomly chosen for payment at the end of each session. Please refer to the succeeding research design and study procedures for the details and purpose of variations.
We use different frames and provide more information in succeeding treatments to observe changes in the intended posterior distributions.
Experiment 2
Experiment 2 consists of four parts, following the same decision problem in Experiment 1. Similarly, the design varies several variables regarding the decision problem of the subject. Each subject will do 92 rounds of a simple guessing task where they guess whether or not the randomly chosen urn for a given round contains a hexagon. Subjects are allowed to gather information by taking out polygons from the chosen urn. In all parts, we ask how much information they want to gather, and we vary the frame regarding this question between the first two parts. We vary the reward for guessing the chosen urn across the rounds, and we also vary the probability that the chosen urn has a hexagon across the rounds. In parts 3 and 4, they face similar questions to those in the first half and choose options for levels of information based on their answers in the first two parts. Two rounds are randomly chosen for payment at the end of each session.
|
|
Field
Primary Outcomes (End Points)
|
Before
Random Posterior Distribution: the possible set of beliefs and their corresponding probability over the state space. Elicited as the intended probability that their guess would be correct.
Information Structure: a collection of conditional probabilities over the signal space given the state. Elicited as the number of polygons to be drawn from the unknown urn.
Guess: the subject's guess given their realized signal.
|
After
Random Posterior Distribution: the possible set of beliefs and their corresponding probability over the state space. Elicited as the intended probability that their guess would be correct.
Information Structure: a collection of conditional probabilities over the signal space given the state. Elicited as the number of polygons to be drawn from the unknown urn.
Guess: the subject's guess given their realized signal.
Choice of level of information based on previous answers: some questions about desired informativeness (in terms of random posterior distributions or information structures) may involve options based on the participant's previous answers.
|
|
Field
Experimental Design (Public)
|
Before
The design varies several variables regarding the decision problem of the subject. Each subject is invited to participate in three sessions to do 168 round of a simple guessing task where the guess whether or not the randomly chosen urn for a given round contains a hexagon. Subjects are allowed to gather information about the chosen urn by taking out polygons from the chosen urn. We ask how much information they want to gather, and we vary the frame regarding this question between the three sessions. Within a session, we vary the reward for guessing the chosen urn across the rounds; and we also vary the probability the chosen urn has a hexagon across the rounds. Two rounds are randomly chosen for payment at the end of each session. Please refer to the succeeding research design and study procedures for the details and purpose of variations.
|
After
Experiment 1
The design varies several variables regarding the decision problem of the subject. Each subject is invited to participate in three sessions to do 168 round of a simple guessing task where the guess whether or not the randomly chosen urn for a given round contains a hexagon. Subjects are allowed to gather information about the chosen urn by taking out polygons from the chosen urn. We ask how much information they want to gather, and we vary the frame regarding this question between the three sessions. Within a session, we vary the reward for guessing the chosen urn across the rounds; and we also vary the probability the chosen urn has a hexagon across the rounds. Two rounds are randomly chosen for payment at the end of each session. Please refer to the succeeding research design and study procedures for the details and purpose of variations.
Experiment 2
Experiment 2 consists of four parts, following the same decision problem in Experiment 1. Similarly, the design varies several variables regarding the decision problem of the subject. Each subject will do 92 rounds of a simple guessing task where they guess whether or not the randomly chosen urn for a given round contains a hexagon. Subjects are allowed to gather information by taking out polygons from the chosen urn. In all parts, we ask how much information they want to gather, and we vary the frame regarding this question between the first two parts. We vary the reward for guessing the chosen urn across the rounds, and we also vary the probability that the chosen urn has a hexagon across the rounds. In parts 3 and 4, they face similar questions to those in the first half and choose options for levels of information based on their answers in the first two parts. Two rounds are randomly chosen for payment at the end of each session.
|
|
Field
Planned Number of Clusters
|
Before
100 university students
|
After
Experiment 1: 100 university students
Experiment 2: 100 university students
|
|
Field
Planned Number of Observations
|
Before
50400 decision rounds from 100 university students over three session
|
After
Experiment 1: 50400 decisions from 100 university students over three session
Experiment 2: 27600 decisions from 100 university students
|
|
Field
Sample size (or number of clusters) by treatment arms
|
Before
100 university students invited for three sessions
|
After
Experiment 1: 100 university students invited for three sessions
Experiment 2: 100 university students
|
|
Field
Power calculation: Minimum Detectable Effect Size for Main Outcomes
|
Before
Using data from Ko, Mahmood, Tayawa (2025, under review), we calculate several hypothesized coefficients for the effect of the reward level and prior on the chosen level of informativeness. For the effect of a change in reward, the coefficients are 0.1140, 0.1254, 0.1368, and 0.1482 percentage points for priors 0.5, 0.55, 0.6, and 0.65, respectively. For the effect of a change in the prior when the subject is represented as by a Prior-Invariant model, the coefficients are -0.0684, -0.1140, -0.1596, -0.2052 percentage points for reward levels 0.3, 0.5, 0.7, and 0.9, respectively.
|
After
Experiment 1
Using data from Tayawa, Ko, and Mahmood (preprint), we calculate several hypothesized coefficients for the effect of the reward level and prior on the chosen level of informativeness. For the effect of a change in reward, the coefficients are 0.1140, 0.1254, 0.1368, and 0.1482 percentage points for priors 0.5, 0.55, 0.6, and 0.65, respectively. For the effect of a change in the prior when the subject is represented by a Prior-Invariant model, the coefficients are -0.0684, -0.1140, -0.1596, -0.2052 percentage points for reward levels 0.3, 0.5, 0.7, and 0.9, respectively.
Experiment 2
In addition to the analysis for identifying inattentive behavior as in Experiment 1, we also test the difference in behavior between parts 1 and 2 by running a joint significance test on the change in the slope and intercept parameters for the effect of the prior on the chosen level of informativeness.
We also provide additional analysis to identify weakly responsive levels of informativeness in relation to reward. Specifically, we calculate a region of practical equivalence for the estimated coefficient representing the effect of the reward level on the chosen level of informativeness. This interval is [-0.2439, 0.2439]. This interval is determined by imposing an upper bound on the expected utility loss from deviating from the optimal level of informativeness, assuming weak responsiveness is optimal. In particular, the percentage loss is capped at 5% if the participant deviates by at most 0.02439% from optimal informativeness for every 5 percentage-point increase in the reward. Given the parametrization of the payment rule, the expected earnings are at least $20.00, and therefore, the expected loss is capped at $1.00.
The same argument and calculations apply to determining the region of practical equivalence for the estimated coefficient representing the effect of the prior when classifying the participant to exhibit a behavior consistent with the Uniform Posterior Separable model. In other words, the interval [-0.2439, 0.2439] ensures that the percentage loss is capped at 5% if the participant deviates by at most 0.02439% from optimal informativeness for every 5 percentage-point increase in the prior.
For the analysis of parts 3 and 4, we investigate whether the participant’s choice aligns with consistency in a particular part or is entirely dependent on the frame. This consistency could manifest in either consistently leaning towards direct frame options (part 1) or natural frame options (part 2). We compare this model of consistency to a random choice model (choosing either option with a 50% chance). To provide bounds on how well this random choice model can predict, we simulated it for 102 decisions (34 rounds × 3 decisions per round) at different confidence levels: 90%, 95%, and 99%. For each confidence level, the random choice model can predict at most 59, 61, and 64 options that are consistent with a single frame. Therefore, we use these three threshold numbers of choices consistent with a frame (frame consistency) to reject the random choice model in favor of consistency, which can be weakly or strongly favored. Additionally, we consider the possibility that the participant’s choice aligns with a frame (either part 1 or 2), conditional on their current frame (frame dependence). We use the same threshold numbers of choices that align to a specific frame conditional on the current frame (in part 3 or 4) to reject the random choice model in favor of frame dependence.
Finally, we use our secondary measures—frame easiness, preference, and distinctiveness—to gain further insights into changes in information acquisition behavior between parts 1 and 2, as well as the behavior on frame consistency or dependence in parts 3 and 4.
|
|
Field
Intervention (Hidden)
|
Before
Each subject repeatedly face the simple task of guessing between two urns: Hexagon and No Hexagon Urn. Each urn has 30 polygons. In the No Hexagon Urn, all 30 polygons are septagons (seven-sided polygon). In the Hexagon Urn, there is one hexagon (six-sided polygon), and 29 polygons are septagons. At the beginning of each round, the computer will randomly select one the Hexagon Urn with probability p% (prior). The DM is rewarded $20 with probability r% if they guess correctly and zero otherwise.
They can inspect the urn before making their guess. They do so by drawing polygons from the assigned urn without replacement. For each draw, a polygon appears for 0.7 seconds, which then disappears, followed by another polygon after 0.3 seconds. The number of polygons drawn is determined by the subject’s chosen level of informativeness. The manner of asking this level of informativeness is determined by the frame, which is either the direct or the natural frame.
In the natural frame, it is straightforward to ask the subjects how many they want to draw from the assigned urn. In the direct frame, we elicit the intended random posterior by asking the following "How likely do you want the guess NH to be correct in case no hexagon appears?"
Notice that when the subject 'inspects the urn', there are two possible cases: either (1) they observe a hexagon or (2) they don't observe a hexagon. In the case that they observe a hexagon, the subject is 100% certain that the assigned urn is the Hexagon Urn. In the other case that they do not observe a hexagon, the subject is NOT 100% certain that the assigned urn is No Hexagon since both urns contain mostly pentagons. Thus, their answer for the level of certainty fully reveals their intended random posterior.
Once they report their intended random posterior, we will translate their chosen level of certainty to the appropriate number of polygons to be drawn to achieve that level. Note that increasing the level of certainty means increasing the number of polygons to inspect from the assigned urn. By choosing 100% certainty, the subject elects to inspect all polygons from the assigned urn.
Each subject participates in three separate sessions, each consisting of 168 rounds of urn-guessing tasks. In each session, we vary the reward level r% from 30% to 90%, and prior p% from 50% to 85%.
The first session implements the direct frame, and the second and the last sessions implement the natural frame. For the robustness check, the last session provides the exact Bayesian mapping between the number of draws n and the random posterior when subjects decide on the number of draws, while the second session does not.
|
After
Experiment 1
Each subject repeatedly face the simple task of guessing between two urns: Hexagon and No Hexagon Urn. Each urn has 30 polygons. In the No Hexagon Urn, all 30 polygons are septagons (seven-sided polygon). In the Hexagon Urn, there is one hexagon (six-sided polygon), and 29 polygons are septagons. At the beginning of each round, the computer will randomly select one the Hexagon Urn with probability p% (prior). The DM is rewarded $20 with probability r% if they guess correctly and zero otherwise.
They can inspect the urn before making their guess. They do so by drawing polygons from the assigned urn without replacement. For each draw, a polygon appears for 0.7 seconds, which then disappears, followed by another polygon after 0.3 seconds. The number of polygons drawn is determined by the subject’s chosen level of informativeness. The manner of asking this level of informativeness is determined by the frame, which is either the direct or the natural frame.
In the natural frame, it is straightforward to ask the subjects how many they want to draw from the assigned urn. In the direct frame, we elicit the intended random posterior by asking the following "How likely do you want the guess NH to be correct in case no hexagon appears?"
Notice that when the subject 'inspects the urn', there are two possible cases: either (1) they observe a hexagon or (2) they don't observe a hexagon. In the case that they observe a hexagon, the subject is 100% certain that the assigned urn is the Hexagon Urn. In the other case that they do not observe a hexagon, the subject is NOT 100% certain that the assigned urn is No Hexagon since both urns contain mostly pentagons. Thus, their answer for the level of certainty fully reveals their intended random posterior.
Once they report their intended random posterior, we will translate their chosen level of certainty to the appropriate number of polygons to be drawn to achieve that level. Note that increasing the level of certainty means increasing the number of polygons to inspect from the assigned urn. By choosing 100% certainty, the subject elects to inspect all polygons from the assigned urn.
Each subject participates in three separate sessions, each consisting of 168 rounds of urn-guessing tasks. In each session, we vary the reward level r% from 30% to 90%, and prior p% from 50% to 85%.
The first session implements the direct frame, and the second and the last sessions implement the natural frame. For the robustness check, the last session provides the exact Bayesian mapping between the number of draws n and the random posterior when subjects decide on the number of draws, while the second session does not.
Experiment 2
The decision problem used in Experiment 1 is implemented in Experiment 2. We similarly vary the prior p and reward r, but restricting r to be between 50 and 90 percent. There are 92 rounds of decision-making across four parts.
In parts 1 and 2, subjects face a similar protocol as in Experiment 1, wherein the Direct Frame (decision variable: accuracy level of inspection) is used for part 1 and the Natural Frame (decision variable: size of sampling) for part 2. Both parts consist of 29 rounds. In each decision round, the subject faces a fixed prior p and is asked to specify their desired level of information for inspecting three urns with three corresponding reward levels: 50, 70, and 90 percent. The computer randomly chooses one of the three urns and uses the specified level of information to determine how many polygons to take out from the selected urn in the round.
There are 17 rounds each for parts 3 and 4. Each round will also have a fixed prior p and three urns corresponding to the three reward levels. However, the choice of informativeness is now restricted to two options: the exact information levels the subject previously selected in Parts 1 and 2 under the same p and r. Particularly, for a given prior p and reward r, subjects are asked to choose between their two previous answers from parts 1 and 2, with the interface explicitly indicating which option originated from which part. Part 3 follows the framing of part 1, while part 4 follows part 2.
Across all parts, to mitigate and potentially eliminate the effects of non-Bayesian updating, we provide interface tools that allow subjects to check the relationship between the decision variables in each frame. Additionally, to control for potential order effects, we counterbalance the order of Parts 1 and 2, as well as Parts 3 and 4, across sessions.
|
|
Field
Secondary Outcomes (End Points)
|
Before
|
After
Experiment 2
Frame easiness: On a scale of 0-10 (0 extremely difficult to 10 extremely easy), we ask the easiness of answering their desired informativeness given the frame.
Frame preference: We ask which frame is preferred over the other, allowing for indifference.
Frame distinctiveness: On a scale of 0-10 (0 not being different to 10 extremely different), we ask how much they feel that they treat the two frames differently.
|