Mental Models and Learning in Complex Environments

The experiment employs a 2 × 2 between-subjects design with four treatments that differ in the information environment. The first two treatments are replications of the binary-states experiment by Esponda et al. (2024), while the last two are their analogues with an additional state.

(1) SP treatment (Simple environment with Primitives). Subjects are provided with the following messages:
• Of the 100 projects, there are 15 projects that are successes and 85 projects that arefailures.
• The test result can be either Positive or Negative and has a reliability of 80 %. It means that:
– If the project is a Success, the test result will be Positive with 80 % chance and the test result will be Negative with 20 % chance.
– If the project is a Failure, the test result will be Negative with 80 % chance and the test result will be Positive with 20 % chance.
(2) SNP treatment (Simple environment without Primitives). Subjects are provided with the following messages:
• Of the 100 projects, a certain number of them are successes and the remaining ones are failures. But we will not tell you how many of them are successes or how many arefailures.
• The test result can be either Positive or Negative and has a reliability of R %. It means that:
– If the project is a Success, the test result will be Positive with R % chance andthe test result will be Negative with (100 − R) % chance.
– If the project is a Failure, the test result will be Negative with R % chance andthe test result will be Positive with (100 − R) % chance.
• The reliability R is a speciffc numbers between 0 and 100, but we will not tell you this number.
(3) CP treatment (Complex environment with Primitives). Subjects are provided with the following messages:
• Of the 100 projects, there are 10 projects that are successes, 25 projects that are neutrals, and 65 projects that are failures.
– If the project is a Success, the test result will be Positive with 80 % chance and the test result will be Negative with 20 % chance.
– If the project is a Neutral, the test result will be Positive with 45 % chance and the test result will be Negative with 55 % chance.
– If the project is a Failure, the test result will be Positive with 15 % chance and the test result will be Negative with 85 % chance.
CNP treatment (Complex environment without Primitives). Subjects are provided with the following messages:
• Of the 100 projects, a certain number of them are successes, a certain number of them are neutrals, and the remaining ones are failures. But we will not tell you how many of them are successes, how many are neutrals, or how many are failures.
– If the project is a Success, the test result will be Positive with R3 % chance and the test result will be Negative with (100 − R3) % chance.
– If the project is a Neutral, the test result will be Positive with R2 % chance and the test result will be Negative with (100 − R2) % chance.
– If the project is a Failure, the test result will be Positive with R1 % chance and the test result will be Negative with (100 − R1) % chance.
• R1 < R2 < R3 are speciffc numbers between 0 and 100, but we will not tell you the exact numbers.

2026-05-28

2026-06-28

Bpossuc: Belief elicited in each round, Pr(Success | Positive)
Bposneu: Belief elicited in each round, Pr(Neutral | Positive)
Bposfail: Belief elicited in each round, Pr(Failure | Positive)
Bnegsuc Belief elicited in each round, Pr(Success | Negative)
Bnegneu Belief elicited in each round, Pr(Neutral | Negative)
Bnegfail Belief elicited in each round, Pr(Failure | Negative)

(1) The absolute value of the distance between the submitted beliefs and the Bayesian benchmarks conditional on a Positive and a Negative test result, respectively.
| Bpossuc_it − Bpossuc^Bay |
| Bnegsuc_it − Bnegsuc^Bay |
| Bposneu_it − Bposfail^Bay |
| Bnegneu_it − Bnegfail^Bay |
| Bposfail_it − Bposfail^Bay |
| Bnegfail_it − Bnegfail^Bay |
(2) In order to improve the comparability of the data between the Simple and Complex environments, we assign a numerical value ω ∈ (0, 1) to each of the possible states,Success, Neutral, and Failure, and then calculate the subjective conditional expectation ofthe resulting random variable. For example, the numerical value assigned to state Success equals the probability of generating a Positive test result when the selected project is a Success.
(3)following Ba et al. (2025), we will measure the distortion from normative Bayesian benchmarks by comparing subjective and objective movements in beliefs in terms of the expected state.

In the main task, subjects in the CP and CNP treatments will be asked to assess the probabilities that a project is a Success, a Neutral, or a Failure for each possible test result (Positive or Negative); subjects in the SP and SNP treatments will be asked to assess the probabilities that a project is a Success or a Failure for each possible test result (Positive or Negative). In Stage 5, we will ask subjectsto recall the feedback they received during rounds 1–200 of the main task.
· Stage 1, round 001: Tutorial. Introduce the BDM method and strategy method using simple examples.
· Stage 2, round 001: The Main task.
· Stage 3, round 002–100: Learning: Repetition of the main task. Beliefelicited at every single round.
· Stage 4, round 101–200: Learning: Repetition of the main task. Beliefelicited at every 10th round.
· Stage 5, round 201: Recollection of feedback.
· Stage 6, round 202: Summary tables. True feedback statistics revealed. Main task.
· Stage 7, round 203: Summary tables. 1000-round statistics (200 real+ 800 simulated) revealed. Main task.
· Stage 8, round 204: Summary tables. Frequency derived from the1000-round statistics revealed. Main task.
· Stage 9, round 205: Transfer of learning. Primitives changed. Main task.
· Survey.

by a computer

Treatment assignment occurs at the session level (between-subjects).

Yes

We will run 3 sessions per treatment with 20 subjects per session, yielding 60 subjects per treatment and a total sample size of 240.

With 205 rounds per session, this generates 12,300 subject-round observations per treatment.

60 subjects SP (Simple environment with Primitives), 60 subjects SNP (Simple environment without Primitives), 60 subjects CP (Complex environment with Primitives), 60 subjects CNP (Complex environment without Primitives).

We assume a two-tailed test with aType I error rate of 0.05, and the anticipated effect size for the interaction coefficient is specified as a standardized regression coefficient of 0.2, tested against a null value of 0. The overall model R^2 is set to 0.3. (1) The effects of providing: Primitives (SP versus SNP, CP versus CNP, respectively) The sample size is 120. Under these parameters, the computedType II error probability was 0.262, yielding a statistical power of 0.738. Hence, our studyhas approximately 73.8 % power to detect a non-zero coeffffcient in these models. (2) The effects of increasing the complexity of information environment: The sample size is 240 (all four treatments). Under these parameters, the computedType II error probability was 0.042, yielding a statistical power of 0.958. It indicates that,given the speciffed effect size and sample size, our study has approximately 95.8 % power todetect a non-zero coeffffcient at the 0.05 signiffcance level.

Pre-Analysis Plan: Mental Models and Learning in Complex Environments