Illusion of Control in a Complex Environment

Last registered on May 17, 2024

Pre-Trial

Trial Information

General Information

Title
Illusion of Control in a Complex Environment
RCT ID
AEARCTR-0013367
Initial registration date
April 15, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
April 16, 2024, 3:52 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
May 17, 2024, 5:23 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
University of Bonn

Other Primary Investigator(s)

Additional Trial Information

Status
Completed
Start date
2024-04-16
End date
2024-05-16
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
I study the effect of how players in a board game generate their rolls on their beliefs of certain winning probabilities in the board game. In particular, it is varied whether players use a tablet computer vs. whether they use a physical die and a dice shuffler to generate their die rolls. The study is implemented as an RCT laboratory experiment.
External Link(s)

Registration Citation

Citation
Laubel, Alexander. 2024. "Illusion of Control in a Complex Environment." AEA RCT Registry. May 17. https://doi.org/10.1257/rct.13367-1.1
Experimental Details

Interventions

Intervention(s)
Laboratory experiments to study the effect of whether a die roll is generated by a computer vs. by physically rolling a die on beliefs of winning probabilities in a board game.
Intervention (Hidden)
The experimental study employs subjects in three roles: Player1, Player2 and Observer. Player1 and Player2 provide beliefs of they themselves winning, Observer provides beliefs of Player1 winning. Each subject provides these beliefs for five different game situations that systematically vary the extent to which luck vs. skill matters. Beliefs are incentivized; at the end of each session, three randomly selected subjects (one in each role) and one randomly selected game situation are actually played and incentivization is based on this implemented game.

Between-subjects (between sessions) it is varied how Player1 generate his / her die rolls. In "Treated," Player1 uses a physical die and a dice shuffler. In "Control," Player1 uses a die roll computer. Player2 always uses a die roll computer. (In "Control" both players use separate die roll computers.)

A related experiment has been pre-registered at the AEA RCT Registry (AEARCTR-0008469). This first experiment employed a similar design and studied a closely related research question. The author is happy to share the data upon request. Crucially however, the implementation of this first design was incredibly time-consuming: 110 sessions in total were needed for two treatment conditions with 55 subjects per treatment arm and role. The current design improves upon this observation-per-cost ratio greatly: for the most important role, it is possible to collect 1,250 observations (625 in each treatment arm clustered on the subject level as 5 observations per 125 subjects) in only approx. 20-22 sessions (depending on show-up rates).

For more details, see the pre-analysis plan.
Intervention Start Date
2024-04-16
Intervention End Date
2024-05-16

Primary Outcomes

Primary Outcomes (end points)
Subjects' beliefs (incentivized) of the probability that a certain player wins a board game given a specific game situation. Each subject provides 5 such beliefs (for each situation one belief).
For more details, see the attached pre-analysis plan.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
For each game situation, subjects additionally provide:
- Belief about their confidence in their probability estimate
- Belief about the extent to which skill vs. luck matters in a given situation.

"Tertiary" outcomes:
- Subjects' belief about importance of winning.
- Subjects' belief about how often a certain number is rolled when the experimenter uses their die rolling device six times.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
In the BonnEconLab at the University of Bonn, I will conduct a laboratory experiment in April 2024. The timeline of one session is as follows:
1. Before subjects enter the laboratory, one of two treatment conditions is randomly determined (Treated vs. Control). Approximately 25 subjects enter the laboratory, are randomly assigned to a role (Player1, Player2, Observer), and are seated in individual cubicles. Treatment variation concerns how Player1 generates die rolls: with a physical die (Treated) vs. with a die roll computer (Control); Player2 always uses a separate die roll computer. (The Observer role will serve as a control role, providing beliefs for the same events as Player1.)
2. Subjects complete the experiment individually in their cubicles, involving the following steps:
a. Subjects read experimental instructions as well as the rules of the board game “Ludo” and complete comprehension questions for both. (The board game is highly popular in Germany and nicely combines experimental control with a natural setting.)
b. Subjects provide three beliefs each for five different game situations (first three beliefs for the first situation, then three beliefs for the second situation etc.). The order of the five situations is randomized, but within each situation, the belief elicitation order of the three beliefs is as follows: belief of winning (B1 – B5) (for Player1 and Player2: of themselves winning, respectively; for Observer: of Player1 winning); confidence in belief of winning (Confidence1 – Confidence5); belief of the extent to which skill vs. luck matters in a given situation (Control1 – Control5). B1 – B5 is the main outcome of the experiment.
c. Subjects provide the same three beliefs for a sixth situation (B6, Confidence6, Control6) with the one difference being that the event is not winning the board game (which is highly complex or even impossible to calculate, depending on the game situation) but rolling a certain number in the next die roll (simple and straightforward to calculate).
d. Subjects provide additional data: belief about the importance of winning (Importance); decision which of two pieces to move for three separate situations (serving as “Ludo puzzles” measuring strategic understanding of the game, used for exclusion restriction later); belief how often a certain number is rolled if the experimenter uses the die / die roll computer (DieRollBet).
e. Subjects provide additional data: familiarity with the game; gender; religiosity; automation preferences; belief about whether cheating in the game was possible.
3. When all subjects have completed the survey, three subjects from this session are randomly selected (one of each role), and one game situation is randomly selected. This game situation is then implemented with the randomly selected subjects: Player1 and Player2 actually play against each other, while Observer watches a live stream in a separate room. Simultaneously, the other subjects receive their compensation in another room and are then dismissed. After the game is over, the three randomly selected subjects also receive their compensation and are then dismissed.
Experimental Design Details
Randomization Method
Coin flip
Randomization Unit
Experimental session
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
20-22 sessions (depending on show-up).
Sample size: planned number of observations
I obtained 6,462.50 € funding such that after pre-registered exclusion criteria (see pre-analysis plan for exclusion criteria), there should be approx. 250 subjects in role Player1 and 100 subjects in role Observer left. Total funding is sufficient for approx. 517 subjects. This includes subjects excluded from data analysis and approx. 50 subjects in the role "Player2" that are not relevant for data analysis but only to make the sessions implementable (see experimental details). I will collect data until the 6,462.50 € are exhausted.
Sample size (or number of clusters) by treatment arms
10-11 sessions Treated, 10-11 sessions Control (depending on show-up).
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
The primary outcome measure is the belief regarding the probability of winning, pooled across all five situations (B1 - B5) for Player1. Power analysis indicates that with 250 subjects, a treatment effect of 2 percentage points can be detected with a statistical power of 0.80 at a significance level of 0.05.
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials