x

NEW UPDATE: Completed trials may now upload and register supplementary documents (e.g. null results reports, populated pre-analysis plans, or post-trial results reports) in the Post Trial section under Reports, Papers, & Other Materials.
Analysis of player decisions in a mobile game to study the influence of counterfactual thinking on insurance demand
Last registered on June 18, 2020

Pre-Trial

Trial Information
General Information
Title
Analysis of player decisions in a mobile game to study the influence of counterfactual thinking on insurance demand
RCT ID
AEARCTR-0005996
Initial registration date
June 11, 2020
Last updated
June 18, 2020 9:51 AM EDT
Location(s)
Region
Region
Region
Region
Region
Region
Region
Region
Region
Region
Region
Region
Region
Region
Region
Region
Region
Region
Region
Region
Region
Region
Region
Region
Region
Region
Region
Region
Region
Region
Region
Region
Primary Investigator
Affiliation
LMU Munich
Other Primary Investigator(s)
Additional Trial Information
Status
In development
Start date
2020-06-19
End date
2020-07-18
Secondary IDs
Abstract
We analyze the impact of counterfactual thinking on the risk management decisions of players of a mobile game. In our experiment, individuals engage in risky lotteries that can lead to losses of their in-game wealth. To prevent a loss, individuals can purchase insurance coverage. It is well known that individuals adapt their behavior following a factual outcome (e.g., individuals exert a higher level of risk aversion after a loss). However, it is less known how individuals react to receiving the information of how close they were to avoiding or suffering a loss. We additionally examine whether counterfactual information affect the self-reported satisfaction of the preceding period’s insurance demand. The field experiment is incentive compatible and allows for tests against the null hypothesis that counterfactual thinking does not affect risk taking behavior.
External Link(s)
Registration Citation
Citation
Strümpel, Dennis. 2020. "Analysis of player decisions in a mobile game to study the influence of counterfactual thinking on insurance demand." AEA RCT Registry. June 18. https://doi.org/10.1257/rct.5996-3.0.
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details
Interventions
Intervention(s)
Intervention Start Date
2020-06-19
Intervention End Date
2020-07-18
Primary Outcomes
Primary Outcomes (end points)
We examine the difference in insurance demand for the treatment and control group. Since the control group doesn't receive any information regarding the counterfactual closeness, we can reject that counterfactual thinking don't affect insurance demand when we observe systematically different insurance choices between the two groups after the same factual outcome (loss, null).

We also examine the difference in the self-reported satisfaction of the players insurance demand for the treatment and control group. Since the control group doesn't receive any information regarding the counterfactual closeness, we can evaluate how this information affects the self-reported satisfaction for the same factual outcome (loss or null) and insurance coverage.
Primary Outcomes (explanation)
The task of the participants is to select one of 12 boxes. The participants are informed that the boxes are either empty or contain a bomb. As we analyze decisions under risk, the distribution of empty boxes and bombs is shown to the participants. However, the participants do not know the content of a specific box until it has been opened. If the participants choose a box with a bomb, they lose some of their in-game currency. In contrast, choosing an empty box does not cause a loss. For individuals in the control group, only the content of the selected box is revealed. For individuals in the treatment group, the contents of both adjacent boxes are also disclosed.
We define an outcome as “clear” if the revealed content of both adjacent boxes is equal to the content of the box the individual selected (e.g., “clear loss” means that an individual selects a box that contains a bomb and receives the information that both adjacent boxes also contained a bomb). We define an outcome as “close” if at least one of the adjacent boxes would have resulted in a different outcome for the individual (e.g., “close loss” means that the individual selects a box that contains a bomb and receives the information that both adjacent boxes were empty).
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
The task of the participants is to select one of 12 boxes. The participants are informed that the boxes are either empty or contain a bomb. As we analyze decisions under risk, the distribution of empty boxes and bombs is shown to the participants. However, the participants do not know the content of a specific box until it has been opened. If the participants choose a box with a bomb, they lose some of their in-game currency. In contrast, choosing an empty box does not cause a loss.
Experimental Design Details
The experiment is divided into two phases: In the first phase, the participants select a box without the option to purchase insurance. In the second phase, before selecting a box, all participants have the opportunity to purchase insurance coverage in 20% increments that reduces the loss if a box containing a bomb is opened. This two phase desig allows us to study the influence of the outcome of phase 1 on the insurance demand in phase 2 and enables the analysis of the difference between the treatment and control group for the same factual outcomes. After the factual (and counterfactual for the treatment group) outcomes are revealed after the second phase, players are asked how satisfied they were with their insurance purchase. Before the start of phase 1, the computer randomly chooses one of four lotteries A = {l, b/12, 0} with l ∈ L = {-60, -120} and b ∈ B = {4, 6}. The selected lottery is used for both phases, with the only difference that the player can buy insurance in the second phase. The insurance premium in phase 2 is equivalent to the actuarially fair price times the randomly determined loading factor v with v ∈ V = {1.0, 1.5}. Analysis A1 is descriptive. We plot the insurance demand of both groups in dependence of the factual (loss, null) and counterfactual (clear-loss, close-loss, clear-null, close-null) outcomes for each of the lotteries. Analysis A2 is inductive. We use an OLS model to calculate the differences between both groups insurance demand (dependent variable). We use the factual and counterfactual information as the independent variables. The model additionally includes controls (self-reported age, self-reported gender, country, in-game wealth). The analysis also includes fixed effects for the different lotteries offered to the subjects. Standard errors are heteroscedasticity-robust and clustered on the level of the subject. Analysis A3 is descriptive. We plot the average self-reported satisfaction of both groups insurance demand in dependence of the factual (loss, null) and counterfactual (clear-loss, close-loss, clear-null, close-null) outcomes over all lotteries. For this, we bin their insurance demand in phase 1 in steps of 20. Analysis A4 is inductive. We use an OLS model to calculate the differences between both groups self-reported satisfaction with their insurance demand (dependent variable). We use the factual and counterfactual information as well as their previous insurance demand as the independent variables. The model additionally includes controls (self-reported age, self-reported gender, country, in-game wealth). The analysis also includes fixed effects for the different lotteries offered to the subjects. Standard errors are heteroscedasticity-robust and clustered on the level of the subject. Moreover, Analysis A2 and A4 include one additional indicator. The indicator equals one if the loss outcome results in a previously affordable item no longer being obtainable. This aims to control for systematically different choices at levels of wealth that directly affect the subjects options to unlock game content. We additionally collect a variety of in-game data, which are potential interesting control variables for inductive multivariate tests. However, we will not interact them with the independent variables because this does not address our main research question.
Randomization Method
Subjects are randomly allocated to the control or treatment group when agreeing to the consent form. 25% of the subjects are allocated to the control group. 75% of the subjects are allocated to the treatment group.
The lotteries will show up everytime a player completes at least three levels in three different runs within the same game session and are randomly chosen by the computer.
Randomization Unit
The randomization unit of the group allocation is the individual subject.
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
Not applicable to our case (see above).
Sample size: planned number of observations
Our setting deviates from traditional experiments in the sense that the number of downloads in Google’s Play Store determines the number of subjects in our experiment. We will upload the game to Google’s Play Store on 12 June 2020 and expect the game to be public on the same day (delays are possible due to Google's processing). We will additionally upload a promotional video to YouTube on 19 June 2020 (this date is, however, subject to discretion of the promoting influences). We expect only a small number of game downloads until the release of the promotional video and thus see the date of the promotional video as our trial start. For the analyses described, we only use data from the first 30 days after the release of the promotional video. The sample size of our experiment is therefore dependent on the number of downloads from Google’s Play Store and the game’s retention rate. Thus, the sample size is not predictable at this stage. We will additionally filter our data as described below and provide for each filter corresponding reason in parentheses: We remove players that are uniquely identifiable based on the collected demographic variables (self-reported age, self-reported gender, and country). [data protection law] We are only allowed to process the choice of players that specify an age greater or equal to 16. [data protection law] The lotteries will show up after every second run a player completes at least one level within the same game session[experimental design]. Example 1: Session 1: Run 1: Completed Levels: 1 Session 1: Run 2: Completed Levels: 0 Session 1: Run 3: Completed Levels: 2 Session 1: Lottery is displayed to the player. Session 1: Run 4: Completed Levels: 5 Session 1: Run 5: Completed Levels: 3 Session 1: Lottery is displayed to the player. Example 2: Session 1: Run 1: Completed Levels: 2 Session 1: Run 2: Completed Levels: 3 Session 1: Lottery is displayed to the player. Session 2: Run 1: Completed Levels: 5 Session 3: Run 1: Completed Levels: 3 Session 3: Run 2: Completed Levels: 3 Session 3: Lottery is displayed to the player.
Sample size (or number of clusters) by treatment arms
We have two treatment arms. We allocate 25% of the subjects in the control group and 75% into the treatment group.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
For our aggregated analysis, a power test is not applicable. The sample size is dependent on the number of downloads from Google’s Play Store and the game’s retention rate and is, thus, not predictable at this stage.
Supporting Documents and Materials

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
See the attached document "Comment_IRB_Approval"
IRB Approval Date
2020-03-15
IRB Approval Number
N/A
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
No
Is data collection complete?
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports, Papers & Other Materials
Relevant Paper(s)
REPORTS & OTHER MATERIALS