Analysis of player decisions in a mobile game to study the influence of counterfactual thinking on insurance demand

2020-06-19

2020-07-18

We examine the difference in insurance demand for the treatment and control group. Since the control group doesn't receive any information regarding the counterfactual closeness, we can reject that counterfactual thinking don't affect insurance demand when we observe systematically different insurance choices between the two groups after the same factual outcome (loss, null).

We also examine the difference in the self-reported satisfaction of the players insurance demand for the treatment and control group. Since the control group doesn't receive any information regarding the counterfactual closeness, we can evaluate how this information affects the self-reported satisfaction for the same factual outcome (loss or null) and insurance coverage.

The task of the participants is to select one of 12 boxes. The participants are informed that the boxes are either empty or contain a bomb. As we analyze decisions under risk, the distribution of empty boxes and bombs is shown to the participants. However, the participants do not know the content of a specific box until it has been opened. If the participants choose a box with a bomb, they lose some of their in-game currency. In contrast, choosing an empty box does not cause a loss. For individuals in the control group, only the content of the selected box is revealed. For individuals in the treatment group, the contents of both adjacent boxes are also disclosed.
We define an outcome as “clear” if the revealed content of both adjacent boxes is equal to the content of the box the individual selected (e.g., “clear loss” means that an individual selects a box that contains a bomb and receives the information that both adjacent boxes also contained a bomb). We define an outcome as “close” if at least one of the adjacent boxes would have resulted in a different outcome for the individual (e.g., “close loss” means that the individual selects a box that contains a bomb and receives the information that both adjacent boxes were empty).

The task of the participants is to select one of 12 boxes. The participants are informed that the boxes are either empty or contain a bomb. As we analyze decisions under risk, the distribution of empty boxes and bombs is shown to the participants. However, the participants do not know the content of a specific box until it has been opened. If the participants choose a box with a bomb, they lose some of their in-game currency. In contrast, choosing an empty box does not cause a loss.

The experiment is divided into two phases: In the first phase, the participants select a box without the option to purchase insurance. In the second phase, before selecting a box, all participants have the opportunity to purchase insurance coverage in 20% increments that reduces the loss if a box containing a bomb is opened. This two phase desig allows us to study the influence of the outcome of phase 1 on the insurance demand in phase 2 and enables the analysis of the difference between the treatment and control group for the same factual outcomes.
After the factual (and counterfactual for the treatment group) outcomes are revealed after the second phase, players are asked how satisfied they were with their insurance purchase.

Before the start of phase 1, the computer randomly chooses one of four lotteries A = {l, b/12, 0} with l ∈ L = {-120, -240} and b ∈ B = {4, 6}. The selected lottery is used for both phases, with the only difference that the player can buy insurance in the second phase. The insurance premium in phase 2 is equivalent to the actuarially fair price times the randomly determined loading factor v with v ∈ V = {1.0, 1.5}.



Analysis A1 is descriptive. We plot the insurance demand of both groups in dependence of the factual (loss, null) and counterfactual (clear-loss, close-loss, clear-null, close-null) outcomes for each of the lotteries.

Analysis A2 is inductive. We use an OLS model to calculate the differences between both groups insurance demand (dependent variable). We use the factual and counterfactual information as the independent variables. The model additionally includes controls (self-reported age, self-reported gender, country, in-game wealth). The analysis also includes fixed effects for the different lotteries offered to the subjects. Standard errors are heteroscedasticity-robust and clustered on the level of the subject.

Analysis A3 is descriptive. We plot the average self-reported satisfaction of both groups insurance demand in dependence of the factual (loss, null) and counterfactual (clear-loss, close-loss, clear-null, close-null) outcomes over all lotteries. For this, we bin their insurance demand in phase 1 in steps of 20.

Analysis A4 is inductive. We use an OLS model to calculate the differences between both groups self-reported satisfaction with their insurance demand (dependent variable). We use the factual and counterfactual information as well as their previous insurance demand as the independent variables. The model additionally includes controls (self-reported age, self-reported gender, country, in-game wealth). The analysis also includes fixed effects for the different lotteries offered to the subjects. Standard errors are heteroscedasticity-robust and clustered on the level of the subject.

Moreover, Analysis A2 and A4 include one additional indicator. The indicator equals one if the loss outcome results in a previously affordable item no longer being obtainable. This aims to control for systematically different choices at levels of wealth that directly affect the subjects options to unlock game content.

We additionally collect a variety of in-game data, which are potential interesting control variables for inductive multivariate tests. However, we will not interact them with the independent variables because this does not address our main research question.

Subjects are randomly allocated to the control or treatment group when agreeing to the consent form. 25% of the subjects are allocated to the control group. 75% of the subjects are allocated to the treatment group.
The lotteries will show up everytime a player completes at least three levels in three different runs within the same game session and are randomly chosen by the computer.

The randomization unit of the group allocation is the individual subject.

No

Not applicable to our case (see above).

Our setting deviates from traditional experiments in the sense that the number of downloads in Google’s Play Store determines the number of subjects in our experiment. We will upload the game to Google’s Play Store on 12 June 2020 and expect the game to be public on the same day (delays are possible due to Google's processing). We will additionally upload a promotional video to YouTube on 19 June 2020 (this date is, however, subject to discretion of the promoting influences). We expect only a small number of game downloads until the release of the promotional video and thus see the date of the promotional video as our trial start. For the analyses described, we only use data from the first 30 days after the release of the promotional video. The sample size of our experiment is therefore dependent on the number of downloads from Google’s Play Store and the game’s retention rate. Thus, the sample size is not predictable at this stage. We will additionally filter our data as described below and provide for each filter corresponding reason in parentheses: We remove players that are uniquely identifiable based on the collected demographic variables (self-reported age, self-reported gender, and country). [data protection law] We are only allowed to process the choice of players that specify an age greater or equal to 16. [data protection law] The lotteries will show up everytime a player completes at least three levels in three different runs within the same game session[experimental design].

We have two treatment arms. We allocate 25% of the subjects in the control group and 75% into the treatment group.

For our aggregated analysis, a power test is not applicable. The sample size is dependent on the number of downloads from Google’s Play Store and the game’s retention rate and is, thus, not predictable at this stage.