Minimum detectable effect size for main outcomes (accounting for sample
design and clustering)
This section details the power analysis conducted for our game experiment, where the unit of analysis is individual level. In this experiment, we anticipate a 100% take-up rate for all treatment arms, as participants are automatically assigned to one of the treatment groups. It is important to note that failure to access the game during its duration does not count as attrition in this study, as it potentially reflects both attrition and free-riding behavior. Instead, attrition is defined strictly as the failure to respond to the post-game endline survey. In a pre-test conducted in February 2024, the observed attrition rate was 13%. Using this figure as a reference, we conservatively estimated the attrition rate to be 15% for this power analysis.
We performed power calculations for selected primary outcomes: the burning amount in the Joy-of-Destruction (JoD) experiment, the social cohesion index, and the index of beliefs about refugees’ abilities. These calculations are based on a hypothesis test with a significance level of 5% and 80% power, comparing two trial arms with 500 participants each in balanced treatment assignment. For the burning amount in the JoD experiment, our analysis builds upon the study by Higuchi et al. (2024), which implemented the JoD experiment against Rohingya refugees in 2020, targeting 1,679 households in Bangladesh. They reported a mean payment of 23.9 BDT and a standard deviation of 28.1 BDT to participants to decrease donations. For the indexes measuring social cohesion and beliefs about refugees’ abilities, we use standardized scales (z-scores) with a mean of 0 and a standard deviation of 1.
In our power analysis, the anticipated effect size with a 15% attrition rate is 0.19, which Cohen (1988) categorizes as a small effect. We are powered to detect a MDE of 5.41 BDT for the burning amount in the JoD experiment, which is approximately 23% of the mean reported in previous studies. For the social cohesion index and the index of beliefs about refugees’ abilities, the MDEs are set at 0.19 z-scores. These MDEs are considered reasonable for our experimental design, especially in light of findings from prior studies, such as one indicating that each standard deviation increase in distance from the refugee camp correlated with an increase of up to 7.5 BDT in the amount paid in the JoD experiment (Higuchi et al., 2024).