Minimum detectable effect size for main outcomes (accounting for sample
design and clustering)
The sample size was determined based on power calculation for the test of differences in proportions. We adopt the data from one similar study (Chen, Pillutla, & Yao, 2009). We estimate the sample size for the regression model for all samples by the “sampsi” command in Stata.
The previous study shows that in the first round, in the control condition, the individual's contribution portion is 45.8%, in the monetary incentive condition, the individual's contribution portion is increased to 61.2%, and in the appeal condition, the contribution portion is increased to 62.8%. In the second round, in the control condition, the individual's contribution portion is 30.3%, in the monetary incentive condition, the individual's contribution portion is decreased to 35.0% due to removal, and in the appeal condition, the contribution portion decreases to 50.1% due to removal.
Among the comparison results, the smallest impact is between the control group (45.8%) and the monetary incentive group (61.2%) in round 1. Thus, we use this minimal detectable impact from the previous study to identify our sample size. Other impacts are all larger than the result of this comparison.
In specific, In STATA 16.1 the command [sampsi 0.458 0.612, power(0.8) alpha(0.05)] provided a sample size of 177 per experimental arm. We rounded up to 180 observations. The target total sample size for two treatment arms and one control condition is therefore 540 (3 conditions * 180 villagers).
In addition, we can control for individual-level covariates and we double the number of observations by playing the game twice. We can even identify smaller impacts given these conditions.