Minimum detectable effect size for main outcomes (accounting for sample
design and clustering)
We ascertain our sample size through a meticulous estimation with a hypothesized main effect and employ this derived sample size to determine the minimum effect size that can be detected for the hypothesized absence of an interaction effect.
We ascertain our sample size through a meticulous estimation with a hypothesized main effect (H1) and employ this derived sample size to determine the minimum effect size that can be detected.
We derive the primary effect size from the study conducted by Powdthavee and Riyanto (2015), which is of direct relevance to this study. Their research, which focused on a coin prediction game featuring options for participants to pay for predictions without any access to information about the prediction generation process. In their study, the effect size became significantly larger and larger after observing a longer streak of correct predictions. For instance, the effect size of a streak of two accurate predictions on the purchasing behavior is 0.153, corresponding to an Odds ratio of 8.85 at round 3. The effect size of having longer streaks is higher (i.e., the effect sizes associated with streaks of three and four accurate predictions on purchasing behavior are 0.211 and 0.273, respectively). For the purpose of our analysis, and since the treatment effect is already relatively strong with a streak of two accurate predictions, we will focus our experiments on it. R^2 other X=0.25X parm π=0.25, a one-tailed z-test, and the test procedure with variance correction proposed by Hsieh et al. (1998), we conducted sample size estimation using G*Power 3.1. The analysis yields a total sample size of n = 204, providing an actual power = 0.800218. In other words, a sample size of n = 204 will provide a statistical power of more than 80% for our fundamental hypothesis (H1).
Since the proposed experiment will involve a two-dimensional treatment (With/Without Financial Intervention × Paying for Advice/Donate for Luck Setting), the total necessary sample size is 204×4=816.
In summary, our approach to sample size determination combines meticulous estimation based on the hypothesized main effect with rigorous sensitivity analysis. This enables us to ascertain a sample size that fulfills the demands of our research objectives and study design.