Sample size (or number of clusters) by treatment arms
In part one of the study we will collect 528 subjects in total, 264 per treatment (i.e., 264 in Baseline RFT, and 264 in Baseline PGG). Based on pilot data our treatment manipulation had an effect size of approximately 0.88 (comparing the difference in mean beliefs between the high and low signal with a Mann-Whitney test). Power calculations indicate we require a sample size of 224 subjects per task (ensuring n is divisible by group size of 4) to detect such an effect (alpha=.001, beta=0.999). In the pilot 88% of subjects who completed Part one subsequently completed Part two. Assuming the same retention rate, we plan to recruit 264 subjects per task for a total sample size of 264*2 = 528 subjects.
Part two will take place one week later on the Prolific platform and will invite all 528 subjects back. Subjects within each treatment arm will be randomised to either see a high or low signal about the average number of tokens previous groups placed in the blue bucket. Of the 264 subjects in each treatment 132 will see the high signal and 132 will see the low signal (i.e., 132 in Baseline RFT high signal, 132 in Baseline RFT low signal, 132 in Baseline PGG high signal, and 132 in Baseline PGG low signal).
Part two will remain active on the Prolific platform for two weeks. Depending on the number of subjects who return to complete part two we will recruit additional new subjects to ensure that the final subject size in part two is divisible by the required group size.
Exclusion Criteria: We will exclude any incomplete data, and consider the first 264 complete submissions in each task from subjects who submitted a completion code on Prolific. Subjects’ who answer comprehension questions incorrectly 2 or more times in Baseline RFT (3 or more times in Baseline PGG) are screened out of the study. In our analysis we will exclude: (i) subjects’ who self-report that they did not pay attention during the study* (ii) subjects in Baseline PGG who completed the study in 10 minutes or less (iii) subjects in Baseline RFT who completed the study in 8 minutes or less (iv) subjects who report they did not understand the decision scenario**. After applying these exclusion criteria if the sample size in any task drops below 264 we will recruit additional subjects to make up the difference in the impacted task (e.g., if the sample drops to 263 in Baseline RFT, we will recruit 1 additional subject fulfilling the aforementioned non-exclusion criteria to the Baseline RFT task). *This is based on subjects’ response to the following question: “The following question does not affect your payment, and is only used for data quality purposes. Please indicate the extent to which you agree with the statement: Throughout the study I was paying sufficient attention.” with response options Strongly Disagree, Disagree, Agree, Strongly Agree. We will exclude those who do not indicate Agree or Strongly Agree. **This is based on subjects’ response to the following question: “Please indicate how certain you are that you understand how this interaction works and how your bonus payment is calculated. 0% means ‘completely uncertain’ and 100% means 'completely certain’.” With response options 0/10/20/30/40/50/60/70/80/90/100%. We will exclude those who indicate 50% or lower.