|
Field
Last Published
|
Before
May 08, 2025 02:27 AM
|
After
May 08, 2025 02:34 AM
|
|
Field
Planned Number of Observations
|
Before
200 Prolific recruiters each reviewing 8 profiles
50 PhD student recruiters each reviewing 8 profiles
|
After
200 Prolific recruiters each reviewing 6 profiles
50 PhD student recruiters each reviewing 6 profiles
|
|
Field
Sample size (or number of clusters) by treatment arms
|
Before
The randomization will be done at individual evaluation level. However, given our randomization method, the expected number of observations falling in different conditions will be:
(1) 750 AI-generated and identified
(2) 250 AI-generated but not detected
(3) 750 human-generated and recognized
(4) 250 human-generated but mistaken as AI-generated
|
After
The randomization will be done at individual evaluation level. However, given our randomization method, the expected number of observations falling in different conditions will be:
(1) 562 AI-generated and identified
(2) 188 AI-generated but not detected
(3) 562 human-generated and recognized
(4) 188 human-generated but mistaken as AI-generated
|
|
Field
Power calculation: Minimum Detectable Effect Size for Main Outcomes
|
Before
Given our pilot study results which tests the effect of perceived use, given the mean value of the baseline overall evaluation (human-generated) is 6.14, and the standard deviation is 1.74, to observe a 0.2 sd treatment effect with 95% significance level and 80% power, we need at least 389 observations per group. If each recruiter evaluates 8 profiles, we need at least 98 recruiters.
|
After
Given our pilot study results which tests the effect of perceived use, given the mean value of the baseline overall evaluation (human-generated) is 6.14, and the standard deviation is 1.74, to observe a 0.2 sd treatment effect with 95% significance level and 80% power, we need at least 389 observations per group. If each recruiter evaluates 6 profiles, we need at least 130 recruiters.
|