|
Field
Trial Status
|
Before
in_development
|
After
completed
|
|
Field
Last Published
|
Before
January 02, 2024 10:50 AM
|
After
November 02, 2025 11:01 PM
|
|
Field
Planned Number of Clusters
|
Before
600
|
After
4000 customers
|
|
Field
Planned Number of Observations
|
Before
600
|
After
4000 customers
|
|
Field
Sample size (or number of clusters) by treatment arms
|
Before
300 customers in control group and 300 customers in treatment group
|
After
2000 customers in control group and 2150 customers in treatment group
|
|
Field
Power calculation: Minimum Detectable Effect Size for Main Outcomes
|
Before
|
After
Before our experiment, we used the customers who entered the platform in 2023 to conduct the power calculation. The baseline conversion rate was 9.9% percent during this period. We set the significance level as 0.1, and power as 0.8. Then, we varied the conversion rate for the treatment group from 0.12 to 0.20. The required sample size is from 300 to 19112. Given our target sample size, the minimum effect size detectable for conversion rate is 7.9% and this requires the conversion rate for treatment group to be 12.3%.
|