Minimum detectable effect size for main outcomes (accounting for sample
design and clustering)
We use outcome data on sales from two past RCTs to estimate sample size requirements. For our key outcome, we focus on sales outcomes at the intensive and extensive margin. Our first pilot RCT showed that 14% in the control versus 97% in the treatment arm completed at least one sale over the pilot period (a cumulative measure, in contrast to the monthly rate reported for the second pilot below). Effect sizes on sales outcomes are substantial: Mean sales are 0.46 (SD = 1.35) in the control group and 5.54 (SD = 2.34) in the treatment group. Pooling across arms, the overall standard deviation of sales in this pilot was 3.1. Since this pilot in 2018, we have improved data quality through better IT systems and reduced measurement error; we therefore conservatively assume this overall SD has dropped slightly, to 2.7.
We complement this and use outcome data on sales at the month level from a second pilot RCT (N = 96 respondents, four arms, conducted 2019) to estimate sample size requirements. For our key outcome, we focus on sales outcomes at the intensive and extensive margin. The pilot showed that 13.1% of respondents in the control arm completed at least one sale in a given month, compared with 41.7–49.3% across the three treatment arms. Effect sizes on sales outcomes were substantial: mean monthly sales were 0.19 (SD = 0.55) in the control group and ranged from 2.07 to 3.28 (SD = 3.72–5.80) across the treatment arms.
We will implement an individual-level randomization and stratify randomization by district and current employment status (excluding farming, dairy or daily wage work). We plan to use a regression specification that includes district-fixed effects and individual-level exogenous covariates (e.g., household size, age) to increase power. Importantly, the new RCT will collect respondent-month panel data over 3–5 months, which substantially raises effective sample size; we adopt a conservative intra-respondent correlation of ρ = 0.05. All calculations follow J-PAL/World Bank standards with α = 0.05 and power = 0.80.
The study includes 280 respondents: a control group and three treatment arms of 70 participants each. Assuming T = 4 monthly observations per respondent (ρ = 0.05). Below, we present the resulting minimum detectable effects (MDEs):
5. % Completing a Sale:
a. Across our two pilots, with a control completion rate of 13 to 14%, we are able to detect an MDE of approximately 9.7 to 10.0 percentage points (~0.29 SD of the control rate) in the share of respondents completing at least one sale, with 80% power.
b.
6. Number of Sales:
a. Across our two pilots, the observation-level standard deviation of monthly sales ranges from about 2.7 (Pilot 1) to 4.1 (Pilot 2). Given this range, we are able to detect an MDE of approximately 0.7 to 1.0 additional monthly sales (~0.25 SD) with 80% power.
For our key outcome, we focus on sales outcomes at the intensive and extensive margin. Our pilot RCT showed that 3% in the control versus 97% in the treatment arm completed at least one sale. We conservatively assume that the SD has slightly dropped slightly from SD = 3.1 to SD = 2.7.
Additionally, we are in a position to implement an individual-level randomization and thus do not need to implement cluster randomisation, which increases our power. Finally, we plan to use a regression specification that includes district-fixed-effects and individual-level exogenous covariates (e.g., household size, age) to increase power.
Key outcomes:
- % Completing a Sale:
Based on pilot data, we conservatively estimate that 5% of the control group would make a sale. Based on this assumption, with a sample of 60 Dharmalife women entrepreneurs per arm, we can detect an MDE of 0.65 SD (19.36% increase in the number of individuals completing at least one sale) with 80% power.
- Number of Sales:
Based on a sample of 60 Dharmalife women entrepreneurs per arm, we are able to detect an MDE of 0.50 SD increase (33.69% increase in the number of products sold) with 80% power.