Minimum detectable effect size for main outcomes (accounting for sample
design and clustering)
Here we detail our approach to conducting a power calculation for our study. Unfortunately, we did not have access to data from all 1,700 households needed to directly estimate the power calculation before the intervention. Instead, we utilized historical data from approximately 300 households equipped with smart meters.
To determine if a 5% detectable effect size was feasible with our sample size, we conducted an estimation using 1,300 samples. This number was chosen in early May when the success of recruitment was uncertain, so we opted for a conservative estimate for our calculations.
We leveraged bi-weekly historical electricity usage data from the smart meters, covering the period from January 30th to May 7th, which included 8 measurement points. To calculate the statistical power, we assumed a 5% reduction in electricity usage as our effect size. We used data from 300 households, resampling their usage patterns to create 1,300 observations, divided into three groups: one control group and two treatment groups.
The initial two months of data (comprising 4 points in time) served as the baseline. For the subsequent period (the next two months, also with 4 points in time), we assumed a 5% reduction in electricity usage for the two treatment groups. We utilized a fixed effects model to ascertain the significance of the effects. By replicating this process 1,000 times, we observed that 846 iterations yielded significant results at a 5% significance level, resulting in an estimated power of approximately 84.6%.
Given these findings, we concluded that a sample size of 1,300 households, each providing bi-weekly electricity data, should enable us to achieve an estimated power of 84.6%. Therefore, our total sample of 1,700 households is more than sufficient to ensure the robustness of our study’s findings.