Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
The sample size of 3,600 firms has been calculated to meet two goals:
• High statistical power to detect small changes in formalization rates;
• Sufficient statistical power to analyze the effect of being formal on firm performances, assuming that the program has an effect on formalization (i.e., formal businesses increase at least by 25 percentage point).
Assuming that at most 10% of the businesses in the control group formalize during the study period, a sample size of 1,000 in each treatment group would give us a power of 91.3% to detect a 5 percentage point increase in the formalization rate. Our target is a 25 percentage point increase in formalization, in order to get sufficient take-up of formality and examine impacts of being formal on other firm outcomes.
It would also yield the same power to detect a 5 percentage point increase in the proportion of firms paying taxes or receiving a bank loan. To examine power for continuous outcomes like firm profitability, or amount of sales, we use the data collected during the listing/baseline survey. In this data, standard deviations of both profits and sales are equal to the mean. It means that if the treatment leads to a 25 percentage point increase in the formalization rate, with a baseline and two rounds of follow-up data, using ANCOVA, we will have 81.2% power to detect a 36% increase in firm profits from formalizing.
The above power calculations assume a 100% take up rate and no attrition, but these assumptions may be unrealistic given the context. However, program take-up is likely to be very high because the organization in charge of the program is likely to be able to get in touch with most of the informal businesses to deliver the different treatments. If we expect to reach 95% of informal businesses with our intervention (the “take up-rate”), then we would need a sample of 1,108 in order to detect a 5 percentage points increase in formalization rates [1/(0.95)^2)*1000].
In addition, we may “lose” part of the sample due to attrition (i.e. businesses that we cannot survey during the follow-up surveys). We think that we will be able to keep the attrition rate below 10% (since we are only targeting businesses with a fixed location it seems a realistic assumption).
With 10% attrition and a take-up rate of 95%, we get our sample size of 1,200 informal businesses in each of the three groups, two treatment groups and one control (and one treatment group further split in two sets of 300 and 900 businesses), for a total number of 3,600 informal businesses.