|
Field
Last Published
|
Before
February 27, 2019 11:55 AM
|
After
February 27, 2019 12:01 PM
|
|
Field
Planned Number of Clusters
|
Before
The number of clusters is the same as the number of units. It is planned to be 2,000 firms, but is currently 943 and the total sample will depend on the numbers inducted in 2018.
|
After
The number of clusters is the same as the number of units. It is 753 firms.
|
|
Field
Planned Number of Observations
|
Before
2,000 firms planned, currently 943
|
After
753 firms
|
|
Field
Sample size (or number of clusters) by treatment arms
|
Before
The sample size is equally allocated across the five groups (subject to rounding).
The target is 400 in each group, but currently we have:
Training 190 firms; consulting 188 firms; insourcing 187 firms; outsourcing 191 firms; control 187 firms
|
After
The sample size is equally allocated across the five groups (subject to rounding).
Control - 149 firms
Training - 153 firms
Consulting - 149 firms
Insourcing - 152 firms
Outsourcing- 150 firms
|
|
Field
Power calculation: Minimum Detectable Effect Size for Main Outcomes
|
Before
The sample size is determined by the project budget and geographic distribution of applicants (firms applying from other parts of Nigeria will not be offered the insourcing or outsourcing treatments, but will still be randomized to control, grants, training, or consulting, and to date we have 730 control, 434 training, and 434 consulting outside of Abuja and Lagos – we will use this other sample to measure impacts of these first two treatments in those regions).
Our power calculations are then designed to determine the minimum detectable effect possible with this sample. We are in a better position for making these calculations than most studies because the 2016 application window has closed and gives us data on the initial 943 experimental firms. Moreover, based on the first batches, we have over 90% take-up for the consulting, insourcing, and outsourcing treatments (training take-up data not yet available). We therefore assume 90% take-up in these calculations. We report the minimum detectable TOT effect assuming this take-up rate.
Table 2: Minimum Detectable Effect (80% power, 5% significance level)
Business Practices Score Employment Log Profits
Unit Level Firm Firm Firm
Variable Mean 6.31 4.22 13.05
Variable SD 0.75 2.59 1.35
MDE (in SD) 0.22 0.18 0.19
MDE (as % of reference mean) 2.6% 4.3% 25%
Business practices: calculation is made using only a single round of follow-up data. MDE for the ITT is then 0.15, so for the TOT is 1.1*0.15 = 0.165. This is 0.22 s.d., or 2.6% increase in the mean. Since the score is on a 10-point scale, this is equivalent to a 0.02 improvement in the proportion of practices implemented. This is less than half the size found in business training experiments cited in the literature review.
Employment: power calculations assume Ancova estimation, controlling for baseline value of employment. Autocorrelation is assumed to be 0.5. MDE is then 0.43 workers , so a TOT of 1.1*0.43 = 0.47 workers. This is 0.18 S.d., or a 4.3% increase on the control mean. For employment we believe the increase in the number of workers is the most useful metric. 0.47 workers increase in employment will occur if half the firms hiring an insourced worker keep this worker on, even if they do not grow enough to hire more workers. McKenzie (2017) finds high-growth enterprises achieve employment gains of 4-5 workers per firm from winning the business plan competition, so we are only trying to detect increases one-tenth of that size.
Log monthly sales: power calculations assume Ancova estimation, controlling for baseline value of sales. Autocorrelation is assumed to be 0.5. MDE for the ITT is 0.23, so TOT is 0.25 log points, or a 25% increase in the control mean. We aim to reduce this further through three steps: i) controlling for baseline covariates like sector, business practices, and employment size that can soak up some of the variation in size and reduce heterogeneity; ii) budget allowing, use multiple measurements of monthly sales within a given follow-up period as in McKenzie (2012); and iii) reduce measurement error and hence noise by utilizing a cross-checking survey technology.
|
After
See pre-analysis plan.
|