Minimum detectable effect size for main outcomes (accounting for sample
design and clustering)
We ran a small pilot that included 10 youth resellers and one agro-dealer during the post-harvest season of July to September 2021 that follows the short growing season. Using the revenue baseline mean of 13,400 Kenyan shillings from the pilot, we assumed an increase of 5 percent as a result of the training, which would be in line with the average increase in sales from business training interventions as discussed in McKenzie (2021). We based our standard deviation assumptions based off the standard deviation of 0.05 for sales in McKezie and Puerto (2021). They ran an RCT with 3,357 firms in 157 rural markets in Kenya where firms were randomly assigned to receive business training. Knowing that we most likely wouldn’t be able to achieve nearly as large of a sample size given financial resources and time constraints, we assume a standard deviation of 0.15 for revenue. This higher value is warranted given that a smaller sample size will have higher levels of variability in the main outcome variable. We concluded that we could detect a reasonable MDE between 800 to 1300 Kenyan shillings with a sample size in the range of 250 to 350 youth participants. This was determined assuming different scenarios with intra-cluster correlation coefficients of 0.01, 0.04, 0.07, and 0.1. These scenarios gave us a cohen’s d value of 6 to 10 percent. These calculations were also done assuming 0.05 significance level, 0.8 statistical power, and a cluster size of 10 youth (per youth group). To account for expected early attrition, we determined that our ideal sample size would be to oversample and recruit 400 youth to take part in the intervention. This attrition is important to note as rural youth can be a transient group that may relocate in search of economic opportunities (Bezu & Holden, 2014). This number was also comparable to the size of the treatment and control groups in other youth employment studies conducted in Kenya (see Alvares de Azevedo et al., 2013; Hicks et al., 2015; Honorati, 2015; Brudevold-Newman et al., 2017).
References:
Alvares de Azevedo, T., Davis, J., & Charles, M. (2013). Testing What Works in Youth Unemployment: Evaluating Kenya’s Ninaweza Program (Summarative Report No. Volume 1). Available at http://www.iyfnet.org/sites/default/files/library/GPYE_KenyaIm pactEval_V1.pdf (accessed July 12, 2021).
Bezu, S., & Holden, S. T. (2014). Are Rural Youth in Ethiopia Abandoning Agriculture? World Development, 64, 259–272.
Brudevold-Newman, A., Honorati, M., Jakiela, P, & Ozier, O. (2017). A Firm of One’s Own: Experimental Evidence on Credit Constraints and Occupational Choice. World Bank Policy Research Working Paper 7977. World Bank, Policy Research Department, Washington, D.C.
Hicks, J. H., Kremer, M., Mbiti, I., & Miguel, E. (2015). Vocational Education in Kenya-A Randomized Evaluation. 3ie Grantee Final Report. New Delhi: International Initiative for Impact Evaluation (3ie).
Honorati, M. (2015). The Impact of Private Sector Internship and Training on Urban Youth in Kenya. World Bank Policy Research Working Paper 7404. World Bank, Policy Research Department, Washington, D.C.
McKenzie, D. (2021). Small Business Training to Improve Management Practices in Developing Countries: Re-assessing the Evidence for “Training Doesn’t Work”. Oxford Review of Economic Policy, 37(2), 276-301.
McKenzie, D., & Pueto, S. (2021). Growing Markets through Business Training for Female Entrepreneurs: A Market-Level Randomized Experiment in Kenya. American Economic Journal: Applied Economics, 13(2): 297–332.