Experimental Design
Researchers evaluate the Empowerment and Livelihood for Adolescents (ELA)
program using a randomized control trial. BRAC has established branch offices throughout Uganda, ten of which were chosen for the evaluation. Five of these branches are located in the urban or semi-urban regions of Kampala and Mukono; the other branches are located in the mostly rural region around Iganga and Jinja. In each branch, fifteen communities with the potential to host an ELA club were identified. From this list, ten locations within each branch office were randomly assigned to receive the treatment, i.e., to set up a club and deliver the ELA program, with the remaining five locations randomly assigned as controls. In each treatment community, a single club was opened up. Hence, the research design delivers 100 treatment and 50 control communities, stratified by branch office.
The practicalities of program implementation led to possible non-compliance with the research design: an adolescent girl resident in a control community wishing to attend a club in a treated community is always able to do so. However, in practice the number of participants from control communities is negligible, and such non-compliance biases the estimated impact towards zero. Of course in treated communities, as club participation is voluntary, not all eligible girls will comply with the design and decide to take-up the offer of receiving the ELA program. Therefore, researchers estimate both ITT and TOT impacts.
The primary data sources are a baseline survey administered to adolescent girls conducted at baseline, and a follow-up survey two years after the ELA program is initiated. Baseline interviews were conducted from March to June 2008. The vast majority of ELA clubs were established between June and September 2008, and the follow-up survey was fielded from March to June 2010. The questionnaire covers topics including: (i) those directly related to the vocational skills component, such as financial literacy, analytical ability, labor market and income generating activities; (ii) those related directly to the life skills component, such as engagement in sex, childbearing and marriage/cohabitation, HIV related knowledge; (iii) other margins such as educational investments, time use, expenditures, and further measures of economic and social empowerment.
In total, at baseline 5,966 adolescents were surveyed: 3,964 reside in treatment communities, and 2,002 girls in controls, with an average of 39.9 (39.7) girls being surveyed in each rural (urban) community. Despite the high degree of geographic mobility of girls in Uganda in this age range, 4,888 adolescents were tracked to follow-up, corresponding to a two-year tracking-rate of 82%. This is comparable to tracking rates from studies in similar contexts. While residing in a treatment community does not predict attrition, researchers present robustness checks on our core findings that account for attrition.
Researchers first estimate the intent-to-treat (ITT) impact of the ELA program using OLS, controlling for adolescent’s age at baseline, a series of indicator variables for branch areas as we stratify the sample of communities by branch before randomly assigning them to treatment or control status. The disturbance term is clustered by community as there are likely to be common unobserved factors within communities that determine outcomes.
Because some outcomes are related to dichotomous or censored outcomes, researchers also report ITT estimates based on non-linear Probit and Tobit specifications analogous to the OLS specification. Given that club participation is not universal, the ITT underestimates the impact of the program on actual club participants. Therefore, researchers also present estimates of the treatment-on-the-treated (TOT) effect of the program following a standard approach of instrumenting individual club participation with the community treatment dummy (i.e., treatment offer). The impacts are computed using a standard two-stage least squares procedure and, therefore, all coefficients are computed using a linear probability model in the first stage. In the first stage, researchers regress a dummy variable for participation on the treatment dummy, branch fixed effects and individual baseline characteristics. Under the assumption that treatment assignment has no spillover impacts on non-participants, the IV estimate produces the local average treatment effect (LATE) on those that are induced to participate in the presence of a club in their community.
Researchers assess program impact on economic empowerment, as reflected in business skills and income generating activities, and control over the body, as reflected in, childbearing, marriage, and sex and on aspirations on childbearing and marriage. They also document program fixed and variable costs to benchmark how large the per-girl benefits would have to be for the intervention to be effective for a social planner. Researchers assess ITT anticipation impacts of future assignment to microfinance. To check the robustness of the results to panel attrition, researchers re-estimate each ITT impact using weights, where these weights are constructed from the inverse of the estimated probability of panel inclusion. They also check for whether particular subsamples of the data are driving the main impacts on economic and social empowerment. They check for heterogeneous impacts among outcomes related to current investments in schooling and explore impact heterogeneity along the following dimensions: (i) rural versus urban households; (ii) rich versus poor households, as defined by whether the household’s asset values at baseline are above or below the median for all households; (iii) girls aged above 16 at baseline versus older girls at baseline.