Using Household Grants to Benchmark the Cost Effectiveness of a USAID Workforce Readiness Program

This study experimentally evaluates the relative cost-effectiveness of alternative programs to improve the employment and productive potential of underemployed Rwandan youth. We pursue an individually randomized trial with 1,848 youth to understand how a `standard' package of training, soft skills, and networking interventions compares not only to an experimental control group but to an additional arm that receives household grants of equal cost to the donor---a cash benchmark. The study follows youth who fit the eligibility criteria of the study and who expressed interest in participating in the training program. The core program is called Huguka Dukore (HD), which means`work well done' in Kinyarwanda; it follows USAID's strategy on workforce readiness and skills training and was implemented by Education Development Center, Inc. (EDC). The benchmarking cash transfer program was implemented by GiveDirectly (GD), a US-based nonprofit that specializes in making unconditional household grants via mobile money. These two treatments are compared to a control group, namely a set of individuals that receive neither program, and a combined arm that receives both.

2018-01-01

2019-06-30

There are five primary outcomes:
1. Employment status. A binary indicator variable taking a value of one if the beneficiary spent
any time in a wage job or as primary operator of a microenterprise. 1 week recall, per ILO
definition. Defined as ’Yes’ if any of the following questions are affirmative:
 Processing or trading of agricultural goods (B02qagroprocesshrs)
 Agricultural (off farm) wage labor (B02qfarmhours)
 Non-agricultural wage labor (B02qnoagrichrs)
 Non-agricultural microenterprise (B02qenterphrs)
 Microenterprise or other self employment (B02qsemployhrs)
While the primary version of this outcome is its value at midline, we will also present, descriptively,
the month-by-month impacts of each intervention on employment status for the
six months prior to the midline, using data from the rolling phone survey.

2. Off-own-farm productive time use. Defined as the number of productive hours over the past
7 days. Sum of hours from questions:
 Processing or trading of agricultural goods (B02qagroprocesshrs)
 Agricultural (off farm) wage labor (B02qfarmhours)
 Non-agricultural wage labor (B02qnoagrichrs)
 Non-agricultural microenterprise (B02qenterphrs)
 Microenterprise or other self employment (B02qsemployhrs)
Apprenticeship (B02qapprenticehrs)

3. Beneficiary’s (monthly) income. Defined as the sum of the following monthly recall questions:
 Agricultural own-farm income (B02qagricearn)
 Agricultural wage income (B02qfarmwage)
 Non-agricultural wage income (B02qnoagricwage)
 Microenterprise profits (B02qenterpwage +B02qsemploywage);
 Livestock rearing income (B02qlivestockwage)
 Agricultural processing and trading income (B02qagroprocessearn)
 Apprenticeship income (B02qapprenticewage)

4. Productive assets under beneficiary control. (Sum of asset values from beneficiary enterprise
module that are reported as used in the beneficiary’s business—B05tools, machinery,furniture,
inventories, and other physical assets.)

5. Household consumption per capita. Sum of monthly purchase values of Section H10, divided
by adult-equivalent household members.

1. Subjective well being. Index of responses to B10_swb_happiness and B10_swb_lifesatisfaction.
Index formed by taking z-scores of both responses and averaging for each individual.

2. Mental health index of section B11 responses. Z-score of the simple average across all questions
for each beneficiary.

3. Index of Soft Skills
(a) Locus of control: Index of responses to B09.
(b) Aspirations: Index of responses to B13.
(c) Conscientiousness, agreeableness, and emotional stability from BFI (Section B12) Following
EDC’s analysis of Akaze Kanoze employers,3 we will examine program impacts
on the three most highly-rated components of the Big-Five Index from employers’ perspective:
conscientiousness, agreeableness, and emotional stability. See Appendix E.1 for
details.

4. Stock of savings. Beneficiary stock of savings, sum of values in B06. Plus household stock of
savings from analogous questions (H06).

5. Stock of debt. Access to finance is the outcome of interest. Beneficiary sum of borrowed
amounts from all (formal and informal) sources (B07). Plus household borrowings from analogous
questions (H07)

6. Business knowledge. Index of B14

7. Business attitudes. Index of B15.

8. Household net wealth. Sum of values of household assets (H12), plus savings value (H06),
value of loans outstanding that are expected to be repaid (H08) , less debt value (H07).

9. Beneficiary consumption expenditures (sum of values from Section B08).

The allocation of these study households to treatment was undertaken on a randomized basis across
eligible, interested individuals using a public lottery. Lotteries were conducted at the sector level
in each of the 13 sectors in the study, and the proportions assigned to each treatment were fixed
at each lottery. This results in a fairly standard ‘blocked’ randomization structure across the 13
blocks in the study. Participants drew their own treatment status as tokens of different colors from
a sack, where each token corresponded to a given treatment arm and the number of tokens in the
hat was determined by IPA according to the number of participants.
The detailed protocol for the lottery is as follows:
1. Beneficiaries did not have to be physically present at the lottery to be included in the study.
We explicitly recognized the right of EDC/HD to eliminate from eligibility any individuals
who they feel, for whatever reason, was not ‘serious’ about the program and that they did not
believe will fully enroll in HD if selected.
2. Detailed information about GD was not provided prior to the lottery, but GD was be described
in detail at the lottery and every effort was made to preserve the separate identity of HD and
GD so as not to provoke confusion about the broader HD program. All information provided
at the lottery was be given to everyone, and there was not an attempt to separate groups and
give private information.
3. A representative of both GD and HD (or its local partner) were present at every lottery.
4. Individuals were notified whether they have been assigned to the GD, the ‘combined’ arm or
the HD arm at the time of the lottery.
5. Individuals assigned to GD received a variety of colors which correspond to different transfer
amounts. This means that the random assignment to GiveDirectly simultaneously randomly
assigned individuals to the different transfer size amounts. GD would not reveal financial
amounts at the time of the lottery, but would reveal timing of transfers to all beneficiaries,
and would describe their enrollment process. GD will explain that youth randomized to GD
will be contacted soon after the lottery to orient them to the program, and that they will be
visited at their place of residence to undertake the enrollment process by [month].

13 different public lotteries, one for each sector included in the study.

Individual beneficiary.

No

The study is not clustered; it takes place in roughly 250 villages in 13 sectors of Rwanda but was individually randomized.

1848 eligible youth, and their households.

The assignment of individuals to the main study arms was as follows:
1. HD beneficiaries (485 individuals);
2. Recipients of unconditional household grants (672 individuals);
3. Combined arm who received both HD and the household grants intervention (203 individuals)
4. A comparison group, in which no program was offered (488 individuals).