Using Household Grants to Benchmark the Cost Effectiveness of a USAID Workforce Readiness Program

Last registered on November 15, 2019

Pre-Trial

Trial Information

General Information

Title
Using Household Grants to Benchmark the Cost Effectiveness of a USAID Workforce Readiness Program
RCT ID
AEARCTR-0004388
Initial registration date
June 26, 2019

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
July 12, 2019, 10:46 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
November 15, 2019, 7:56 PM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
IRPS/UCSD

Other Primary Investigator(s)

PI Affiliation
McCourt School of Policy, Georgetown University

Additional Trial Information

Status
On going
Start date
2017-12-01
End date
2021-01-31
Secondary IDs
Abstract
Using a randomized experiment, this study will benchmark the impact of a youth employment education intervention, Huguka Dukore (HD), to household grants in the Rwandan context. The focus for the study will be on underemployed youth, a high-risk demographic who may display particularly powerful long-term benefits from receiving additional knowledge and resources at a pivotal moment in their transition to adulthood. Youth enrolled in the study will be drawn from Rwamagana, Muhanga and Nyamagabe districts. Within these districts, the study will broadly target individuals from poor households (within the poorest two categories, Ubudehe 1 and 2), but study participants will consist only of individuals who are eligible for HD \textit{and} who express a willingness to enroll in the program at baseline. Impacts will be measured 18 months after baseline; primary outcomes are employment status, time use, beneficiary income, household consumption, and value of productive assets owned by the beneficiary's household and under beneficiary control. In addition we measure a range of secondary outcomes and intermediate mechanisms. The study is designed to test a cost-equivalent impact from the funder’s perspective, meaning that the amount spent per beneficiary in the two arms is made equivalent. Transfer amounts in the household grants arm are randomized in a range around the expected cost per beneficiary of HD so as to be able to calculate a regression-adjusted costing using the actually observed costs, and in addition a larger transfer amount was chosen so as to maximize the cost-effectiveness of cash given fixed costs. A third arm receives both HD and cash. Careful tracking of costs during implementation will allow the study to provide a cleanly benchmarked impact of an exactly cost-equivalent household grant to HD.
External Link(s)

Registration Citation

Citation
McIntosh, Craig and Andrew Zeitlin. 2019. "Using Household Grants to Benchmark the Cost Effectiveness of a USAID Workforce Readiness Program." AEA RCT Registry. November 15. https://doi.org/10.1257/rct.4388-2.1
Former Citation
McIntosh, Craig and Andrew Zeitlin. 2019. "Using Household Grants to Benchmark the Cost Effectiveness of a USAID Workforce Readiness Program." AEA RCT Registry. November 15. https://www.socialscienceregistry.org/trials/4388/history/57172
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
This study experimentally evaluates the relative cost-effectiveness of alternative programs to improve the employment and productive potential of underemployed Rwandan youth. We pursue an individually randomized trial with 1,848 youth to understand how a `standard' package of training, soft skills, and networking interventions compares not only to an experimental control group but to an additional arm that receives household grants of equal cost to the donor---a cash benchmark. The study follows youth who fit the eligibility criteria of the study and who expressed interest in participating in the training program. The core program is called Huguka Dukore (HD), which means`work well done' in Kinyarwanda; it follows USAID's strategy on workforce readiness and skills training and was implemented by Education Development Center, Inc. (EDC). The benchmarking cash transfer program was implemented by GiveDirectly (GD), a US-based nonprofit that specializes in making unconditional household grants via mobile money. These two treatments are compared to a control group, namely a set of individuals that receive neither program, and a combined arm that receives both.
Intervention (Hidden)

The Government of Rwanda, in its Second Economic Development and Poverty Reduction Strategy 2013-2018 (EDPRS 2), sets out ambitious targets for the reduction of poverty. Against this background, the current study seeks to identify and rigorously evaluate the cost-effectiveness of household grants relative to education and employability skills training as a tool to decrease poverty, improve child nutrition, and enhance the ability of underemployed youth to make a successful transition to a productive adulthood.

Against the backdrop of this program is a question now gaining currency internationally as a critical one: it may not be too surprising to show that an expensive, intensive project such as HD `works’ relative to a control group that receives nothing, but is it more cost-effective than taking the cost of the program and simply distributing that money to beneficiaries? Given many studies in many different contexts showing that household grants programs can be effective at reducing poverty, and given the low cost and simplicity of distributing household grants, there is increasing interest in establishing household grants as the `index fund’ of economic development. This is a research agenda that is respectful of the judgment and authority of poor households, assuming that they may know best what is the highest value use of money for their own households. This research agenda has particular relevance in the Rwandan context because the Government of Rwanda is itself moving heavily into the use of household grants as an anti-poverty tool (for example the Vision 2020 Umurenge Program, which recently received \$50m from the World Bank), meaning that the interest and capacity to implement this alternate intervention broadly already exists. This study is the second benchmarking activity being run by this team in Rwanda, the first being the benchmarking of the Gikuriro child malnutrition intervention being run by CRS.

Our study aims to bridge these knowledge gaps by conducting a carefully calibrated, four-armed randomized trial. In one arm, beneficiaries will receive the HD intervention as it is been designed by EDC and USAID/Rwanda. In a second arm, beneficiaries will receive household grants calibrated to have equal cost to the funder (meaning that the same amount of money is spent by the funder, and differences in overhead rates across interventions translate into differences in the monetary value of the benefits received by individuals on the ground. A smaller third arm will receive both HD and household grants. A control group (fourth arm) will receive no benefits during the term of the study, to allow both interventions to be evaluated in absolute as well as relative terms. By focusing on the a broad set of economic and health outcomes, this study will provide timely and policy-relevant information as to how under-employed youth can best be assisted in making the transition into a productive adulthood.


Using an individual-level randomization including youth from roughly 250 villages, we compare the HD program to cash transfers. The USAID Huguka Dukore Rwanda Youth Employment Project is a five-year project (2017-2021). HD will provide 40,000 vulnerable youth (women and men) with employability skills by scaling up successfully proven Akazi Kanoze interventions across 19 (of 30 total) districts countrywide, using a series of inclusive innovations that will invite more youth to participate in Rwanda’s historic transformation, particularly women, youth with disabilities and other vulnerable groups. Recognizing the diversity of youth backgrounds and goals, HD will offer multiple program pathways including: i) employment preparation; ii) individual and cooperative youth microenterprise start-up; iii) business development for existing microenterprises; and iv) continuation onto additional formal TVET training. Transfer amounts were randomized across four different amounts. A `combined' arm was offered both the HD intervention and GD transfers.



To permit a rigorous comparison of cost-effectiveness, we costed both programs in detail prior to, and after, the intervention period, following \cite{levin2001cost}. The ex ante costing exercise was used to identify the approximate total cost of the HD intervention, as well as the estimated overhead costs to GiveDirectly of providing household grants in this context. The ex-ante costing of HD arrived at a per-beneficiary cost of \$452.47. We then randomized transfer amounts at the individual level in the cash arm across four possible transfer amounts. These amounts were chosen to provide informative benefit/cost comparisons across two different margins: HD vs cash, and small versus large cash transfer amounts. Incorporating GiveDirectly's operating costs, the amount actually received by households that generates the same expected cost to USAID as HD is $410.19. The comparison between these two arms therefore provides a straightforward window on expected cost-equivalent impacts. Because we anticipate the exact numbers from the ex-post costing exercise will differ somewhat from the ex-ante exercise, we have randomized two bracketing cash transfer arms which transfer $317.31 and $503.04 to households. A fourth and substantially larger transfer arm transfers $750 to beneficiaries; this amount was chosen by GiveDirectly as maximizing their own cost-effectiveness given the returns to transfers and the fixed costs in providing cash transfers via mobile money. The inclusion of this arm provides a statistically high-powered way of examining how benefit/cost ratios shift as the transfer amount rises. Using the final, ex-post costing exercise, we will arrive at an exact cost per eligible household for both implementers, and use a linear regression adjustment across all four GD transfer amounts to analyze comparative impacts at exactly equivalent costs to the donor, USAID.
Intervention Start Date
2018-01-01
Intervention End Date
2019-06-30

Primary Outcomes

Primary Outcomes (end points)
There are five primary outcomes:
1. Employment status. A binary indicator variable taking a value of one if the beneficiary spent
any time in a wage job or as primary operator of a microenterprise. 1 week recall, per ILO
definition. Defined as ’Yes’ if any of the following questions are affirmative:
 Processing or trading of agricultural goods (B02qagroprocesshrs)
 Agricultural (off farm) wage labor (B02qfarmhours)
 Non-agricultural wage labor (B02qnoagrichrs)
 Non-agricultural microenterprise (B02qenterphrs)
 Microenterprise or other self employment (B02qsemployhrs)
While the primary version of this outcome is its value at midline, we will also present, descriptively,
the month-by-month impacts of each intervention on employment status for the
six months prior to the midline, using data from the rolling phone survey.

2. Off-own-farm productive time use. Defined as the number of productive hours over the past
7 days. Sum of hours from questions:
 Processing or trading of agricultural goods (B02qagroprocesshrs)
 Agricultural (off farm) wage labor (B02qfarmhours)
 Non-agricultural wage labor (B02qnoagrichrs)
 Non-agricultural microenterprise (B02qenterphrs)
 Microenterprise or other self employment (B02qsemployhrs)
Apprenticeship (B02qapprenticehrs)

3. Beneficiary’s (monthly) income. Defined as the sum of the following monthly recall questions:
 Agricultural own-farm income (B02qagricearn)
 Agricultural wage income (B02qfarmwage)
 Non-agricultural wage income (B02qnoagricwage)
 Microenterprise profits (B02qenterpwage +B02qsemploywage);
 Livestock rearing income (B02qlivestockwage)
 Agricultural processing and trading income (B02qagroprocessearn)
 Apprenticeship income (B02qapprenticewage)

4. Productive assets under beneficiary control. (Sum of asset values from beneficiary enterprise
module that are reported as used in the beneficiary’s business—B05tools, machinery,furniture,
inventories, and other physical assets.)

5. Household consumption per capita. Sum of monthly purchase values of Section H10, divided
by adult-equivalent household members.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
1. Subjective well being. Index of responses to B10_swb_happiness and B10_swb_lifesatisfaction.
Index formed by taking z-scores of both responses and averaging for each individual.

2. Mental health index of section B11 responses. Z-score of the simple average across all questions
for each beneficiary.

3. Index of Soft Skills
(a) Locus of control: Index of responses to B09.
(b) Aspirations: Index of responses to B13.
(c) Conscientiousness, agreeableness, and emotional stability from BFI (Section B12) Following
EDC’s analysis of Akaze Kanoze employers,3 we will examine program impacts
on the three most highly-rated components of the Big-Five Index from employers’ perspective:
conscientiousness, agreeableness, and emotional stability. See Appendix E.1 for
details.

4. Stock of savings. Beneficiary stock of savings, sum of values in B06. Plus household stock of
savings from analogous questions (H06).

5. Stock of debt. Access to finance is the outcome of interest. Beneficiary sum of borrowed
amounts from all (formal and informal) sources (B07). Plus household borrowings from analogous
questions (H07)

6. Business knowledge. Index of B14

7. Business attitudes. Index of B15.

8. Household net wealth. Sum of values of household assets (H12), plus savings value (H06),
value of loans outstanding that are expected to be repaid (H08) , less debt value (H07).

9. Beneficiary consumption expenditures (sum of values from Section B08).
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The allocation of these study households to treatment was undertaken on a randomized basis across
eligible, interested individuals using a public lottery. Lotteries were conducted at the sector level
in each of the 13 sectors in the study, and the proportions assigned to each treatment were fixed
at each lottery. This results in a fairly standard ‘blocked’ randomization structure across the 13
blocks in the study. Participants drew their own treatment status as tokens of different colors from
a sack, where each token corresponded to a given treatment arm and the number of tokens in the
hat was determined by IPA according to the number of participants.
The detailed protocol for the lottery is as follows:
1. Beneficiaries did not have to be physically present at the lottery to be included in the study.
We explicitly recognized the right of EDC/HD to eliminate from eligibility any individuals
who they feel, for whatever reason, was not ‘serious’ about the program and that they did not
believe will fully enroll in HD if selected.
2. Detailed information about GD was not provided prior to the lottery, but GD was be described
in detail at the lottery and every effort was made to preserve the separate identity of HD and
GD so as not to provoke confusion about the broader HD program. All information provided
at the lottery was be given to everyone, and there was not an attempt to separate groups and
give private information.
3. A representative of both GD and HD (or its local partner) were present at every lottery.
4. Individuals were notified whether they have been assigned to the GD, the ‘combined’ arm or
the HD arm at the time of the lottery.
5. Individuals assigned to GD received a variety of colors which correspond to different transfer
amounts. This means that the random assignment to GiveDirectly simultaneously randomly
assigned individuals to the different transfer size amounts. GD would not reveal financial
amounts at the time of the lottery, but would reveal timing of transfers to all beneficiaries,
and would describe their enrollment process. GD will explain that youth randomized to GD
will be contacted soon after the lottery to orient them to the program, and that they will be
visited at their place of residence to undertake the enrollment process by [month].

Experimental Design Details
Randomization Method
13 different public lotteries, one for each sector included in the study.
Randomization Unit
Individual beneficiary.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
The study is not clustered; it takes place in roughly 250 villages in 13 sectors of Rwanda but was individually randomized.
Sample size: planned number of observations
1848 eligible youth, and their households.
Sample size (or number of clusters) by treatment arms
The assignment of individuals to the main study arms was as follows:
1. HD beneficiaries (485 individuals);
2. Recipients of unconditional household grants (672 individuals);
3. Combined arm who received both HD and the household grants intervention (203 individuals)
4. A comparison group, in which no program was offered (488 individuals).
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
UCSD Social and Behavioral Sciences
IRB Approval Date
2017-09-13
IRB Approval Number
161112
IRB Name
Rwanda National Ethics Committe
IRB Approval Date
2017-09-13
IRB Approval Number
IRB 143/RNEC/2017
IRB Name
Innovations for Poverty Action
IRB Approval Date
2017-09-13
IRB Approval Number
IPA IRB 13730.
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials