Benchmarking USAID-Funded Health Programs against Household Grants: An Application to Health in Rwanda

This project will provide rigorous, experimental evidence of the cost-effectiveness of alternative programs to improve the nutritional status of vulnerable households with children under the age of five and women of reproductive age, with an emphasis on the 1,000 day windows of opportunity from pregnancy until a child's second birthday. To do so, the study will conduct a cluster-randomized controlled trial that evaluates the impacts of two programs: a multi-dimensional nutritional program implemented by Catholic Relief Services (the program is called Gikuriro, which means Good Growth), and a household grant program implemented by GiveDirectly. These two treatments are compared to a control group, namely a set of villages that receive neither program. This study design achieves a fundamental kind of cost-effectiveness benchmarking increasingly called for in recent years: the comparison of a standard and widespread development intervention with the outcome that would occur if the cost of the intervention were simply given away to the beneficiaries for them to spend as they see fit.

Gikuriro consists of four components: a Village Nutritional School, Farmer Field Learning Schools, Savings and Internal Lending Communities, and Sustainable Sanitation and Hygiene for All. This combination has been developed by CRS over the course of many years implementing anti-malnutrition programs across the world. GiveDirectly has pioneered the use of household grants, an intervention which has been found in Ethiopia, Kenya, Zambia, and Zimbabwe to increase consumption, bringing with this improvements in dietary diversity. While very different in design, both of these approaches appear to hold out tremendous promise for improvements in child nutrition. These alternative approaches have rarely been evaluated side-by-side, in a setting that not only rigorously evaluates impacts for a statistically comparable population, but also carefully tracks costs in a comparable way across distinct projects. Doing so allows governments and donors to identify the most cost-effective means to achieve the critical goal of decreasing malnutrition.

2016-09-01

2017-08-31

a. Final Outcomes.
The study will measure and analyze impacts on five dimensions of household outcomes. Here we briefly summarize each; details of the construction of these outcomes are included in Appendix A.
1. Household monthly consumption per capita.
2. Dietary Diversity. This will be measured as the WHO standard Household Dietary Diversity Score.
3. Anemia: measured with a biomarker test following DHS protocols at endline only.
4. Child growth and development: measured in height-for-age, weight-for-age and Mid Upper Arm Circumference at baseline and endline for children under the age of 5 in eligible households.
5. Value of household non-land net wealth. This outcome includes both productive and consumption assets; the value of the household’s dwelling, if owned; and the value of the stock of net savings, less the stock of debt.

See PAP

i. Stock of borrowing.
ii. Stock of savings.
iii. Birth outcomes, comprising the likelihood of pregnancy and likelihood of live birth within 12 months prior to endline. This is a critical outcome for the study because to the extent that fertility patterns change, then impacts on children born during the study cannot be interpreted in a simple way.
iv. Health knowledge and sanitation practices.
v. Disease burden and mortality. Mortality is measured as the likelihood that an individual member of the household from baseline has died prior to endline. Disease burden is measured as the prevalence of fever, fever with diarrhea or vomiting, or coughing with blood at endline,
vi. Health-seeking behavior/preventative care. We focus on the share of pregnancies resulting in births in medical facilities, the share of children under two years of age with at least one vaccination in the prior year, and the share of children under two years of age with a complete dose of vaccines.
vii. Household productive assets. Note this is a subset of the primary outcome household non-land wealth.
viii. Housing quality. Two measures are used: the self-reported replacement cost of the current dwelling (irrespective of ownership status), and an index of housing construction quality, constructed from measures of wall and roof materials and from the number of rooms in the dwelling.

See PAP

Villages enrolled in the study were drawn from two districts: Kayonza and Nyabihu. A computer was used to conduct the randomization by the researchers based on a frame of villages agreed to by CRS and government officials. Randomization occurred at the village level across 248 villages. Of these, 74 were assigned to the Gikuriro intervention, 74 were assigned to the control group (no intervention), and 61 were assigned to GiveDirectly household grants in a range around the anticipated equivalent cost of Gikuriro (19-21 villages in each of 3 transfer amount cells). A final 34 villages were assigned to the large GiveDirectly transfer intended to maximize cost effectiveness of cash.

The total GD costs (inclusive of overhead) chosen to bracket expected Gikuriro based on ex-ante estimation of costs were $77, $119, $152. Once GD overheads are deducted, this translates into $41.32, $83.63, and $116.91 actually transferred to beneficiaries in GD villages. Then, the large transfer amount selected to optimize GiveDirectly’s benefit/cost ratio was $532 actually transferred to households. All transfer amounts were translated into Rwandan Francs at an exchange rate of 790 RwF per US dollar, and were rounded to the nearest hundred.

The Village-level component of the randomization, as well as household-level choice experiment components, were randomized by computer by the research team.

The key treatment assignment was at the village (cluster) level; this provides the assignment to arm, including the cash transfer amount from GiveDirectly. Some components of the GiveDirectly intervention, including whether they were assigned to lump-sum transfers, monthly flow transfers, or the choice between these, were randomized at the household level.

Yes

248 village clusters.

1,494 'eligible' households, 1,294 'ineligible' households across the 248 study villages.

74 villages were assigned to the Gikuriro intervention, 74 were assigned to the control group (no intervention), and 61 were assigned to GiveDirectly household grants in a range around the anticipated equivalent cost of Gikuriro (19-21 villages in each of 3 transfer amount cells). A final 34 villages were assigned to the large GiveDirectly transfer intended to maximize cost effectiveness of cash.