Project Generasi: Conditional community block grants in Indonesia

Last registered on April 26, 2017

Pre-Trial

Trial Information

General Information

Title
Project Generasi: Conditional community block grants in Indonesia
RCT ID
AEARCTR-0000332
Initial registration date
May 15, 2014

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
May 15, 2014, 3:46 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
April 26, 2017, 11:56 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Primary Investigator

Affiliation
MIT

Other Primary Investigator(s)

PI Affiliation
World Bank
PI Affiliation
World Bank
PI Affiliation
World Bank

Additional Trial Information

Status
Completed
Start date
2007-06-01
End date
2013-12-01
Secondary IDs
Abstract
We report an experiment in 3,000 villages that tested whether incentives improve aid efficacy. Villages received block grants for maternal and child health and education that incorporated relative performance incentives. Subdistricts were randomized into incentives, an otherwise identical program without incentives, or control. Incentives initially improved preventative health indicators, particularly in underdeveloped areas, and spending efficiency increased. While school enrollments improved overall, incentives had no differential impact on education, and incentive health effects diminished over time. Reductions in neonatal mortality in non-incentivized areas did not persist with incentives. We find no scoring manipulation and no funding reallocation toward richer areas.

In 2016-2017, we returned to the same locations for a followup study. Since 2010 all locations were using the incentivized version of the program, but the original randomization into treatment and control subdistricts was almost entirely intact.
External Link(s)

Registration Citation

Citation
Olken, Benjamin et al. 2017. "Project Generasi: Conditional community block grants in Indonesia." AEA RCT Registry. April 26. https://doi.org/10.1257/rct.332-3.0
Former Citation
Olken, Benjamin et al. 2017. "Project Generasi: Conditional community block grants in Indonesia." AEA RCT Registry. April 26. https://www.socialscienceregistry.org/trials/332/history/16921
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
A total of 264 subdistricts were randomized into either a comparison group or one of the two versions of the Generasi program: the “incentivized” version with a pay-for-performance component, or the otherwise identical, “non-incentivized” version without pay-for-performance incentives.
Intervention Start Date
2007-07-02
Intervention End Date
2010-01-01

Primary Outcomes

Primary Outcomes (end points)
Health, education, impacts of incentives, cost effectiveness of program
Primary Outcomes (explanation)
- Impact on health indicators: Over the two years of the program, the targeted maternal and child indicators, including prenatal visits, delivery by trained midwives, childhood immunizations, and growth monitoring, were an average of 0.04 standard deviations higher in incentivized areas than in non-incentivized areas. This effect was driven primarily by increases in the number of prenatal visits, which was 8.2 percent higher in incentivized areas than non-incentivized areas, and regular monthly weight checks for children under five, which were 4.5 percent higher in incentivized areas than non-incentivized areas. While these differences are modest, the impact of the incentives was more pronounced in areas with low baseline levels of service delivery: the incentives improved the targeted maternal and child health indicators as much as 0.11 – 0.14 standard deviations in the poorer, off-Java provinces.

- Impact on education indicators: There were no differences between incentivized and non-incentivized areas on the four education indicators examined (primary and junior secondary enrollment and attendance).

- Adverse impacts of incentives: Researchers found little evidence that providing incentives had adverse effects. There was no evidence of a multi-tasking problem, or that immunization records or school attendance was manipulated.

- Mechanism of impact: The results suggest two main channels through which the incentives may have had an impact. First, the incentives appear to have led to a 16 percent decrease in spending on school supplies and uniforms, and a 6.5 percent increase on health expenditures. Despite the reallocation of funds away from school supplies and uniforms, households were no less likely to receive these items and they were of no lesser value, and were actually more likely to receive education scholarships in the incentivized areas. The results suggest that the change in budgets resulted from more efficient spending. The incentives also led to an increase in the labor of midwives, who are the major providers of maternal and child health services in the area. By contrast, there was no change in labor supplied by teachers.

- Cost-effectiveness analysis: Researchers calculated the total benefits of the program as the total number of “bonus points” the program created, using the weighting scheme assigned to each of the 12 indicators to calculate a village’s performance. Researchers calculated that the Generasi program as a whole cost about US$8-US$11 to generate one additional bonus point. Translating bonus points back into outcomes suggests, for example, that the implied cost of preventing one malnourished child was US$384-US$528, and the cost of enrolling one more child in primary school was US$200-US$275. When the additional costs of the performance incentives are isolated, the cost of the incentives themselves comes out to US$0.60 per point (e.g., US$30 per additional malnourished child prevented). This suggests that while the program itself is not particularly cost-effective, providing incentives is a way to make it more cost-effective than a normal block grant program. Adding similar performance incentives to other, pre-existing block grant schemes (holding the total amount of money spent) could be a very cost-effective way to improve aid effectiveness.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The randomization was conducted at the subdistrict (kecamatan) level, so all villages within a subdistrict either received the same version (either all incentivized or all non- incentivized) or were in the control group. Since some services (e.g. health services, junior secondary schools) service multiple villages within the same subdistrict, but rarely serve people from other subdistricts, randomizing at the subdistrict level and treating all villages within the subdistrict estimates the program’s true net impact, rather than possible reallocations among villages. A total of 264 eligible subdistricts were randomized into either one of the two treatment groups or the control group.

In the first year, both treatment groups received program funds based on the number of target beneficiaries in each village (i.e. the number of children and the expected number of pregnant women). In the second year, funds were allocated in the same way in non-incentivized villages, but in incentivized villages 20 percent of the funds were distributed based on village’s performance on the 12 indicators during the last year. The purpose of this bonus was to increase villages’ effort at achieving the targeted indicators, both by encouraging a more effective allocation of funds and by stimulating increased efforts to encourage mothers and children to obtain appropriate health care and increase educational enrollment and attendance.

The block grants averaged US$8,500 in the first year of the program and US$13,500 in the second year of the program. To decide how to spend the grant money, trained facilitators helped each village elect an 11-member village management team, as well as select local facilitators and volunteers. Consultation workshops were then held with local health and education providers to gather information and technical assistance, and coordinate the use of the funds.
Experimental Design Details
Randomization Method
By computer
Randomization Unit
Subdistrict (kecamatan) level
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
264 subdistricts with 3,120 villages
Sample size: planned number of observations
12,306 Households; 2,315 villages; 300 community health centers; 1,177 village midwives; 1,197 primary schools; 867 junior secondary schools; 2,397 village head post cadres
Sample size (or number of clusters) by treatment arms
Total Subdistricts in initial randomization:
- Incentivized Generasi: 100
- Non-incentivized Generasi: 100
- Control: 100

But 36 subdistricts were not eligible because they had been selected (prior to randomization) to receive other programs.
- Sample of villages for implementation: 264 sudistricts @ 12 villages. Total is 3,120 villages.
- The sample for survey covers 300 districts. In each subdistrict, eight villages were randomly selected (unless the subdistrict had fewer than 8 villages, in which case all were selected). Total is 2,313 villages sampled for the survey.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
MIT COUHES
IRB Approval Date
2008-07-14
IRB Approval Number
0806002801
IRB Name
Harvard University Committee on Use of Human Subjects in Research
IRB Approval Date
2006-04-28
IRB Approval Number
F13663-101 (determined that it was exempt)
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Yes
Data Collection Completion Date
January 01, 2010, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
246 subdistricts
Was attrition correlated with treatment status?
Final Sample Size: Total Number of Observations
Final Sample Size (or Number of Clusters) by Treatment Arms
Data Publication

Data Publication

Is public data available?
Yes

Program Files

Program Files
Yes
Reports, Papers & Other Materials

Relevant Paper(s)

Abstract
We report an experiment in 3,000 villages that tested whether incentives improve aid efficacy. Villages received block grants for maternal and child health and education that incorporated relative performance incentives. Subdistricts were randomized into incentives, an otherwise identical program without incentives, or control. Incentives initially improved preventative health indicators, particularly in underdeveloped areas, and spending efficiency increased. While school enrollments improved overall, incentives had no differential impact on education, and incentive health effects diminished over time. Reductions in neonatal mortality in non-incentivized areas did not persist with incentives. We find no scoring manipulation and no funding reallocation toward richer areas.
Citation
Olken, Benjamin A., Junko Onishi, and Susan Wong, "Should Aid Reward Performance? Evidence from a field experiment on health and education in Indonesia," American Economic Journal: Applied Economics, forthcoming.

Reports & Other Materials