Primary Outcomes (explanation)
- Impact on health indicators: Over the two years of the program, the targeted maternal and child indicators, including prenatal visits, delivery by trained midwives, childhood immunizations, and growth monitoring, were an average of 0.04 standard deviations higher in incentivized areas than in non-incentivized areas. This effect was driven primarily by increases in the number of prenatal visits, which was 8.2 percent higher in incentivized areas than non-incentivized areas, and regular monthly weight checks for children under five, which were 4.5 percent higher in incentivized areas than non-incentivized areas. While these differences are modest, the impact of the incentives was more pronounced in areas with low baseline levels of service delivery: the incentives improved the targeted maternal and child health indicators as much as 0.11 – 0.14 standard deviations in the poorer, off-Java provinces.
- Impact on education indicators: There were no differences between incentivized and non-incentivized areas on the four education indicators examined (primary and junior secondary enrollment and attendance).
- Adverse impacts of incentives: Researchers found little evidence that providing incentives had adverse effects. There was no evidence of a multi-tasking problem, or that immunization records or school attendance was manipulated.
- Mechanism of impact: The results suggest two main channels through which the incentives may have had an impact. First, the incentives appear to have led to a 16 percent decrease in spending on school supplies and uniforms, and a 6.5 percent increase on health expenditures. Despite the reallocation of funds away from school supplies and uniforms, households were no less likely to receive these items and they were of no lesser value, and were actually more likely to receive education scholarships in the incentivized areas. The results suggest that the change in budgets resulted from more efficient spending. The incentives also led to an increase in the labor of midwives, who are the major providers of maternal and child health services in the area. By contrast, there was no change in labor supplied by teachers.
- Cost-effectiveness analysis: Researchers calculated the total benefits of the program as the total number of “bonus points” the program created, using the weighting scheme assigned to each of the 12 indicators to calculate a village’s performance. Researchers calculated that the Generasi program as a whole cost about US$8-US$11 to generate one additional bonus point. Translating bonus points back into outcomes suggests, for example, that the implied cost of preventing one malnourished child was US$384-US$528, and the cost of enrolling one more child in primary school was US$200-US$275. When the additional costs of the performance incentives are isolated, the cost of the incentives themselves comes out to US$0.60 per point (e.g., US$30 per additional malnourished child prevented). This suggests that while the program itself is not particularly cost-effective, providing incentives is a way to make it more cost-effective than a normal block grant program. Adding similar performance incentives to other, pre-existing block grant schemes (holding the total amount of money spent) could be a very cost-effective way to improve aid effectiveness.