The overarching research objectives in our evaluation are to assess the impact of social accountability (SA) interventions in UP on: (1) objective measures of health service quality in practice, (2) village-level satisfaction with health services, and (3) village-level health outcomes.
Importantly, the evaluation will seek to identify whether information provision, which is a standard part of social accountability interventions, has an independent effect on outcomes that is comparable to the combined effect of information provision and community engagement / facilitation. We also test the effectiveness of alternative models of delivering information in order to inform implementation of accountability interventions at large scale in public policy settings. Given that the community health workers targeted by the intervention focus on maternal and child health, our measures of health system performance will also emphasize maternal and child health. Measures of health service quality include availability of services such as immunization and primary care services, distribution of food and nutritional supplementation as recommended in the national nutrition program, and provider absenteeism. Measures of satisfaction with local health services will include process measures such as availability of service, waiting time for services and whether or not community members perceive that they are treated by providers with respect as well as general subjective assessments of satisfaction with services. Health outcome measures will include child anthropometrics (weight-for-age and weight-for-height), self-reported morbidity in the preceding two weeks (diarrhea, cough, fever, headache, days of usual activities lost due to illness), and neonatal (0-28 days), infant (under age one), and child (under age five) mortality as well as maternal health indicators such as percentage of facility deliveries among mothers who gave birth in the past year.
The SA interventions were designed to address two specific aims: (1) to test the impact of SA interventions when implemented at scale as a policy instrument, and (2) to study mechanisms through which the information and facilitation interventions lead to increasing collective action efforts at the village level and improve governance of public service delivery at the local level. We investigate these two specific aims in two parallel sets of interventions.
51-Block intervention: The SA intervention will be implemented in 51 out of 87 blocks in 10 districts (covering a population of over 27 million in these districts). This intervention will include information dissemination as well as facilitation of community participation in Village Health, Sanitation and Nutrition Committees (VHSNCs). These activities will be led by Gram Panchayat Coordinators (GPCs), recruited and trained by the State Institute of Rural development (SIRD).
120 village intervention: In order to study mechanisms through which social accountability interventions might lead to improvement in outcomes, the intervention is also implemented in 120 villages that were selected at random within 2 districts. The SA interventions aim to distinguish the effect of provision of information and facilitated engagement of community members, from that of the effect of information alone. The 120 villages are randomized to either a control arm, or one of two treatment arms described below.
TREATMENT ARM 1: INFORMATION & AWARENESS
Community members will receive information about their health care rights and entitlements, and about certain health outcomes in their village. Additionally, they will receive notifications of upcoming health-related activities happening in their village. Importantly, besides assessing the effect of providing information on health service delivery and various health outcomes, our evaluation will also determine how best to provide the information.
Another innovation in this project is to disseminate monthly information about health system related issues using interactive voice response messages (IVRs), phone calls, or home visits to households in the treatment villages. IVRs include a brief message about a health (or health system) indicator that is specific to the village, or information regarding upcoming VHSNC meetings or Village Health and Nutrition Days (VHNDs), as well as a response option that can be used to collect data on what information households have received and about their participation in VHSNC meetings/VHNDs. These IVRs will be sent out every month to cover approximately 24,000 households in 80 treatment villages. During preparatory phase that was conducted in parallel with the baseline survey, we collected detailed data on social networks in each village and identified central individuals in the village.
We will use the data on networks in villages, combined with the information that we disseminate on a monthly basis to econometrically estimate how information dissemination within networks affects awareness and participation in social accountability activities as well as in utilization of health services.
TREATMENT ARM 2: INFORMATION PLUS COMMUNITY ENGAGEMENT
The community engagement component aims to enhance the participation of the community in creating social accountability. The intervention will provide trained facilitators to help community members engage in a participatory process with VHSNCs and PRIs and identify key deficiencies for improvement in health services that most concern community members. The facilitators are trained to help organize meetings and are provided a detailed checklist of activities that need to be undertaken prior to the day of the meetings such as informing all community representatives about the date, inviting the block level officers and ensuring that logistics requirements for VHNDs are conveyed to VHSNC members in advance of the meetings. The facilitated meetings with healthcare providers and local and block level representatives aim to empower community members to demand better health services and convey these demands more effectively to providers and officials. The three key health workers at the village level (ASHA, ANM, and AWW) report to the local (village level) elected representatives and block level authorities, who receive feedback from the community in the accountability interventions. Moreover, through repeated community meetings village-level health workers are expected to respond to transparency and accountability innovations by improving quality of services delivered to their local constituents resulting in improvements in population health outcomes.
These interventions will focus on services delivered by village-level health workers including those providing primary care and maternal and child health services. The households surveyed in the project will be those with children less than 5 years of age. The interventions will cover and include all population subgroups, castes, and religious minorities in the treatment villages, and also collect data on all of these subgroups.