High-Frequency Program Monitoring and Bureaucratic Performance: Experimental Evidence from India

We study the impact of a centralized high-frequency monitoring (HFM) system in the context of government cash transfer program in the Indian state of Telangana. This program, known as the Rythu Bandhu Scheme, is a flagship initiative of the state government that provides land-owning farmers with Rs. 4000 (approximately USD60) for each acre of land that they own (total value of approximately USD2 billion per year). This money is intended to be distributed prior to each the growing season and used for agricultural inputs such as fertilizer and seeds.

Prior to the distribution of checks and deployment of the HFM system, the relevant bureaucrats in treatment areas (mandal agricultural officers) will be informed that they have been randomly selected for introduction of the HFM system. They will be informed about the types of data that will be collected and told that a report from their mandal will be provided to them and their superiors, including a publicly shared ranking of implementation in their mandal relative to other treatment mandals. Implementation information will be collected from 100 farmers per mandal, and the report will be generated as described. We will compare the implementation of the Rythu Bandhu scheme in areas with HFM systems in place against those where the system is not in place.

We propose to evaluate the effects of a new method for collecting high-quality, real-time data on program implementation: a high-frequency monitoring (HFM) system based on outbound phone calls to program beneficiaries. Under the proposed HFM system, a call center will place phone calls to program beneficiaries that collect information on their experience with a scheme. This information is collated and provided to the relevant parties within government.

In the context of our evaluation, this type of information aggregation system can have two main effects: 1) Motivate frontline officials towards better performance by providing reliable information on their performance to their superiors; and 2) Generate real-time data on locally-specific problems for resolution by local officials.

We study the effects of such a system in the context of government cash transfer program in the Indian state of Telangana in April-May 2018. This program, known as the Rythu Bandhu Scheme (RBS), is a flagship initiative of the state government that provides land-owning farmers with Rs. 4000 (approximately USD60) for each acre of land that they own. This money is intended to be distributed prior to each the growing season and used for agricultural inputs such as fertilizer and seeds. The money is distributed to farmers in the form of “order cheques”, which can be exchanged for cash at any bank branch of the bank listed on the cheque with identification that matches the name and unique id number listed on the cheque. The distribution of these cheques is carried out at a series of village meetings over the course of 10 days with the Department of Agriculture. The individuals distributing the cheques are agricultural extension workers, who are overseen by “mandal agricultural officers”. These mandal agricultural officers are the focus of our intervention and oversee the Department of Agriculture work within the level of a “mandal” (a geographical agglomeration of approximately 65,000 people).

Given the value of money being distributed (approximately US$2 billion per year), the state government was concerned about problems in implementation. Among their worries were that cheques would not reach the intended beneficiaries, that there may be corruption during the distribution process (e.g. those distributing the cheques demanded payment for handing them out), or that there would be delays in distribution, a major concern given the need to purchase agricultural inputs within a particular time window.

The high frequency monitoring call center collects information on: 1) whether the farmer received their cheque; 2) date of cheque receipt; 3) whether they encashed their cheque; 4) problems in receipt of the check (e.g. asked to pay money, delays); 5) problems in encashing of cheque; 6) satisfaction with the scheme; 7) other feedback on the scheme. Calls will be made via some combinations of human phone surveyors and interactive voice response calls, where recipients indicate their responses via a touch-tone system. We will test the relative efficacy of each system and provide recommendations on best practices going forward (e.g. how to reweight responses from automated calls). The HFM system will complete calls with approximately 100 farmers per mandal in each round of calling. This information will be compiled into reports to state, district, and mandal level officials. These reports will not identify individual phone respondents for the sake of their privacy, only aggregate information.

2018-05-10

2019-03-15

Percent of beneficiaries who receive their checks within the target distribution period without any irregularities (e.g. having to pay a bribe); Percent of beneficiaries who report being satisfied with program implementation; Value of program to beneficiaries

The value of the Rythu Bandhu scheme to beneficiaries is equal to the total size of the cheque that they received minus the cost of receiving the cheque. We will measure the cost of accessing the transfer based on wait time (multiplied by some estimate of time cost), and trips to bank (transport, waiting cost), and put in Rs terms. This allows us to directly compare the benefits of the scheme with the cost of implementation (i.e. call center). These questions will only be asked on the field household survey. We wll sum the licit costs, the illicit, and the total received cheque value and regress those values against the treatment rather than regressing individual components against the treatment (since there are so many components).

a) Support for political party in power;
b) During the endline survey, we will collect data from three sources: (1) Interactive Voice Response Phone Survey; (2) Human Surveyor Phone Surveys; and (3) Field Surveys. This is not part of the impact evaluation, but we wish to measure how closely the first two data sources replicate the field surveys.

The respondents will be asked “Do you approve of the work of the current state government, led by KCR and the Telangana Rashtra Samithi?” and “In the next elections in 2019, do you plan to vote for the Telangana Rashtra Samithi?”

Our experimental design has one treatment and one control group. Mandal agricultural officers (MAOs) will be randomly assigned to one of two groups. Due to the randomization, we can compare across the groups to determine the effect of implementing a high frequency monitoring system.

Our experimental design has one treatment and one control group. Mandal agricultural officers (MAOs) will be randomly assigned to one of two groups. Due to the randomization, we can compare across the groups to determine the effect of implementing a high frequency monitoring system.

In the treatment group (120 MAOs):
- Prior to the distribution of cheques and start of the HFM system, treatment MAOs are informed via letter and video conference that they have been selected for a new pilot program. They will be informed about the HFM system, and the types of data that will be collected.
- They will also be told that regular reports from the HFM system will be provided to them and their superiors so that they can better respond to problems. In addition to getting reports, they will be told that this information will generate an implementation performance rating for their mandal
- In order to reduce the risk of spillovers, treatment MAOs are explicitly told the identify of other treatment MAOs in their district and that no other mandals in their district are part of the pilot program.
- After the HFM data is collected, treatment MAOs are provided with a report on program implementation in their mandal. These reports will also be provided to higher level department staff, along with the mandal rating.
- After the initial report is issued, regular performance data will continue to be collected and reported to the MAOs on whether the implementation situation has improved in these mandals.

In the control group (remaining MAOs):
- These MAOs will not be explicitly informed about the existence of the HFM pilots in other mandals. If they ask, they will be told that the data collection may occur in future rounds of RBS in their mandals, such as the next season, but not in the present round.
- HFM data will still be collected from 30-100 farmers per mandal. This data will be used to evaluate the effects of HFM and for district level reports, but not to generate reports on the performance of control MAOs.

randomization done in office by a computer

Mandal (a unit of population containing approximately 65,000 individuals)

Yes

564 mandals

564 mandals, which contain approximately 28 million individuals that fall within either treatment or control. We will collect data from approximately 50,000 farmers from within these mandals.

132 treatment mandals, 420 control mandals

3.5 percentage points

20180601 TS HFM PaP - round 1 HFM and admin data