Back to History Current Version

High-Frequency Program Monitoring and Bureaucratic Performance: Experimental Evidence from India

Last registered on October 08, 2018

Pre-Trial

Trial Information

General Information

Title
High-Frequency Program Monitoring and Bureaucratic Performance: Experimental Evidence from India
RCT ID
AEARCTR-0002942
Initial registration date
April 27, 2018

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
April 30, 2018, 1:55 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
October 08, 2018, 11:30 PM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
University of Southern California

Other Primary Investigator(s)

PI Affiliation
UC San Diego
PI Affiliation
UC San Diego
PI Affiliation
University of Virginia

Additional Trial Information

Status
In development
Start date
2018-05-10
End date
2019-03-15
Secondary IDs
Abstract
A fundamental challenge for public service delivery is obtaining real-time data on the quality of service delivery to hold front-line service providers accountable. Such data can reduce corruption and facilitate targeted intervention by higher level decision makers. In this project, we evaluate the effect of a centralized, phone-based high-frequency monitoring (HFM) system in the context of a government cash transfer program. We study whether the introduction of a HFM system improves the performance of bureaucrats overseeing a cash transfer program in India. We will conduct a randomized experiment in which the HFM system is rolled out in some parts of the state and then compare outcomes across treatment and control areas to determine whether the HFM system is effective in improving program implementation.
External Link(s)

Registration Citation

Citation
Muralidharan, Karthik et al. 2018. "High-Frequency Program Monitoring and Bureaucratic Performance: Experimental Evidence from India." AEA RCT Registry. October 08. https://doi.org/10.1257/rct.2942-3.1
Former Citation
Muralidharan, Karthik et al. 2018. "High-Frequency Program Monitoring and Bureaucratic Performance: Experimental Evidence from India." AEA RCT Registry. October 08. https://www.socialscienceregistry.org/trials/2942/history/196359
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
We study the impact of a centralized high-frequency monitoring (HFM) system in the context of government cash transfer program in the Indian state of Telangana. This program, known as the Rythu Bandhu Scheme, is a flagship initiative of the state government that provides land-owning farmers with Rs. 4000 (approximately USD60) for each acre of land that they own (total value of approximately USD2 billion per year). This money is intended to be distributed prior to each the growing season and used for agricultural inputs such as fertilizer and seeds.

Prior to the distribution of checks and deployment of the HFM system, the relevant bureaucrats in treatment areas (mandal agricultural officers) will be informed that they have been randomly selected for introduction of the HFM system. They will be informed about the types of data that will be collected and told that a report from their mandal will be provided to them and their superiors, including a publicly shared ranking of implementation in their mandal relative to other treatment mandals. Implementation information will be collected from 100 farmers per mandal, and the report will be generated as described. We will compare the implementation of the Rythu Bandhu scheme in areas with HFM systems in place against those where the system is not in place.
Intervention Start Date
2018-05-10
Intervention End Date
2019-03-15

Primary Outcomes

Primary Outcomes (end points)
Percent of beneficiaries who receive their checks within the target distribution period without any irregularities (e.g. having to pay a bribe); Percent of beneficiaries who report being satisfied with program implementation; Value of program to beneficiaries
Primary Outcomes (explanation)
The value of the Rythu Bandhu scheme to beneficiaries is equal to the total size of the cheque that they received minus the cost of receiving the cheque. We will measure the cost of accessing the transfer based on wait time (multiplied by some estimate of time cost), and trips to bank (transport, waiting cost), and put in Rs terms. This allows us to directly compare the benefits of the scheme with the cost of implementation (i.e. call center). These questions will only be asked on the field household survey. We wll sum the licit costs, the illicit, and the total received cheque value and regress those values against the treatment rather than regressing individual components against the treatment (since there are so many components).

Secondary Outcomes

Secondary Outcomes (end points)
a) Support for political party in power;
b) During the endline survey, we will collect data from three sources: (1) Interactive Voice Response Phone Survey; (2) Human Surveyor Phone Surveys; and (3) Field Surveys. This is not part of the impact evaluation, but we wish to measure how closely the first two data sources replicate the field surveys.
Secondary Outcomes (explanation)
The respondents will be asked “Do you approve of the work of the current state government, led by KCR and the Telangana Rashtra Samithi?” and “In the next elections in 2019, do you plan to vote for the Telangana Rashtra Samithi?”

Experimental Design

Experimental Design
Our experimental design has one treatment and one control group. Mandal agricultural officers (MAOs) will be randomly assigned to one of two groups. Due to the randomization, we can compare across the groups to determine the effect of implementing a high frequency monitoring system.
Experimental Design Details
Our experimental design has one treatment and one control group. Mandal agricultural officers (MAOs) will be randomly assigned to one of two groups. Due to the randomization, we can compare across the groups to determine the effect of implementing a high frequency monitoring system.

In the treatment group (120 MAOs):
- Prior to the distribution of cheques and start of the HFM system, treatment MAOs are informed via letter and video conference that they have been selected for a new pilot program. They will be informed about the HFM system, and the types of data that will be collected.
- They will also be told that regular reports from the HFM system will be provided to them and their superiors so that they can better respond to problems. In addition to getting reports, they will be told that this information will generate an implementation performance rating for their mandal
- In order to reduce the risk of spillovers, treatment MAOs are explicitly told the identify of other treatment MAOs in their district and that no other mandals in their district are part of the pilot program.
- After the HFM data is collected, treatment MAOs are provided with a report on program implementation in their mandal. These reports will also be provided to higher level department staff, along with the mandal rating.
- After the initial report is issued, regular performance data will continue to be collected and reported to the MAOs on whether the implementation situation has improved in these mandals.

In the control group (remaining MAOs):
- These MAOs will not be explicitly informed about the existence of the HFM pilots in other mandals. If they ask, they will be told that the data collection may occur in future rounds of RBS in their mandals, such as the next season, but not in the present round.
- HFM data will still be collected from 30-100 farmers per mandal. This data will be used to evaluate the effects of HFM and for district level reports, but not to generate reports on the performance of control MAOs.
Randomization Method
randomization done in office by a computer
Randomization Unit
Mandal (a unit of population containing approximately 65,000 individuals)
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
564 mandals
Sample size: planned number of observations
564 mandals, which contain approximately 28 million individuals that fall within either treatment or control. We will collect data from approximately 50,000 farmers from within these mandals.
Sample size (or number of clusters) by treatment arms
132 treatment mandals, 420 control mandals
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
3.5 percentage points
IRB

Institutional Review Boards (IRBs)

IRB Name
Institute for Financial Management and Research
IRB Approval Date
2018-04-21
IRB Approval Number
IRB00007107
Analysis Plan

Analysis Plan Documents

20180601 TS HFM PaP - round 1 HFM and admin data

MD5: 39f5bd3a4edf44a51cc70407c228e6a4

SHA1: 51c3073dc076c7e9435cf30819a6082308142d57

Uploaded At: June 01, 2018

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
July 31, 2018, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
October 31, 2018, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
498 mandal agricultural officers (MAOs); 122 in treatment
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
22,167 farmers contacted over phone; 5,645,937 farmers with administrative data (bank records)
Final Sample Size (or Number of Clusters) by Treatment Arms
122 MAOs in treatment, 378 MAOs in control
Data Publication

Data Publication

Is public data available?
Yes

Program Files

Program Files
Yes
Reports, Papers & Other Materials

Relevant Paper(s)

Abstract
Improving “last-mile” public service delivery is a recurring challenge in developing countries. Could the widespread adoption of mobile phones provide a scalable, cost-effective means for improvement? We use a large-scale experiment to evaluate the impact of phone based monitoring on a program that transferred nearly a billion dollars to 5.7 million Indian farmers. In randomly selected jurisdictions, officials were informed that program implementation would be measured via calls with beneficiaries. This led to a 7.8 percent
reduction in the number of farmers who did not receive their transfers. The program was highly cost-effective, costing 3.6 cents for each additional dollar delivered.
Citation
Muralidharan, Karthik, Paul Niehaus, Sandip Sukhtankar, and Jeffrey Weaver. "Improving last-mile service delivery using phone-based monitoring." American Economic Journal: Applied Economics 13, no. 2 (2021): 52-82.

Reports & Other Materials