Seeking the Truth: Designing a Performance Incentive, Auditing, and Reporting System for Local Bureaucrats

Last registered on April 26, 2021

Pre-Trial

Trial Information

General Information

Title
Seeking the Truth: Designing a Performance Incentive, Auditing, and Reporting System for Local Bureaucrats
RCT ID
AEARCTR-0007558
Initial registration date
April 24, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
April 26, 2021, 10:45 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
UC San Diego

Other Primary Investigator(s)

Additional Trial Information

Status
On going
Start date
2021-01-26
End date
2022-04-26
Secondary IDs
Abstract
Understanding how to improve the performance of frontline bureaucrats is critical, since they are ultimately responsible for the delivery of public services and the implementation of nearly all development programs. Adopting private sector management practices like performance-based incentives is challenging with frontline bureaucrats, since they often collect the data that is used to assess their performance and serve multiple policy objectives. We partner with a government department of an Indian state to redesign their performance assessment and data collection systems, and leverage this setting to experimentally study how incentives for performance and truthful reporting dynamically shape bureaucrat performance and truthfulness.
External Link(s)

Registration Citation

Citation
Brownstone, Steven. 2021. "Seeking the Truth: Designing a Performance Incentive, Auditing, and Reporting System for Local Bureaucrats." AEA RCT Registry. April 26. https://doi.org/10.1257/rct.7558-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
Treatment A provides panchayat secretaries (PSes) performance-based feedback via a scorecard page in the application that shows the PSes their absolute level of performance, highlights areas for improvement, and conveys relative performance within the district.

Treatment B informs PSes that their self-reported performance data will be “truthed” by comparing it against the findings of inspections by the mandal panchayat officer (MPO) (the PS’s boss), and significant deviations will be penalised. This is operationalized by a truth score page within their reporting application. The page will inform PSes whether their data quality has been flagged and why.

Treatment C provides the same data quality flags to the MPOs, in addition to the data quality flags to the PSes, based on data reported by third party auditors.
Intervention Start Date
2021-04-28
Intervention End Date
2021-10-26

Primary Outcomes

Primary Outcomes (end points)
The primary endpoint is the change in the performance score calculated from the third party auditor’s inspections using the department’s importance weights for each audited indicator.
Primary Outcomes (explanation)
Please see the analysis plan for a detailed description of the primary outcome variable will be constructed.

Secondary Outcomes

Secondary Outcomes (end points)
Citizen satisfaction with PS services from phone survey

MPO subjective ranking of PS quality from phone survey

Sarpanch subjective satisfaction with PS.

Per-capita expenditures at the GP
Secondary Outcomes (explanation)
Sarpanch, MPO, and citizen satisfaction are based on a simple rating out of ten. The per-capita expenditures at the GPs are fund flows to the GP for public works through the PRRD department.

Experimental Design

Experimental Design
Initially 75% of mandals will receive treatment A. After approximately two months, 50% of mandals will receive treatment A. After approximately 4 months, the control mandals will receive treatment A. After approximately 6 months, the 50% of mandals that had been receiving treatment B will receive treatment C and the remaining will receive treatment B. The exact timeline is dependent on the decisions of our government partners.
Experimental Design Details
Not available
Randomization Method
All 539 mandals in Telangana were randomly assigned to one of four groups. The randomization was stratified by district and the number of GPs per mandal within the district. Specifically, within each district the mandals were grouped into groups of 4 by size; these quartets were randomized. The remainder mandals in each district were randomized to minimize treatment imbalances within each district.

After randomization, 34 smaller mandals had their MPOs re-assigned to help inspect GPs in 32 large mandals. Of these re-assignments 5 mandals crossed treatment groups. The 5 mandals where the assignment crossed treatment groups will be excluded from the analysis when the analysis covers periods where the mandals had different treatments.
Randomization Unit
Mandal which is the administrative unit above the gram panchayat. An MPO is responsible for the PSes within a mandal.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
539 mandals
Sample size: planned number of observations
We hope to sample 9 gram panchayats in each mandal every two month measurement cycle.
Sample size (or number of clusters) by treatment arms
134 mandals initially control, 405 mandals initially treatment A. Then, 134 control, 135 treatment A, 270 treatment B. Then 269 treatment A and 270 treatment B. Finally, 269 treatment B and 270 treatment C.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Assuming an icc of .1, a basline correlation of .3, and 80% power, the minimal detectable effect size when comparing 269 and 270 mandals is .11 standard deviations. When comparing 134 and 405 mandals the minimal detectable effect size is .13 standard deviations.
IRB

Institutional Review Boards (IRBs)

IRB Name
UCSD Human Research Protections Program
IRB Approval Date
2021-01-26
IRB Approval Number
201817S
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information