Seeking the Truth: Designing a Performance Incentive, Auditing, and Reporting System for Local Bureaucrats

Treatment A provides panchayat secretaries (PSes) performance-based feedback via a scorecard page in the application that shows the PSes their absolute level of performance, highlights areas for improvement, and conveys relative performance within the district.

Treatment B informs PSes that their self-reported performance data will be “truthed” by comparing it against the findings of inspections by the mandal panchayat officer (MPO) (the PS’s boss), and significant deviations will be penalised. This is operationalized by a truth score page within their reporting application. The page will inform PSes whether their data quality has been flagged and why.

Treatment C provides the same data quality flags to the MPOs, in addition to the data quality flags to the PSes, based on data reported by third party auditors.

2021-04-28

2021-10-26

The primary endpoint is the change in the performance score calculated from the third party auditor’s inspections using the department’s importance weights for each audited indicator.

Please see the analysis plan for a detailed description of the primary outcome variable will be constructed.

Citizen satisfaction with PS services from phone survey

MPO subjective ranking of PS quality from phone survey

Sarpanch subjective satisfaction with PS.

Per-capita expenditures at the GP

Sarpanch, MPO, and citizen satisfaction are based on a simple rating out of ten. The per-capita expenditures at the GPs are fund flows to the GP for public works through the PRRD department.

Initially 75% of mandals will receive treatment A. After approximately two months, 50% of mandals will receive treatment A. After approximately 4 months, the control mandals will receive treatment A. After approximately 6 months, the 50% of mandals that had been receiving treatment B will receive treatment C and the remaining will receive treatment B. The exact timeline is dependent on the decisions of our government partners.

All 539 mandals in Telangana were randomly assigned to one of four groups. The randomization was stratified by district and the number of GPs per mandal within the district. Specifically, within each district the mandals were grouped into groups of 4 by size; these quartets were randomized. The remainder mandals in each district were randomized to minimize treatment imbalances within each district.

After randomization, 34 smaller mandals had their MPOs re-assigned to help inspect GPs in 32 large mandals. Of these re-assignments 5 mandals crossed treatment groups. The 5 mandals where the assignment crossed treatment groups will be excluded from the analysis when the analysis covers periods where the mandals had different treatments.

Mandal which is the administrative unit above the gram panchayat. An MPO is responsible for the PSes within a mandal.

Yes

539 mandals

We hope to sample 9 gram panchayats in each mandal every two month measurement cycle.

134 mandals initially control, 405 mandals initially treatment A. Then, 134 control, 135 treatment A, 270 treatment B. Then 269 treatment A and 270 treatment B. Finally, 269 treatment B and 270 treatment C.

Assuming an icc of .1, a basline correlation of .3, and 80% power, the minimal detectable effect size when comparing 269 and 270 mandals is .11 standard deviations. When comparing 134 and 405 mandals the minimal detectable effect size is .13 standard deviations.