State Paralysis: The Impacts of Procurement Risk on Government Effectiveness

Last registered on December 02, 2020

Pre-Trial

Trial Information

General Information

Title
State Paralysis: The Impacts of Procurement Risk on Government Effectiveness
RCT ID
AEARCTR-0006255
Initial registration date
August 01, 2020

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
August 03, 2020, 12:32 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
December 02, 2020, 1:30 PM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
University of Zurich

Other Primary Investigator(s)

PI Affiliation
CONASEMS
PI Affiliation
CONASEMS
PI Affiliation
Fundação Getulio Vargas
PI Affiliation
UC Davis
PI Affiliation
London School of Economics

Additional Trial Information

Status
In development
Start date
2020-08-03
End date
2020-12-31
Secondary IDs
Abstract
Public procurement plays a key role in allocating limited budgetary resources to public service delivery in countries with a functional rule of law. This project studies a puzzling phenomenon: in developing countries like Brazil, substantive shares of the federal and sub-national budgets are not spent despite clear needs for additional resources to improve the quality of public services or to fund emergency spending in contexts of crisis. In line with a growing literature that documents the potential unintended effects of the enforcement of rules on bureaucratic performance, we investigate the role of procurement risk - when passive waste is misinterpreted as active waste – as a driver of unspent public funds by Brazilian municipal governments. Randomizing interventions that decrease procurement risk or that make it less salient within a sample of municipal health secretariats, we investigate its effects on budget execution and health outputs and outcomes, in the context of the COVID-19 pandemic.
External Link(s)

Registration Citation

Citation
Faleiros, Daniel et al. 2020. "State Paralysis: The Impacts of Procurement Risk on Government Effectiveness." AEA RCT Registry. December 02. https://doi.org/10.1257/rct.6255
Experimental Details

Interventions

Intervention(s)
This study introduces exogenous variation in procurement risk by randomly assigning a simple intervention to support local bureaucrats in complying with procurement regulations: a tutorial to guide local bureaucrats in using templates for public procurement documents (from terms of reference to auction procedures), provided by the Brazilian federal government to support municipalities in the context of COVID-19. While those templates were made available to all municipalities, our hypothesis is that most have not effectively put them to use, precisely because they lack the high-capacity personnel to ensure compliance with the complex procurement requirements such documents allude to. The intervention (the tutorial) has the potential to not only shed light on this mechanism, but also to provide a simple and cost-effective policy tool to help mitigate adverse effects of external enforcement on public service delivery, particularly in this time of crisis.

To disentangle the effects of information from those of salience, since tutorials might induce local bureaucrats to worry about procurement risk to a lesser extent regardless of their informational content, the experiment also randomizes a salience intervention: an alternative tutorial to take local bureaucrats through a recent decision by a Brazilian Supreme Court Justice, which determined that bureaucrats cannot be punished by ‘honest mistakes’ in their attempts to manage the COVID-19 crisis. While the intervention provides no guidance in handling procurement risk, it could presumably mitigate the effects of the latter on budget execution and public service delivery.
Intervention Start Date
2020-08-03
Intervention End Date
2020-08-31

Primary Outcomes

Primary Outcomes (end points)
Our baseline data will comprise survey answers by municipal health secretaries about perceived procurement risk, main challenges in budget execution and public service delivery in the context of COVID-19, captured by five questions asked before the intervention. Additionally, right after the interventions, participants will be requested to list a high-official in the municipality to receive further materials about the topic of the tutorial, and asked whether s/he would undertake a costly action (in terms of opportunity costs of time) to connect with other public managers on the topic of the tutorial, by informing all time slots they would be available to meet (in a structure meant to emulate a BDM elicitation procedure). Last, the survey has two optional questions at the end about the allocation of federal transfers to fight COVID-19 and the main difficulties linked to budget execution of those funds. Since those questions are after the intervention, they would allow to test for treatment effects on perceived risks, and to elicit experiment demand effects on previous budget allocations.
The follow-up dataset will repeat the 5 initial questions, and ask specifically about the usage of the templates provided by the Brazilian Attorney General’s Office.

Besides survey data, we have access to monthly data on federal transfers and respective budget leftovers for all municipalities, from CONASEMS. For health budget execution we have quarterly data from SIOPS (the federal system that monitors expenditures of the National Health System). For other, more detailed budget execution metrics (such as planned spending, delivery rates and payment rates, funded by both transfers and municipalities’ own budget), we will try to get access to quarterly data for the States of Ceará, Maranhão, Minas Gerais, Pernambuco, Piauí, Rio Grande do Norte, Rio Grande do Sul, Rio de Janeiro, Sao Paulo and Tocantins (based on contract-level data).

When it comes to health outputs and outcomes, we have monthly data from DATASUS for all municipalities. For other outcomes, such as education, we will investigate dropout rates (School Census in March/21) and standardized test scores (Prova Brasil in November/21). It is possible, nonetheless, we will not find impacts on those, as our intervention is in the early stage of public procurement. Other bottlenecks in execution and payment may prevent budget execution from increasing to a greater extent, or other constraints to quality spending might prevent higher execution from translating into improved public service delivery.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The experiment will be implemented in collaboration CONASEMS, the Brazilian Council of Municipal Health Secretaries. It will take place during the month of August/2020, when municipal health secretaries and other local policymakers will receive a short online questionnaire about their experiences in the context of the COVID-19 response, particularly when it comes to their main challenges for budget execution and public service delivery during the pandemic.

Immediately after the survey questions, respondents will be randomly assigned to 3 groups.

Group 1 (Information). The first group will be shown a video tutorial on how to use the templates for procurement documents elaborated by Brazilian Attorney General’s Office (Advocacia Geral da União, AGU). We will also highlight that some of the strictest State Courts of Accounts have endorsed those templates.

Group 2 (Salience). The second group will be shown a video tutorial about a Supreme Court decision clarifying that only ‘grotesque’ misconducts by public managers in the context of COVID-19 would be subject to legal action.

Group 3 (Control). The third group will be shown a video tutorial on Sanitary and Health Law reports elaborated by the Brazilian National Council of Health Secretaries (CONASS) and made available on the CONASEMS website – completely unrelated to procurement.

All tutorials share a symmetric structure and very similar lengths, and showcase websites where participants can find all relevant information about the object of the tutorial they have been assigned to. We will measure take-up and usage of the tutorials and templates through questions following the intervention and in a follow-up wave, planned to take place one month after the baseline.
Experimental Design Details
CONASEMS has access to the universe of municipal health secretariats (5,570), which we take as our sampling frame. The survey will be distributed through CONASEMS supporters’ network, 216 staff members (often former municipal health secretariats) who help coordinate CONASEMS’s engagement with the health secretariats. The network is a partition of all secretariats, based on geographical proximity, with each supporter responsible for reaching out to a unique subset of health secretariats.

We expect take-up rates around 50% -- based on the previous experience of CONASEMS with online surveys. We do not pre-assign municipalities to treatment conditions; rather, randomization will be done on the spot, stratified by State (because of heterogeneous external enforcement by each State Court of Accounts) and by quartile of budget leftovers from federal transfers (as a fraction of total health transfers to the municipality) as of December/2019 (our measure of budget execution at baseline). The intervention is scheduled to be rolled out on the first week of August.

Within the universe of 5,570 municipal health secretaries to whom the online questionnaire will be sent, a third is expected to be assigned to each treatment arm. There might be slight deviations from those fractions based on the number of municipalities who eventually take up the survey within each stratum.
Randomization Method
Randomization done on the spot by Qualtrics, stratified by State (because of heterogeneous external enforcement by each State Court of Accounts) and by quartile of budget leftovers from federal transfers (as a fraction of total health transfers to the municipality) as of December/2019 (our measure of budget execution at baseline).
Randomization Unit
Municipality
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
~2,850 municipalities
Sample size: planned number of observations
~2,850 municipalities
Sample size (or number of clusters) by treatment arms
~950 municipalities control, ~950 municipalities information, ~950 municipalities salience
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Power calculations indicate that, under a 50% take-up rate of the online survey (in line with CONASEMS’s prior experience), we could detect treatment effects of at least 1.46 percentage points on municipalities’ planning rate (1.69 percentage points differences between the information and salience groups), based on the sample variance for that outcome computed from Lichand and Fernandes (2020); and of at least 0.09 standard deviation on standardized health outputs and outcomes (0.11 differences between the information and salience groups).
Supporting Documents and Materials

Documents

Document Name
End-line survey instrument
Document Type
survey_instrument
Document Description
End-line survey instrument
File
End-line survey instrument

MD5: d0f2fa8787e54b219414578a0c71c529

SHA1: a2fca3332d51c9c18170626e1dcfe15f4fa548cf

Uploaded At: December 02, 2020

Document Name
Treament: second dose
Document Type
other
Document Description
Second dose of the treatment (email sent to study participants within each group)
File
Treament: second dose

MD5: cea8b7642bde0561750f33cfad36de2f

SHA1: 60e36347bca895114947ca248ee711b3734ffddb

Uploaded At: December 02, 2020

Document Name
IRB Approval Document
Document Type
irb_protocol
Document Description
Zurich IRB approval
File
IRB Approval Document

MD5: 051bc44ef8a6c2dbbc39353c14a26a35

SHA1: ded1fd201f9319d9dc6b6f6ddf069ac4e6705853

Uploaded At: August 01, 2020

IRB

Institutional Review Boards (IRBs)

IRB Name
University of Zurich Department of Economics
IRB Approval Date
2020-07-15
IRB Approval Number
OEC IRB # 2020-045
Analysis Plan

Analysis Plan Documents

Updated pre-analysis plan (uploaded before analyzing treatment effects and before the end-line survey)

MD5: ef7fef4f3c7c3854f286fec521a18d02

SHA1: 61a38d03471d54dde5dacd5a71b192378a6f2bdc

Uploaded At: December 02, 2020

PaP_State+Paralysis_registration.pdf

MD5: ee295ecbb4947c0a3f0f20658147e23a

SHA1: 78666e18e260115060bcd273eadebb4c444c91b0

Uploaded At: August 01, 2020

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials