x

NEW UPDATE: Completed trials may now upload and register supplementary documents (e.g. null results reports, populated pre-analysis plans, or post-trial results reports) in the Post Trial section under Reports, Papers, & Other Materials.
State Paralysis: The Impacts of Procurement Risk on Government Effectiveness
Last registered on August 03, 2020

Pre-Trial

Trial Information
General Information
Title
State Paralysis: The Impacts of Procurement Risk on Government Effectiveness
RCT ID
AEARCTR-0006255
Initial registration date
August 01, 2020
Last updated
August 03, 2020 12:32 PM EDT
Location(s)

This section is unavailable to the public. Use the button below to request access to this information.

Request Information
Primary Investigator
Affiliation
University of Zurich
Other Primary Investigator(s)
PI Affiliation
CONASEMS
PI Affiliation
CONASEMS
PI Affiliation
Fundação Getulio Vargas
PI Affiliation
UC Davis
PI Affiliation
London School of Economics
Additional Trial Information
Status
In development
Start date
2020-08-03
End date
2020-12-31
Secondary IDs
Abstract
Public procurement plays a key role in allocating limited budgetary resources to public service delivery in countries with a functional rule of law. This project studies a puzzling phenomenon: in developing countries like Brazil, substantive shares of the federal and sub-national budgets are not spent despite clear needs for additional resources to improve the quality of public services or to fund emergency spending in contexts of crisis. In line with a growing literature that documents the potential unintended effects of the enforcement of rules on bureaucratic performance, we investigate the role of procurement risk - when passive waste is misinterpreted as active waste – as a driver of unspent public funds by Brazilian municipal governments. Randomizing interventions that decrease procurement risk or that make it less salient within a sample of municipal health secretariats, we investigate its effects on budget execution and health outputs and outcomes, in the context of the COVID-19 pandemic.
External Link(s)
Registration Citation
Citation
Faleiros, Daniel et al. 2020. "State Paralysis: The Impacts of Procurement Risk on Government Effectiveness." AEA RCT Registry. August 03. https://doi.org/10.1257/rct.6255-1.0.
Sponsors & Partners
Partner(s)
Type
government
Experimental Details
Interventions
Intervention(s)
This study introduces exogenous variation in procurement risk by randomly assigning a simple intervention to support local bureaucrats in complying with procurement regulations: a tutorial to guide local bureaucrats in using templates for public procurement documents (from terms of reference to auction procedures), provided by the Brazilian federal government to support municipalities in the context of COVID-19. While those templates were made available to all municipalities, our hypothesis is that most have not effectively put them to use, precisely because they lack the high-capacity personnel to ensure compliance with the complex procurement requirements such documents allude to. The intervention (the tutorial) has the potential to not only shed light on this mechanism, but also to provide a simple and cost-effective policy tool to help mitigate adverse effects of external enforcement on public service delivery, particularly in this time of crisis.

To disentangle the effects of information from those of salience, since tutorials might induce local bureaucrats to worry about procurement risk to a lesser extent regardless of their informational content, the experiment also randomizes a salience intervention: an alternative tutorial to take local bureaucrats through a recent decision by a Brazilian Supreme Court Justice, which determined that bureaucrats cannot be punished by ‘honest mistakes’ in their attempts to manage the COVID-19 crisis. While the intervention provides no guidance in handling procurement risk, it could presumably mitigate the effects of the latter on budget execution and public service delivery.
Intervention Start Date
2020-08-03
Intervention End Date
2020-08-31
Primary Outcomes
Primary Outcomes (end points)
Our baseline data will comprise survey answers by municipal health secretaries about perceived procurement risk, main challenges in budget execution and public service delivery in the context of COVID-19, captured by five questions asked before the intervention. Additionally, right after the interventions, participants will be requested to list a high-official in the municipality to receive further materials about the topic of the tutorial, and asked whether s/he would undertake a costly action (in terms of opportunity costs of time) to connect with other public managers on the topic of the tutorial, by informing all time slots they would be available to meet (in a structure meant to emulate a BDM elicitation procedure). Last, the survey has two optional questions at the end about the allocation of federal transfers to fight COVID-19 and the main difficulties linked to budget execution of those funds. Since those questions are after the intervention, they would allow to test for treatment effects on perceived risks, and to elicit experiment demand effects on previous budget allocations.
The follow-up dataset will repeat the 5 initial questions, and ask specifically about the usage of the templates provided by the Brazilian Attorney General’s Office.

Besides survey data, we have access to monthly data on federal transfers and respective budget leftovers for all municipalities, from CONASEMS. For health budget execution we have quarterly data from SIOPS (the federal system that monitors expenditures of the National Health System). For other, more detailed budget execution metrics (such as planned spending, delivery rates and payment rates, funded by both transfers and municipalities’ own budget), we will try to get access to quarterly data for the States of Ceará, Maranhão, Minas Gerais, Pernambuco, Piauí, Rio Grande do Norte, Rio Grande do Sul, Rio de Janeiro, Sao Paulo and Tocantins (based on contract-level data).

When it comes to health outputs and outcomes, we have monthly data from DATASUS for all municipalities. For other outcomes, such as education, we will investigate dropout rates (School Census in March/21) and standardized test scores (Prova Brasil in November/21). It is possible, nonetheless, we will not find impacts on those, as our intervention is in the early stage of public procurement. Other bottlenecks in execution and payment may prevent budget execution from increasing to a greater extent, or other constraints to quality spending might prevent higher execution from translating into improved public service delivery.
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
The experiment will be implemented in collaboration CONASEMS, the Brazilian Council of Municipal Health Secretaries. It will take place during the month of August/2020, when municipal health secretaries and other local policymakers will receive a short online questionnaire about their experiences in the context of the COVID-19 response, particularly when it comes to their main challenges for budget execution and public service delivery during the pandemic.

Immediately after the survey questions, respondents will be randomly assigned to 3 groups.

Group 1 (Information). The first group will be shown a video tutorial on how to use the templates for procurement documents elaborated by Brazilian Attorney General’s Office (Advocacia Geral da União, AGU). We will also highlight that some of the strictest State Courts of Accounts have endorsed those templates.

Group 2 (Salience). The second group will be shown a video tutorial about a Supreme Court decision clarifying that only ‘grotesque’ misconducts by public managers in the context of COVID-19 would be subject to legal action.

Group 3 (Control). The third group will be shown a video tutorial on Sanitary and Health Law reports elaborated by the Brazilian National Council of Health Secretaries (CONASS) and made available on the CONASEMS website – completely unrelated to procurement.

All tutorials share a symmetric structure and very similar lengths, and showcase websites where participants can find all relevant information about the object of the tutorial they have been assigned to. We will measure take-up and usage of the tutorials and templates through questions following the intervention and in a follow-up wave, planned to take place one month after the baseline.
Experimental Design Details
Not available
Randomization Method
Randomization done on the spot by Qualtrics, stratified by State (because of heterogeneous external enforcement by each State Court of Accounts) and by quartile of budget leftovers from federal transfers (as a fraction of total health transfers to the municipality) as of December/2019 (our measure of budget execution at baseline).
Randomization Unit
Municipality
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
~2,850 municipalities
Sample size: planned number of observations
~2,850 municipalities
Sample size (or number of clusters) by treatment arms
~950 municipalities control, ~950 municipalities information, ~950 municipalities salience
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Power calculations indicate that, under a 50% take-up rate of the online survey (in line with CONASEMS’s prior experience), we could detect treatment effects of at least 1.46 percentage points on municipalities’ planning rate (1.69 percentage points differences between the information and salience groups), based on the sample variance for that outcome computed from Lichand and Fernandes (2020); and of at least 0.09 standard deviation on standardized health outputs and outcomes (0.11 differences between the information and salience groups).
Supporting Documents and Materials

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
University of Zurich Department of Economics
IRB Approval Date
2020-07-15
IRB Approval Number
OEC IRB # 2020-045
Analysis Plan

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information