Are Voters Updating when Audit Reports are Informative?

Last registered on May 29, 2020

Pre-Trial

Trial Information

General Information

Title
Are Voters Updating when Audit Reports are Informative?
RCT ID
AEARCTR-0005932
Initial registration date
May 28, 2020

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
May 29, 2020, 3:39 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
University of Oxford

Other Primary Investigator(s)

PI Affiliation
No Affiliation

Additional Trial Information

Status
In development
Start date
2020-06-01
End date
2021-02-28
Secondary IDs
C93
Abstract
This experimental design is implemented in collaboration with the NGO Chile Transparencia. Beginning in February 2020 the Chilean Comptroller General Office rolled out 167 audits in municipalities throughout the country. The treatment consists of the audit report for each municipality in addition to a short information video that highlights the extent to which the municipality has high or low malfeasance counts. We have recruited approximately 49,000 online subjects from these municipalities (plus a control group of 177 non-audited municipalities). All 10,000 subjects will receive a pre-treatment survey measuring their pre-treatment beliefs regarding corruption in their municipality.

Within each audited municipality subjects randomly assigned to the treatment will receive a video publicizing the municipality's audit report -- those randomly assigned to control will receive a placebo ad. We organize the outcomes from this information treatment into three groupings: 1) beliefs about the prevalence of corruption; 2) the salience of these beliefs, and 3) whether these beliefs affect voting behaviour. Relatively few of these audit studies directly measure belief updating as a response to malfeasance information treatments.

Treated subjects are targeted with video treatment within five days of the release of the audit report. A post-treatment survey will be conducted for two weeks and one month after the audit report is released. They are questioned again regarding their beliefs about the prevalence of malfeasance in their municipal government. With similar pre-treatment measures we are able to fairly precisely measure belief updating. In addition, we have subjects in control who are targeted with a placebo video treatment.
External Link(s)

Registration Citation

Citation
Duch, Raymond and Felipe Torres Raposo. 2020. "Are Voters Updating when Audit Reports are Informative?." AEA RCT Registry. May 29. https://doi.org/10.1257/rct.5932-1.0
Sponsors & Partners

Sponsors

Partner

Experimental Details

Interventions

Intervention(s)
Our focus here is on individual-level priors regarding malfeasance in municipal governance and whether individuals update when they are informed about malfeasance in their local municipality. The information in question here is the 2020 municipal audits that will be conducted in 168 of the Chilean municipalities. The content of these audits will provide the basis for the information treatments that we implement in this experiment.

The malfeasance information treatment essentially summarizes the level of malfeasance the Contraloria reports for each municipality. All of the treated subjects see two pieces of information: One element is a simple count of the number of irregularities identified by the Contraloria audit. And a second provides a summary score that is generated by the Contraloria and reflects both the irregularities committed but also their seriousness and their importance relative to the size of the municipal budget. These are presented to subjects on a horizontal colour-coded scale anchored on the left by 0 (Green/Very Good) and on the right by 100 (Red/Very Bad).
%-- does a message about malfeasance in government matter? Scaling up is another matter. But ideally to address this first issue we want to compare a large N of individuals who are randomly assigned to treatment (and who we know are treated) to a large N of individuals randomly assigned to control (and who we know have not been treated).

Subjects are also informed about the relative malfeasance performance of their municipality. There are two versions of the information treatment: ``spatial'' and ``temporal''. A temporal benchmark compares the municipality's performance (irregularities plus score) to its previous audits. We do this by placing the benchmark information on an identical horizontal axis just below the actual audit results. A spatial benchmark version compares the municipality's performance to an average of municipalities from the last audit.
Intervention Start Date
2020-07-20
Intervention End Date
2020-10-24

Primary Outcomes

Primary Outcomes (end points)
-Beliefs about levels of malfeasance
-Attitudes towards the municipal government
-Vote intention
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The experimental design builds on an annual audit that is conducted by the CGO of municipal governments. For the year 2020, the CGO has selected 167 municipalities for an audit. Our audit information treatments are designed in conjunction with the CGO's roll-out of these municipal audits.

The audit outcomes will provide the content for the information treatments that we will implement via WhatsApp. We are leveraging the fact that the audit outcomes will vary -- the number of reported irregularities varies quite significantly across municipalities. Figure 1 in our pre-analysis summarizes the distribution of municipality irregularities reported by the Contraloria audits in 2016. On average, the audits report about 30 irregularities. The sample frame aims to have a balance of subjects in municipalities with above average, ``negative'', audit reports and with below-average, ``positive'', audit reports.

The treatment condition in the experiment is receiving a short video summarizing the results of the CGO audit report. Our subjects are individuals residing in any of the 167 municipalities audited by the CGO in 2020. The video treatments are customized for each municipality. As a result, we expect the treatments to range from very positive -- virtually no irregularities -- to very negative -- in the neighborhood of 90-100 irregularities. We can think of the treatment as a positive or negative "dosage'' rather than a dichotomous treatment arm. As we pointed before, the outcomes of interest are the subjects' beliefs about the prevalence of corruption in municipal government; their attitudes regarding municipal governance; and their vote intentions. The outcome variables are measured at three different periods in the experiment: pre-treatment; post-treatment; and pre-election.
Experimental Design Details
The experimental design builds on an annual audit that is conducted by the CGO of municipal governments. For the year 2020, the CGO has selected 167 municipalities for an audit. Our audit information treatments are designed in conjunction with the CGO's roll-out of these municipal audits.

The audit outcomes will provide the content for the information treatments that we will implement via WhatsApp. We are leveraging the fact that the audit outcomes will vary -- the number of reported irregularities varies quite significantly across municipalities. Figure 1 in our pre-analysis summarizes the distribution of municipality irregularities reported by the Contraloria audits in 2016. On average, the audits report about 30 irregularities. The sample frame aims to have a balance of subjects in municipalities with above average, ``negative'', audit reports and with below-average, ``positive'', audit reports.

The treatment condition in the experiment is receiving a short video summarizing the results of the CGO audit report. Our subjects are individuals residing in any of the 167 municipalities audited by the CGO in 2020. The video treatments are customized for each municipality. As a result, we expect the treatments to range from very positive -- virtually no irregularities -- to very negative -- in the neighbourhood of 90-100 irregularities. We can think of the treatment as a positive or negative "dosage'' rather than a dichotomous treatment arm. An important consideration in recruiting the subject pool is ensuring balance over the range of irregularities illustrated in Figure \ref{fig:irregular}. As we pointed before, the outcomes of interest are the subjects' beliefs about the prevalence of corruption in municipal government; their attitudes regarding municipal governance; and their vote intentions. The outcome variables are measured at three different periods in the experiment: pre-treatment; post-treatment; and pre-election.
Randomization Method
Randomisation will be done using R.
Randomization Unit
Individuals eligible to vote
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
10,000 eligible voters
Sample size: planned number of observations
10,000 eligible voters
Sample size (or number of clusters) by treatment arms
1,500 voters Spatial/Low
1,500 voters Temporal/Low
1,500 voters = 1,500

750 voters Spatial/High
750 voters Temporal/High
500 voters Placebo

2,000 placebo pure control(non-audited municipalities)
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
For more details, please find our power calculations in our pre-analysis plan.
Supporting Documents and Materials

Documents

Document Name
Ethical Approval Nuffield Centre for Experimental Social Sciences
Document Type
irb_protocol
Document Description
Ethical Approval Nuffield Centre for Experimental Social Sciences
File
Ethical Approval Nuffield Centre for Experimental Social Sciences

MD5: 4bc59c48d08f474c3e2ee690878087da

SHA1: e6d25d918cfeeb41528f0a9d082df4fed5f6a598

Uploaded At: May 27, 2020

Document Name
Ethical Approval University of Oxford
Document Type
irb_protocol
Document Description
Ethical Approval University of Oxford
File
Ethical Approval University of Oxford

MD5: 1b740392a068016913f9fc52bd4299df

SHA1: 98076e3228313e04473a501cebf1d12253c3f44c

Uploaded At: May 27, 2020

Document Name
Ethical Approval University of Santiago
Document Type
irb_protocol
Document Description
Ethical Approval University of Santiago Chile
File
Ethical Approval University of Santiago

MD5: 77bfa133907ea66c3663f778af1851b5

SHA1: 698b9ae6c53331dfbcec07f36abb52475eece576

Uploaded At: May 27, 2020

IRB

Institutional Review Boards (IRBs)

IRB Name
Universidad de Santiago de Chile
IRB Approval Date
2020-04-08
IRB Approval Number
104/2020
IRB Name
University of Oxford
IRB Approval Date
2020-03-31
IRB Approval Number
R68234/RE001
IRB Name
Nuffield Centre for Experimental Social Sciences
IRB Approval Date
2020-01-17
IRB Approval Number
OE_0048
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials