Building state and citizenry: A randomized evaluation of a tax-collection campaign in the D.R. Congo

The primary intervention randomly assigns certain neighborhoods to receive an on-the-ground property and rental tax collection program, aided by tablet computers and handheld thermal receipt printers. This study will attempt to measure the impact of the program on citizen beliefs about the government and citizen efforts to hold the government accountable. Specifically, the study will exploit random variation in taxes paid to examine the theorized link between taxation and civic engagement. Additionally, two cross-randomized interventions will be used to study how to limit bribe taking among tax collectors administering this new program. First, a collector monitoring (“audit”) intervention will be randomly assigned among neighborhoods that receive the on-the-ground taxation program. Second, a citizen-empowerment information intervention will be randomly assigned among all neighborhoods in the city.

2016-04-18

2016-12-30

For the main intervention (random assignment of on-the-ground tax collection), the main outcome variables are as follows:
- Tax receipts and government revenues: measured using official government data.
- Citizen views about the government: measured with survey questions, implicit-association tests, and behavioral games.
- Citizen expectations of the government: measured with survey questions.
- Citizen engagement in politics and efforts to hold the government accountable: measured with survey questions, feedback experiments, public meeting experiments (to determine show-up rates), election turnout data (if accessible).

For the cross-randomized interventions aimed at reducing corruption among tax collectors, outcome variables are as follows:
- Availability of valid receipt: e.g. if a participant has a printed receipt from the program.
- Discrepancies between citizen receipts and official program records: e.g. if a tax collector issues a receipt but does not submit this money to the tax ministry.
- Discrepancies between the amounts citizens report paying and official program data: e.g. if a household reports paying X (without regard to whether the household has a receipt) and in the official tax ministry data, the reported amount is X-b.
- The amount citizens report paying, including bribe amounts less than the official tax.

For the main intervention, variables will be constructed as follows:
- Tax receipts and government revenues: these outcomes will simply be taken from official government data.
- Citizen views about the government
o Survey questions: standardized indices of underlying survey questions will be constructed and used as the principal outcomes of interest. Individual survey questions will be examined to explore mechanisms and heterogeneity, but only in the case where results for other individual questions are reported.
o Implicit-association tests: the standard D-score from the psychology literature will be reported.
o Behavioral games: the standard “amount sent” to the player 2 will be used as the dependent variable.
- Citizen expectations of the government
o Survey questions. Standardized indices of underlying survey questions will be constructed and used as the principal outcome of interest. Individual survey questions will be examined to explore mechanisms and heterogeneity, but only in the case where results for other individual questions are reported. Vignette-style “fill in the outcome” questions will also be used.
- Citizen engagement in politics and efforts to hold the government accountable
o Survey questions: standardized indices of underlying survey questions will be constructed and used as the principal outcome of interest. Individual survey questions will be examined to explore mechanisms and heterogeneity, but only in the case where results for other individual questions are reported. Additionally, a survey-based quiz about the government will be used to measure citizen knowledge of the government. Finally, a separate survey module will be administered in which participants can choose to continue receiving information about the government if they so desire; the decision to continue is taken as an interest in engaging more in politics.
o Government feedback experiments: whether or not individuals choose to send feedback to the government via several channels created by the program. Indicator variables will be calculated whether a citizen makes the effort to provide feedback. When possible more continuous measures will be defined using the distance from an individual’s house to the location where one deposits a feedback form for example.
o Public meeting experiments: whether or not an individual shows up will determine a dummy variable outcome; again, analysis with this variable will condition on the distance from a person’s house to the location.
o Election data: a dummy variable will be defined if an individual votes in the December 2016 elections (likely to be postponed until sometime in 2017) again, analysis with this variable will condition on the distance from a person’s house to the location. If official polling station data are unavailable, self-reports may be used.

For the cross-randomized interventions, outcome variables will be constructed as follows:
- Availability of valid receipt: an indicator that equals 1 if a citizen possesses a valid receipt, conditional on reporting having paid the tax.
- Discrepancies between citizen receipts and official program data: an indicator that equals 1 if a household reported having paid the tax but there is no record in the official tax ministry data.
- Discrepancies between the amounts citizens report paying and official program data: the difference between the amount citizens report paying and the amount recorded in the official data of the tax ministry. This quantity will be examined as well as an indicator variable that equals 1 if the above quantity is not zero.
- The precise amount citizens report paying: this will just be an integer measured in Congolese Francs. In addition to self reports, two other more subtle elicitation methods will be used since the quantity of the bribe is potentially sensitive information.

The information intervention will be administered in randomly selected neighborhoods during a baseline survey conducted before the start of the tax program. Audits will be administered in selected neighborhoods during the roll out of the program, which will take an estimated 6-8 months to complete. Collection of outcomes of the effects on citizens’ expectations of the governments and their efforts to hold it accountable will begin roughly 6 months after program roll out.

Before the start of the tax program, enumerators administer a baseline survey in all 431 polygons in Kananga. They implement the active information treatment in 216 of these polygons, and the control information treatment in 215 polygons. In other words, the information intervention is cross-randomized across all polygons (treatment and control) with respect to the main taxation intervention. To achieve a random sample within each polygon, enumerators follow a skip pattern, selecting every X house to complete a baseline survey. X is determined by an estimate of the size of the polygon to yield at least 5 surveys per polygon. In addition to houses selected for the baseline survey, enumerators also distribute information fliers to every 5th house in the polygon.

Among the 431 polygons in the city of Kananga, 253 are selected to receive the on-the-ground tax program aided by tablets and mobile thermal printers. The remaining 178 polygons will keep the old system in which individuals are supposed to themselves report to the tax ministry to pay their taxes. Before tax collectors start collecting taxes in a given polygon, they conduct a brief census in which they visit every house in the polygon and take down a few pieces of information about the residents that is relevant to tax collection; during this visit they also assign a unique household code.

After the census is completed in a given polygon, the tax collectors start tax collection. Tax collectors are randomly assigned to polygons in pairs. They will receive a small weekly performance-based bonus for working on the campaign.

Audit surveys administered by enumerators begin once tax agents have finished the first phase of collection in a polygon. Enumerators visit houses within a polygon to ask information about households’ experiences with the tax collectors. They conduct such surveys in both treated and control polygons (to make sure collectors are respecting polygon boundaries). Within treated polygons, they conduct these visits in audited and un-audited polygons. However, only information from audited polygons is shared with the tax ministry leadership. Tax collectors are informed before starting work in a polygon whether or not the results from these audits will be made known to their supervisors. Results from audit surveys will be compared to the official program data housed at the tax ministry to determine the evaluation statistics to be reported in audited polygons (as well as the corruption outcome variables specified above).

The impact of the tax program on citizens’ beliefs and behaviors will be measured by enumerators six months after the launch of the program. Enumerators will visit households in treated and control polygons, randomly sampling which households receive the full suite of questions and activities. The government plans to roll out the tax collection program in control polygons, upon completion of the study.

The random assignment of treatments to polygons will be conducted in STATA using a specified seed. Additionally, randomized selection for different survey modules will occur using built-in randomization software in the SurveyODK survey program. Random assignment of tax collectors to polygons and to partners will be conducted in R using a specified seed.

The randomization unit for assigning the tax program, the audit treatment, and the information treatment is the polygon. As noted above polygons are drawn on a satellite map of the city. They follow geographic features that are easily distinguishable from the ground, i.e. streets, ravines, known city administrative boundaries, etc.

Yes

The 431 polygons in the city of Kananga.

Estimated sample sizes are 2720 individuals for survey-based variables and 1360 individuals for experimental variables.

On-the-ground tax collection program (main treatment): 1600
Old tax collection program (control): 1120
Information treatment: 1360
Audit treatment: 800

In other words, within the taxed polygons there is a 2x2 matrix with the following combinations of treatments: audit + information, audit + no information, no audit + information, no audit + no information. Each of these cells has 400 units. Half of the control units also receive the information treatment.

The pre-analysis plan will specify the full set of power estimates based on the sampclus command in STATA, Optimal Design, and simulations in R. In short, power calculations were conducted using data gathered in Kananga in previous years. I do not have access to the exact outcome variables I plan to collect. However, I do have several analogs that were conducted for other research in Kananga in past years. Specifically, a version of the Random Allocation Game with the Government as the Player 2 was administered in 2013-2014, which is one of the anticipated measures of views of the government. The outcome is the amount allocated to the second player, an integer between 0 and 3000 Congolese Francs (CF). Second, in past years, information on trust in the provincial government was collected; this is a reasonable approximation of some of the survey-based outcome variables I aim to use gauging citizens’ views of the government. This question uses an ordinal 1-5 Likert-style scale, increasing in trust. The data are on the individual level. The standard deviation for the RAG outcome is 441.25 (CF), and the standard deviation for the trust survey outcome is 1.05. Standard errors are clustered by polygon. In the simplest specification, regressing the outcome on a treatment dummy, the estimated MDE with power of 0.8 for the RAG outcome is 0.201 and 0.204 for the trust questions. In a regression that includes dummies for all four treatment cells (audit + information, etc), MDEs on the coefficients of interest range from 0.26 to 0.29. For the corruption outcomes, the only available proxy for the anticipated dependent variables is a binary bribe indicator taken from surveys with motorbike drivers at tolls in Kananga (in a separate experiment with Otis Reid also currently underway). The standard deviation for this variable is 0.47. According to this estimated dependent variable and a power level of 0.80, the MDE for the coefficient on dummies for each treatment cell of interest (audit + information, etc) range from 0.185 to 0.191. Again, see the Pre-Analysis Plan for more details on these power calculations.

Pre-Analysis Plan