x

We are happy to announce that all trial registrations will now be issued DOIs (digital object identifiers). For more information, see here.
Technology, audit risk, and retail tax evasion: Evidence from an experiment in Rwanda
Last registered on May 13, 2019

Pre-Trial

Trial Information
General Information
Title
Technology, audit risk, and retail tax evasion: Evidence from an experiment in Rwanda
RCT ID
AEARCTR-0004164
Initial registration date
May 01, 2019
Last updated
May 13, 2019 11:40 PM EDT
Location(s)
Region
Primary Investigator
Affiliation
Georgetown University
Other Primary Investigator(s)
PI Affiliation
Columbia University
PI Affiliation
Georgetown University
Additional Trial Information
Status
Completed
Start date
2017-01-01
End date
2019-05-01
Secondary IDs
Abstract
A central question for a country's economic development is how to improve the state's ability to raise revenues domestically in the presence of widespread tax evasion (Besley and Persson, 2009). In this context, the Value Added Tax (VAT) has become one of the most important instruments of revenue mobilization in the developing world. In principle, a VAT has several advantages in terms of tax compliance compared to other tax instruments, as it incentivizes accurate reporting of sales and purchases. Firms are essentially taxed on the difference between their sales and the cost of their inputs, and they must report both to the tax authority and keep a paper trail of the transactions (i.e., receipts) in their books. To minimize tax liability, firms would like to under-report sales, but to over-report inputs. This asymmetry should limit the room for "collusive evasion" in the case of business-to-business transactions, i.e., collusion between firms in misreporting transaction values. The existence of a paper trail should also deter "unilateral evasion", i.e., firms unilaterally misreporting transaction values, as the tax authority could crosscheck the values reported by a firm with records from third parties (i.e., suppliers and clients).

Pomeranz (2015) provided empirical evidence showing the importance of these properties of the VAT for tax compliance. Yet, in practice, limited enforcement capacity may prevent tax authorities from efficiently cross-checking what firms report against records from third parties, and thus limit unilateral evasion. To address this concern, several countries around the world, including Rwanda, have mandated the use of Electronic Billing Machines (EBMs), which generate a unique "hard-to-tamper-with" receipt for each transaction and send information on each receipt to the tax authority on a regular basis (e.g., in real time or daily). This technology limits the ability of firms to unilaterally misreport a transaction once a receipt for that transaction has been issued, which can be particularly effective for business-to-business transactions as firms have an incentive to ask for receipts for their input transactions (to decrease their tax liability). Eissa and Zeitlin (2015) show the roll-out of EBMs improved compliance in Rwanda.

However, final consumers do not have similar incentives to ask for receipts, so firms can more easily under-report sales to final consumers, an issue that can persist with EBMs if firms do not actually issue receipts to final consumers. Eissa and Zeitlin (2015) also document that firms failed to issue receipts in a large share of a sample of retail transactions made after the EBM roll-out in Rwanda. One potential policy response is to incentivize consumers to ask for receipts, e.g., with lotteries and tax rebates based on consumers’ receipts. Naritomi (AER forthcoming) provides evidence that such policies can effectively reduce the under-reporting of sales to final consumers. However, these policies can be quite expensive, as incentives have to be paid to all consumers.

Alternatively, governments can address this "last-mile" problem of the VAT with audits and related enforcement strategies at the retail level. In this light, Electronic Billing Machines can be seen as a technology that changes the cost of such enforcement strategies: by requiring firms to keep electronic accounts, they make it readily visible whether a firm is planning to declare revenue associated with a given transaction. This suggests that tax authorities can combat tax evasion by auditing firms' use of EBMs through unannounced visits, in which staff of the tax authority stop consumers leaving a store, check whether they were issued a valid EBM receipt, and apply penalties in case of non-compliance.

This study investigates the impact of such an enforcement strategy through a randomized control trial conducted in collaboration with the Rwandan Revenue Authority (RRA) with the aim of answering three questions:

1. What are the impacts of such an enforcement strategy on retailers' issuance of receipts, reporting of sales, and tax liability?

2. What is the impact of the associated increase in retailers' tax compliance on prices, i.e., what is the incidence on retailers vs. consumers?

3. Do firms' responses to audit risk depend on the audit risk of their competitors, i.e., does the impact of the treatment vary with the rate of treatment saturation among competitors?

The answer to the first question will allow us to evaluate the effectiveness of a policy to address the last-mile problem that is complementary to the EBM technology. The second question is important conceptually to understand the effects of enforcement policies and the possible public support for these policies. There is very little evidence on the incidence of tax evasion (Kopczuk et al., 2016). To the extent that the evasion rents are broadly shared with consumers (through lower retail prices), policies aiming to reduce tax evasion may face little support in the public as the cost of such policies (higher retail prices) may be more salient than their possible benefits (possible gains from increased government revenue). The answer to the third question is important to shed light on the mechanisms behind our answer to the second question. The ability of retailers to pass the costs associated with stronger tax enforcement onto prices likely depends on the competitive structure of the market in which they operate (e.g., monopoly, monopolistic competition, perfect competition) and on whether their competitors are also subject to the same costs. This is important because experimental enforcement interventions, such as the one we study, often focus on a subset of firms (e.g., often treat competing firms differently) but hope to inform the effect that such interventions could have when applied more broadly (e.g., treating all competing firms similarly).

In practice, this project evaluates the impact of an RRA initiative to send auditors to make surprise visits to retail firms in Kigali over a 10-month period and to observe whether ordinary consumers of those retailers are receiving receipts for their transaction. The evaluation combines innovations in measurement -- we combine VAT declarations with both real-time data on the universe of transactions reported through EBMs and receipt outcomes from natural transactions conducted by "mystery shoppers" sent to control and treatment firms throughout the study period for research purposes only -- with two dimensions of experimental variation: at the firm level, firms are assigned to (zero or) variable frequencies of audit, and across firms, we vary the saturation of the treatment among economic competitors to study strategic complementarities.
External Link(s)
Registration Citation
Citation
Eissa, Nada, Francois Gerard and Andrew Zeitlin. 2019. "Technology, audit risk, and retail tax evasion: Evidence from an experiment in Rwanda." AEA RCT Registry. May 13. https://doi.org/10.1257/rct.4164-1.0.
Former Citation
Eissa, Nada, Francois Gerard and Andrew Zeitlin. 2019. "Technology, audit risk, and retail tax evasion: Evidence from an experiment in Rwanda." AEA RCT Registry. May 13. https://www.socialscienceregistry.org/trials/4164/history/46414.
Experimental Details
Interventions
Intervention(s)
The intervention that we evaluate in this project is a new enforcement strategy conducted by the RRA. Concretely, RRA auditors make surprise visits to specific retail firms in Kigali on a frequent basis (the frequency of those visits varies experimentally across retailers, see below), stop customers leaving the retail shop, and ask to see their EBM receipt. If a valid EBM receipt is provided by the customer, the RRA auditor records the retailer as "compliant" for this visit. If the customer does not provide a valid EBM receipt, the RRA auditor enters the retail shop and consults the retailer's EBM directly for a record of the transaction. If no such record exists (i.e., no EBM receipt was issued), the RRA auditor records the retailer as "non-compliant" for this visit, which entails a set of warnings and penalties specified in the Rwandan legislation (the severity of penalties is increasing in the number of non-compliant visits recorded for a given retailer).

Treatment and control firms were visited by an RRA auditor prior to the start of the intervention, between September 2017 and November 2017, and handed a formal letter from the RRA. The letter reminded firms about their rights and responsibilities as VAT-paying firms, especially regarding the use of EBMs, and usual enforcement strategies. The 375 treated firms were also informed about the new enforcement strategy and the expected frequency of visits per month that they would be subject to in subsequent months (2 visits per month for 284 firms; 4 visits per month for 91 firms). Among control firms, 74 firms did not receive any pre-intervention visit by the RRA (and thus did not receive any letter). This will allow us to separate the effect of the RRA visit/letter and the effect of the enforcement intervention (EBM use audits) itself.

The intervention lasted from November 2017 to September 2018. During that period, RRA auditors conducted the EBM use audits in treated firms following the expected number of visits per month announced to each firm (accompanied by an enumerator from the research team who recorded various aspects of each audit, including the compliance outcome).
Intervention Start Date
2017-10-01
Intervention End Date
2018-10-01
Primary Outcomes
Primary Outcomes (end points)
The primary hypothesis that we will test in our study is whether the intervention had an impact on the following primary outcomes comparing outcomes in treated firms (pooling the two treatment groups together) to those in control firms (pooling the 2 control groups together).

Using the data generated by mystery shopper visits (visit level)
1. Whether or not an EBM receipt was issued
2. Log price paid for the basket of good purchased

Using the data from VAT declarations (quarterly level)
3. Total sales subject to VAT reported in the VAT declaration

Using the data generated by EBM
4. Total sales for goods subject to VAT for which EBM receipts were issued

Our pre-analysis plan provides full details of planned empirical specifications, test statistics, and inferential strategies.
Primary Outcomes (explanation)

The analysis will make use of three sources of data to create these primary outcome variables.

First, we will use data from 10 rounds of mystery shopper visits -- 2 rounds conducted prior to the intervention and 8 rounds conducted during the intervention period -- in both control and treated firms, in which enumerators recorded the use of EBM (whether a receipt was issued) and the price paid for the purchase of a similar pre-specified basket of goods in each firm. These data will allow us to create the first key variable to measure the impact of the intervention on compliance: whether the firm issues an EBM receipt when a customer makes a purchase.

By controlling for the basket of good purchased, we isolate the pure compliance effect from any bias coming from a correlation between compliance and type of good purchased. In so doing, we can also create a first key variable to measure the impact of the intervention on prices paid, and thus the incidence of tax evasion, isolating again the effect from any bias coming from a correlation between prices and types of good purchased.

Second, we will use administrative data from the universe of VAT declarations in Rwanda from January 2016 until December 2018. Rwandan firms that are subject to the VAT must make VAT declarations either on a monthly or a quarterly basis (depending on firm size). We will thus aggregate the data by quarter. The quarters 2016q1 to 2017q3 will be the pre-intervention quarters; the quarters 2017q4 to 2018q3 will be the intervention quarters; the quarter 2018q4 will be a post-intervention quarter. The primary outcome for this study based on these data will be the total sales subject to VAT reported by the firm in that quarter. This outcome will allow us to test whether changes in EBM compliance translated into impacts on the target for such a policy, i.e., the under-reporting of sales to final consumers subject to VAT in VAT declarations.

Third, we will use administrative data from the universe of transactions for which EBM receipts were issued in Rwanda from January 2016 until December 2018. We will aggregate the data to the quarterly level, following the aggregation of the VAT declaration data. We will then use as outcome the total value of sales for goods subject to VAT for which EBM receipts were issued.

This outcome will provide evidence for an important intermediate step, as any impact of the intervention on total sales subject to VAT reported by the firm in VAT declarations should take place through an impact on the total sales for goods subject to VAT for which EBM receipts were issued.

We provide more details on the construction of these outcomes in our pre-analysis plan.
Secondary Outcomes
Secondary Outcomes (end points)
As a secondary analysis, we will test whether the intervention had an impact on secondary outcomes following the same strategy as for the primary outcomes (see our pre-analysis plan):

Using the data from VAT declarations (quarterly-level data)
1. Total VAT liability reported in the VAT declaration
2. Total costs subject to VAT reported in the VAT declaration
3. Total sales not subject to VAT reported in the VAT declaration
4. Total sales reported in the VAT declaration

Using the data generated by EBM
1. Total number of transactions for which EBM receipts were issued that include at least one VAT-liable item
2. Total number of transactions for which EBM receipts were issued
3. Total sales for transactions for which EBM receipts were issued

The first set of secondary outcomes will be constructed using the same administrative data from the universe of VAT declarations as above. They will allow us to study the consequence of the intervention for the VAT tax base. In particular, we will study impact on:
- Total VAT liability, which depends on the difference between total sales subject to VAT and total costs subject to VAT: this will capture the tax implications of the intervention.
- Total costs subject to VAT: this will allow us to test whether the impact of any increase in sales subject to VAT (following the intervention) on the total liability were partly offset by an increase in the total costs subject to VAT as in Naritomi (forthcoming).
- Total sales not subject to VAT: this will allow us to test whether any increase in sales subject to VAT (following the intervention) partly came from reductions in sales not subject to VAT as firms may mis-classify sales in order to under-report sales subject to VAT.
- Total sales: this will allow us to test whether any increase in sales subject to VAT (following the intervention) led to an overall increase in reported sales, despite possible reductions in sales not subject to VAT.

The second set of secondary outcomes will be constructed using the same administrative data from the universe of transactions for which EBM receipts were issued as above. They will allow us to complement our findings based on the VAT declarations. In particular, we will study impact on:
- The total number of transactions for goods subject to VAT for which EBM receipts were issued: this will allow to test whether any increase in the total sales for goods subject to VAT for which EBM receipts were issued come from an "extensive-margin response," i.e., an increase in transactions involving goods subject to VAT for which an EBM receipt was issued.
- The total number of transactions for which EBM receipts were issued: this will allow us to test whether any extensive-margin response for transactions involving goods subject to VAT lead to an overall increase in transactions for which an EBM receipt was issued.
- The total sales for transactions for which EBM receipts were issued: this will allow us to test whether any increase in the total sales for goods subject to VAT for which EBM receipts were issued lead to an overall increase in total sales for which EBM receipts were used.

We provide again more details on the construction of these outcomes in our pre-analysis plan.

As a secondary analysis, we will also test a series of secondary hypotheses for our primary outcomes (see our pre-analysis plan):
- We will test whether the effect of the intervention on these primary outcomes is non-zero in at least one period, allowing for the possibility that firms' responses to the treatment may not be constant over time (and in particular they may be non-zero in only a subset of periods). In practice, we will define time periods over which treatment effects may vary at the quarterly level for all outcomes.
- We will test whether the frequency of EBM audits had a differential impact on these primary outcomes comparing the 91 treated firms assigned to receive an expected number of 4 visits per month vs.\ the 284 treated firms assigned to receive an expected number of 2 visits per month during vs.\ before the intervention.
- We will test whether the pre-intervention RRA visit and the letter had a differential impact on these primary outcomes comparing the 182 control firms that were visited by the RRA prior to the intervention, and thus received the RRA letter vs.\ the 74 control firms that were not visited by the RRA prior to the intervention, and thus did not receive the RRA letter, during vs.\ before the intervention.
- We will test whether the effect of the intervention (primary hypothesis) on the primary outcomes based on the data generated by mystery shopper visits depends on the value of the basket of goods purchased. In practice, for each firm, we constructed both a regular basket of goods and a more expensive basket of goods, which was purchased in a subset of the mystery shopper visits. We will thus test whether the effect of the intervention on whether an EBM receipt was issued and on the price of the basket of good differ for regular and more expensive baskets of goods.
- We will test whether the effect of the intervention (primary hypothesis) on these primary outcomes depends on the number of competitors subject to the same treatment. In practice, we created experimental variation in treatment saturation among firms in the same market.
- We will perform three types of subgroup analyses for the effect of the intervention (primary hypothesis) on these primary outcomes: (i) a subgroup analysis that describes heterogeneity in treatment effects by firm's sector of economic activity, defined as the two-digit ISIC code, coded at baseline; (ii) a subgroup analysis that allows the effect of audits to depend on the number of businesses in the same (geographic) sector in the same two-digit ISIC code (which incorporates the degree of competition from non-VAT-liable firms); and (iii) a subgroup analysis that uses the approach of Athey and Imbens (2016) and Wager and Athey (2018) to estimate subgroup heterogeneity and characterize optimal targeting.

Finally, we will test whether the intervention had an impact on the price paid for the baskets of goods purchased in mystery shopper visits when an EBM receipt is issued, using the data on the universe of transactions with EBM receipt for each firm and the line-item information recorded for each transaction (see our pre-analysis plan). The overall treatment effect on the price paid in mystery shopper visits can come from three effects: an effect on the price paid when no EBM receipt is issued, an effect on the price paid when an EBM receipt is issued, and a compliance effect, i.e., a change from the price paid when no EBM receipt is issued to the price paid when an EBM receipt is issued. The overall price effect and the compliance effect will be estimated in our primary analysis of the mystery shopper data. Combining estimates of the overall price effect, of the effect on the price paid when an EBM receipt is issued, and of the compliance effect, we will then be able to back out an estimate of the effect on the price paid when no EBM receipt is issued.
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
631 VAT-registered firms were identified for inclusion in the study. The threshold for VAT eligibility is an annual turnover in excess of RWF 20 million per year, or RWF 5 million per quarter. At the time of the study, all VAT-registered firms had been issued with EBMs. Starting from the administrative universe of VAT-registered firms, inclusion criteria were as follows:
1. Firms with VAT filings dating back to at least the first quarter of 2016, to avoid potentially high business failure rates among new, small businesses that might create high rates of attrition;
2. Single-location firms based in the city of Kigali, to avoid the risk that firms might shift business across locations in response to an intervention;
3. Firms in a relevant economic sector, which excludes wholesale, repair, and cleaning activities, as well as a small number of miscellaneous or unidentifiable types;
4. Firms with at least 5 percent of EBM-receipted transactions under RWF 6,000 in value, since this was the intended value of our own 'mystery shopper' measurement activities; and
5. Firms that could be located by a team of enumerators, guided by local ('zone'-level) RRA officers.
All firms meeting these criteria were included in the study.

The process of assigning the 631 firms comprising the study population to audit treatment arms was designed to generate the same treatment probabilities for all firms, while inducing patterns of correlation across the network of firms that would facilitate an analysis of spillovers. The compound lottery that this assignment mechanism produced assigned firms to 'treatment' with probability 0.6 and to 'control' with probability 0.4.

375 firms were randomly assigned to the audit treatment. These were then assigned (by simple randomization) to two groups, characterized by varying frequency of audits:
- 284 firms were assigned to receive an expected number of 2 visits per month;
- 91 firms were assigned to receive an expected number of 4 visits per month;
256 firms were randomly assigned to the control group. These were then assigned (again by simple randomization) to two groups:
- 182 received a pre-intervention visit by the RRA (see Intervention section);
- 74 firms did not receive any pre-intervention visit at all.

These assignments were in practice achieved through a two-stage saturation design that takes advantage of knowledge of firms' connections in product and geographic space and anticipates an Athey, Eckles, and Imbens (2018) approach to inference on a connected network of firms. This design provides an assignment strategy that will be well powered under that approach to testing interference hypotheses (see our pre-analysis plan).
Experimental Design Details
Randomization Method
Randomization was done in the office by a computer.
Randomization Unit
The unit of randomization in our analysis is the firm, but randomization was done in a two-stage saturation design that takes advantage of knowledge of firms' connections in product and geographic space and anticipates an Athey, Eckles, and Imbens (2018) approach to inference on a connected network of firms.

The treatment assignment mechanisms involved the following steps:
1. We partition the firms in our sample into those that, for testing interference hypotheses, will be focal firms \citep{AthEckImb17jasa}, and those that, for testing interference hypotheses, will be variant firms. This partition is chosen to maximize the number of focal nodes subject to the constraint that no two focal nodes are adjacent to one another in the network.
2. Focal nodes are assigned to treatments at random, with probabilities that match the proportions in the study design.
3. Focal nodes are assigned saturation rates that are either high (in which case variant nodes that are their direct neighbors are treated with probability 0.95) or low (in which case variant nodes that are their neighbors are treated with probability 0.19). This yields the same probability of treatment among variant nodes as among the focal nodes.
4. Variant nodes are assigned to either treatment or control with probabilities that are the average of the saturation rates of all focal nodes to which they are directly connected.
5. Finally, the high- and low-frequency treatment statuses are assigned to treated variant nodes by simple randomization, and the letter and no-letter statuses are assigned to control variant nodes by simple randomization.

The above process yields the same assignment probabilities for both focal and variant nodes, while increasing the variance of saturation rates surrounding focal nodes for the testing of interference hypotheses.
Was the treatment clustered?
Yes
Experiment Characteristics
Sample size: planned number of clusters
The treatment assignment mechanism employed does induce correlations in treatment status among 'adjacent' firms in the network defined by their geographic location and the products that they sell. We address this non-independence in randomization-inference-based tests of our primary and secondary hypotheses -- those that do not concern the question of interference -- by conducting randomization inference with respect to the set of draws feasible under our assignment mechanism (see our pre-analysis plan).

631 firms were identified for inclusion in the study (see inclusion criteria listed under 'Experimental Design' above.
Sample size: planned number of observations
631 firms.
Sample size (or number of clusters) by treatment arms
375 firms were randomly assigned to the audit treatment. These were then assigned (by simple randomization) to two groups, characterized by varying frequency of audits:
- 284 firms were assigned to receive an expected number of 2 visits per month;
- 91 firms were assigned to receive an expected number of 4 visits per month;
256 firms were randomly assigned to the control group. These were then assigned (again by simple randomization) to two groups:
- 182 received a pre-intervention visit by the RRA (see Intervention section);
- 74 firms did not receive any pre-intervention visit at all.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
We propose specific tests for impacts on primary outcomes in our Pre-Analysis Plan (PAP), which is filed along this registry. To asses the statistical power of the proposed specifications for primary outcomes, we use data from Control firms, and (repeatedly) bootstrap a sample of 631 firms out of these. (We did so in a protected `enclave'; none of the PIs had access to data from treatment firms during this process, until the Registry and PAP were filed. A github commit history documents this process.) For each such bootstrapped sample, we then simulate a simple random assignment of the binary treatment vector, assigning 375 firms to treatment and 256 firms to control. For regression-based tests of primary outcomes (as in mystery-shopper receipt rates, PAP equation 1, and mystery-shopper prices, PAP equation 2), we compute power based on the variance of the estimated treatment effect when we impose the null of no effect on these bootstrapped samples. This yields an minimum detectable effect of 0.02 for the probability of receiving a receipt---that is, we can detect an increase of 2 percentage points with 80 percent power. Similarly, our preferred specification for log prices yields a minimum detectable effect of 0.0475---that is, we can detect a price effect of 4.75 percent with 80 percent power. For outcomes where the KS test is the primary test of differences across treatment and control---as with declared VAT-liable sales and EBM receipts---our approach is slightly different. For a given bootstrapped sample (built out of control firms only), we randomly assign firms to Treatment and Control, and then we impose a treatment effect. We then calculate the KS test statistic and conduct RI on that test statistic, using the distribution of outcomes that results. This approach allows us to simulate power against specific violations of the sharp null. For example, we consider a violation that takes all firm-quarters with zero (EBM or VAT) declared sales and suppose that the treatment changes these to small, positive findings (equivalent to the 5th percentile of nonzero filings). Alternatively, we can consider additive treatment effects that increase only the amount of sales in those firm-quarters in which declared (EBM or VAT) sales are already strictly positive. Here, we report power against additive treatment effects on the intensive margin only, i.e., for simulations in which we impose treatment responses on those firms with non-zero filings only; the KS test appears even more sensitive to treatment effects that switch zero filers to non-zero amounts. We find that for total quarterly EBM sales, we are powered to detect an increase of RWF 0.207m at 78 percent, and an increase of RWF 0.414m in 99 percent of simulations. Note that the pre-treatment standard deviation of EBM-declared sales is RWF 8.16m (with a mean of RWF 5.41m), so these correspond to effect sizes of 0.025 and 0.05 standard deviations respectively. Again, we note that these effect sizes, in simulation, are imposed on non-zero filers only. For VAT-declared sales, the KS statistic is similarly powered in simulations. Pre-intervention filings of VAT-declared taxable sales have a mean of RWF 4.96m, and a standard deviation of 9.44m, conditional on having a non-zero filing in that quarter. For an additive treatment effect of 0.025 standard deviations imposed only on those with strictly positive filings, we observe simulated power of 60.5 percent. For an additive treatment effect of 0.05 standard deviations, again imposed only on those with positive filings, we observe a rejection rate in excess of 99 percent. As with EBM-reported sales, power for treatment effects that shift zero to non-zero filers is even higher.
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
Georgetown University
IRB Approval Date
2016-01-16
IRB Approval Number
2013-1398
Analysis Plan
Analysis Plan Documents
Pre-Analysis Plan

MD5: ff1f78020b71158da5795d5afef968b1

SHA1: 736e43c561c309eb013bb4455f1241635afb63e1

Uploaded At: May 01, 2019

Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
No
Is data collection complete?
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports and Papers
Preliminary Reports
Relevant Papers