Experimental Design
Our experiment consisted of a mailing campaign from Uruguay’s IRS with multiple treatment arms and sub-treatments. Rather than comparing firms that received a letter to firms that did not, all of our analyses are based on comparisons between firms that received letters with subtle variations in their content.
The letters consisted of a single sheet of paper with the name of the recipient in the header, the official letterhead of the IRS, and the scanned signature of the IRS General Director. These letters were folded, sealed in an envelope with the official letterhead of the IRS on the outside, and sent by certified mail.
The baseline letter contained some information about the goals and responsibilities of the tax authority, which the IRS routinely includes in its communications with firms.
In the audit-statistics letter type, we added a paragraph to the baseline letter, providing firms with information about the audit and penalty rates. Note that we communicated the probability that firms will be audited in at least one of the three following years, because IRS experts stated that this was the relevant probability for firms’ decision-making.
To complement the evidence from the audit-statistics sub-treatment, we implemented an alternative way of randomizing perceptions about audit probabilities using an audit-threat letter. We devised a treatment arm that randomly assigned firms to groups with different probabilities of being audited in the following year. We randomly assigned the firms in this treatment arm to two groups, one with a probability of being audited in the following year of 25% (X=25%) and another with a probability of being audited twice as large (X=50%).
We also devised a treatment arm to provide a benchmark for the effect of messages intended to increase tax compliance without directly mentioning audits. We designed a non-pecuniary message that was expected to be most effective at increasing compliance by the IRS staff and authorities: a message providing information about the cost of evasion in terms of the provision of public goods.
We designed a survey to be conducted with a sample of owners from our main subject pool, months after they received the letters. The IRS, with the support of the Inter-American Center of Tax Administrations and the United Nations, had previously administered a survey on the costs of tax compliance for small- and medium-sized businesses. We collaborated with the tax authority in the design and implementation of a new survey, which included a specific module tailored for our research design.
Our experiment was conducted in collaboration with the IRS of Uruguay. As of May 2015, there were 120,142 firms registered in the agency’s database. A subsample of 4,597 firms, preselected by the IRS, was put aside for the audit-threat sample, which we call the secondary experimental sample. Of the remaining firms, we followed a series of criteria to select our main experimental sample. We first excluded some firms by request of the IRS. For instance, we excluded firms subject to special regimes for VAT payments (very small or very large firms). We also kept in the experimental sample only those firms that made VAT payments in at least three different months in the previous 12-month period and those with a total value of at least USD 1,000.
We sent email invitations to participate in the online survey to 3,845 firms in May 2016, about nine months after our mailing experiment. Our purpose was to elicit the beliefs of firm owners. We did not include email addresses that were repeated more than three times in the full sample, as these most likely corresponded to accounting firms representing multiple small and medium enterprises. Even after applying this criteria, the IRS records could not ensure that the registered email address corresponded to the firms’ owners. We thus asked the survey respondent to self-identify as one of the following five types: owner, internal accountant, external accountant, manager, or other employee.