NEW UPDATE: Completed trials may now upload and register supplementary documents (e.g. null results reports, populated pre-analysis plans, or post-trial results reports) in the Post Trial section under Reports, Papers, & Other Materials.
Testing the Effect of Tax Office and Taxpayer Interactions on the Integrity of Refund Claims
Initial registration date
February 24, 2016
June 01, 2017 2:00 AM EDT
Australian National University
Other Primary Investigator(s)
Australian National University
Additional Trial Information
To ensure that businesses pay the correct amount of tax, the Australian Taxation Office (ATO) may contact businesses if a transaction varies considerably from normal business activity. Businesses may receive a personalized letter from the ATO asking them to review the amounts on their business activity statement and to get back to the ATO if an error occurred. The aim of this trial is to test the effect of changes to letters on response rates of taxpayers, the timing of payments and the amount of payments of liabilities. The interventions are: (1) changing due dates, (2) addressing social norms, (3) using a different color and (4) informing taxpayers about tax deductible donations. The ATO sent out 3,000 letters on 12 November 2015. 600 letters were sent to each of the four randomly selected treatment groups; the remaining 600 letters were sent to the control group.
Biddle, Nicholas and Mathias Sinning. 2017. "Testing the Effect of Tax Office and Taxpayer Interactions on the Integrity of Refund Claims." AEA RCT Registry. June 01.
Four interventions will be studied: (1) changing the due date, (2) addressing social norms, (3) using a different color and (4) informing taxpayers about tax deductible donations. Each intervention will involve a single change in letter content or style. The treatment effect of each intervention will be measured by comparing average outcomes between treatment and control group. No interaction between treatments will be tested. Sample letters are included in Appendix A of the analysis plan.
Intervention Start Date
Intervention End Date
Primary Outcomes (end points)
Key outcome variables are:
- The total amount of payments made by the taxpayer after 12 November 2015
- The amount declared on the revised business activity statement
- The net amount of GST on the original business activity statement minus the amount declared on the revised business activity statement
- The revised net amount of all taxation items declared on the business activity statement
- The net amount of the tax for the period to be paid by the taxpayer or refunded by the Commissioner minus the revised net amount of all taxation items declared on the business activity statement
- A variable indicating whether the amount declared on the revised business activity statement was in favor of the ATO or the taxpayer - A variable indicating whether the revision of the net amount of all taxation items declared on the business activity statement was in favor of the ATO or the taxpayer
- The number of payments made by the taxpayer after the 12 November 2015
- The number of days until the first payment was received from the taxpayer after 12 November 2015
Primary Outcomes (explanation)
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
3000 letters were sent out on 12 November 2015. 600 letters were sent to each of the four randomly selected treatment groups; the remaining 600 letters were sent to the control group. Randomization was based on a random variable generator in Stata, using a random choice of the underlying seed.
Experimental Design Details
Randomization was based on a random variable generator in Stata, using a random choice of the underlying seed. A copy of the Stata code that has been written to randomly assign taxpayers to treatment and control groups is included in Appendix B of the analysis plan.
Was the treatment clustered?
Sample size: planned number of clusters
Sample size: planned number of observations
Sample size (or number of clusters) by treatment arms
3000 taxpayers; Treatment group 1: 600 taxpayers; Treatment group 2: 600 taxpayers; Treatment group 3: 600 taxpayers; Treatment group 4: 600 taxpayers; Control group: 600 taxpayers.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
We have chosen to assign about 600 clients to each treatment and control group. We make the following assumptions: standard deviation in each group=0.6, significance level=0.05, power=0.8, population mean of the control group=0, population mean of the treatment group=0.1. The power calculations were done in Stata. The estimated sample size to detect a minimum effect of 0.1 is 1134 (567 observations per group).
INSTITUTIONAL REVIEW BOARDS (IRBs)
Humanities and Social Sciences Delegated Ethics Review Committee of the Australian National University
IRB Approval Date
IRB Approval Number
Post Trial Information
Is the intervention completed?
Intervention Completion Date
February 24, 2016, 12:00 AM +00:00
Is data collection complete?
Data Collection Completion Date
February 24, 2016, 12:00 AM +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
Was attrition correlated with treatment status?
Final Sample Size: Total Number of Observations
Final Sample Size (or Number of Clusters) by Treatment Arms
treatment 1: 585 taxpayers, treatment 2: 589 taxpayers, treatment 3: 588 taxpayers, treatment 4: 587 taxpayers, control: 589 taxpayers
Reports, Papers & Other Materials
This paper presents the findings of two Randomized Controlled Trials (RCTs) that were conducted in collaboration with the Australian Taxation Office (ATO). The first trial tests the effect of changes to letters (timing, social norms, color, and provision of information about charitable donations) on response rates of businesses, the timing of payments and the amount of tax debt payments. The second trial consists of two parts. The first part aims to raise awareness of the relevance of tax debt payment by changing internal guidelines used by field auditors. The second part focuses on studying the effect of changing the phone script used by desk auditors to offer assistance with payment arrangements and simplifying a follow-up letter. The findings of the first trial indicate that none of the treatments had a significant effect on any of the outcome measures considered. In contrast, the results of the second trial indicate that changing the phone script of desk auditors and simplifying the follow-up letter reduced the proportion of default assessments raised by the ATO significantly, suggesting that businesses are responsive to certain types of nudges.
Biddle, Nicholas, Fels, Katja, and Mathias Sinning (2017): Behavioral Insights and Business Taxation: Evidence from Two Randomized Controlled Trials, Tax and Transfer Policy Institute (TTPI) Working Paper No. 2/2017, Australian National University.
REPORTS & OTHER MATERIALS