x

Please fill out this short user survey of only 3 questions in order to help us improve the site. We appreciate your feedback!
Tax Protesters: Evidence from a Large-Scale Field Experiment
Last registered on June 16, 2020

Pre-Trial

Trial Information
General Information
Title
Tax Protesters: Evidence from a Large-Scale Field Experiment
RCT ID
AEARCTR-0005992
Initial registration date
June 15, 2020
Last updated
June 16, 2020 11:42 AM EDT
Location(s)
Primary Investigator
Affiliation
UCLA
Other Primary Investigator(s)
PI Affiliation
University of Texas at Dallas
PI Affiliation
University of Texas at Dallas
Additional Trial Information
Status
On going
Start date
2020-05-20
End date
2020-12-31
Secondary IDs
Abstract
We designed and conducted a field experiment to study people's decision to protest their
property taxes. We examine the role of information frictions and fairness concerns.
External Link(s)
Registration Citation
Citation
Nathan, Bradley, Ricardo Perez-Truglia and Alejandro Zentner. 2020. "Tax Protesters: Evidence from a Large-Scale Field Experiment." AEA RCT Registry. June 16. https://doi.org/10.1257/rct.5992-1.0.
Experimental Details
Interventions
Intervention(s)
Our subject pool is comprised of around 80,000 households who were about to face the opportunity to protest the tax assessments of their homes. We sent letters to around 51,000 of those households, and randomized the information included in the letters. We want to measure how the information included in the letters affected the subsequent decision to protest their taxes
Intervention Start Date
2020-05-20
Intervention End Date
2020-06-15
Primary Outcomes
Primary Outcomes (end points)
The main outcome is a dummy variable indicating whether the household protested or not by the deadline (June 15th, 2020).
Primary Outcomes (explanation)
We will also be able to study this outcome separately by type of protest: i.e., whether the owner protested directly or whether the owner used a tax agent to protest on their behalf. We expect that some of our information treatments may have a bigger effect on direct protests, specially among less affluent households.
Secondary Outcomes
Secondary Outcomes (end points)
The secondary outcome is whether they protest was successful or not. We can construct this outcome as a dummy variable or as an intensity variable (e.g., the %-reduction in the appraised market value as a result of the protest). We may also be able to observe other secondary outcomes described below, depending on data availability. We will also be conducting a supplemental survey with a separate sample (Amazon Mechanical Turk) to provide some evidence about the underlying causal mechanisms.
Secondary Outcomes (explanation)
We may also be able to measure other secondary outcomes. It is possible we access data on the opinion of value included in the protest form, which would give us a measure of the intensity of the protest (e.g., whether the taxpayer is requesting a 1% or a 20% reduction in property taxes). We could also use data on longer-term outcomes observed one year after our intervention: i. whether the subject pays the 2020 property taxes on time; ii. whether the subject protests their 2021 property taxes.
Last, we included a link to a survey in the letter (full survey instrument attached to this registration). Based on evidence from other studies (e.g., Bottan and Perez-Truglia, 2020) we expect not to be able to use the survey outcomes due to a very low and selective response rates (e.g., people who plan to protest and found our letter useful may be substantially more likely to participate in the survey). Due to those limitations, we will conduct a supplemental survey experiment with an auxiliary online sample (Amazon Mechanical Turk) to complement the evidence from the field experiment. The survey instrument for the supplemental survey is attached to this registration too. The supplemental survey tries to replicate the information-provision experiment conducted in the field experiment. The ideal is to measure how the beliefs about the tax rate paid by neighbors affect the perceived fairness and the willingness to protest property taxes. We are planning to collect around 2,000 survey responses from this platform. We will collect this data at around the same dates as the intervention from the field experiment.
Experimental Design
Experimental Design
Our subject pool is comprised of around 80,000 households who were about to face the opportunity to protest the tax assessments of their homes. We sent letters to 50,983 of those households, and randomized the information included in the letters. A sample of the envelope and the letter are attached to this pre-registration. We want to measure how the information included in the letters affected the subsequent decision to protest their taxes.
Experimental Design Details
The subject pool was randomly allocated to:
- Letter: were sent a letter from the researchers. 50,983 letters were sent.
- No-Letter: were not sent a letter from the researchers. 28,322 households did not receive a letter.
The difference between these two treatment arms will give us a sense of whether the "bundle" of information included in the letter has an effect on the decision to protest taxes. This is not the core objective of the intervention however. The main purpose of the experiment is to understand the causal mechanisms. To do so, within the letter group we randomized some of the information included in the letter. We cross-randomized two main treatment arms:
- Information about the average taxes paid by other households: the table in the first page of the letter always includes information on the proposed values and property taxes of the recipient's household estimated using data provided by the county. We randomized whether the table includes additional information about the average values of those two variables in the whole county or not. Our regression model exploits treatment heterogeneity. In other words, our main interest is NOT to compare the average behavior between those who received the additional information on county averages or not. We also want to exploit the rich variation in signals given to the subjects, as in the disclosure-randomization design from Bottan and Perez-Truglia (2020). The RHS variable of interest will be the interaction between the dummy indicating if the individual received information about county averages and a variable indicating the intensity of the information (i.e., the difference between the recipient's and the county average).
- Aid message: All letters include information on how to protest on the first page. All letter recipients receive a link to our website that contains information on how to protest, but we anticipate that not all letter recipients will access the website. We randomized whether on the second page the letters include additional information on the protest process. More specifically, this aid-message provides a concrete example that the recipient could use to fill a protest. For this message, we identified a nearby home that was similar to that of the recipient and was recently sold for less than the proposed value of the recipient's home. We wrote a suggested argument in a way that the recipient could use directly in the protest form.
We also cross-randomized a more minor treatment arm:
- Tax amounts vs. Tax rates: one challenge with the identification is that we do not know if the recipients cares about the tax amount that they pay relative to the county average, or whether they care more about the tax rate that they face relative to the corresponding county average. Since the tax amounts and tax rates are not perfectly correlated to each other, we can introduce both interactions simultaneously in the regression to figure out what recipients react to. Moreover, we cross cross-randomized another feature of the table to explore this further: we randomized whether the table includes a third row with the tax rates (i.e., the ratio between the property tax and the proposed value). This is not providing any new information (it is just the ratio of two numbers already shown in the letter). However, the addition of this row may affect the framing/salience of tax rates and thus affect the way in which people react to the information provided to them (i.e. making tax rates more important than tax amounts).

Some important details about the analysis:
- As explained in more detail in the section "Planned Number of Clusters" below, some subject will need to be discarded from the sample (e.g. if they sold the property before we sent the letters).
-Since some individuals may have protested before we started sending letters, we might be able to do an event-study analysis of the effects of our intervention.
- We have panel data including all the outcome variables for the previous year. For that reason, we'll be able to conduct falsification tests using the pre-treatment outcomes as dependent variables.
- We have rich data on the subjects, including the full history of their protests and proxies for whether they proposed value is above or below market. We'll incorporate those as control variables to reduce the variance of the error term and thus increase statistical power.

Last, we have rich data on household and property characteristics to conduct an heterogeneity analysis. Find below some of the key sources of heterogeneity that we anticipate exploring:
- We constructed two proxies for whether the property is over-valued or under-valued by the tax agency (one measure based on home value estimates from Redfin, and another measure based on the number of valid examples we could find for the aid message). Due to the hassle cost of protesting, it is likely that individuals only protest whenever they think they have a decent shot at a successful protest. For this reason, we'll explore this heterogeneity. This heterogeneity may be specially important for the aid message: i.e., if the example that we suggested does not provide a "strong" argument, the household should be less likely to use it in a protest.
-We know whether the households have hired a tax agency to represent them or not. With these tax agencies, it is often the case that the agency is the one deciding when to protest and how to protest and then charge the household a fraction of the tax savings. For that reason, households with tax agencies may be able to be totally inattentive to the whole protesting process, meaning that our letters will fall on deaf ears. For that reason, we'll separate the effects on households who had tax agents in the past vs. those who never had a tax agent.
- Two of our comparisons (letter vs. no letter and aid message vs. no aid message) examines how the provision of graduated levels of information concerning the property tax protest process affects the decision to protest. In this regard, the county decided at the last minute not to send notifications to households whose proposed values stayed the same or declined (about a quarter of the sample). It is plausible that the effects of our information on the protest process is stronger for those households, because they may have forgotten about the opportunity to protest otherwise. We will test this hypothesis. - A key heterogeneity for the letter vs. no letter comparison and the aid-message sub-treatment is between richer and poorer households. One fact that motivated our research design is that richer households are substantially more likely to file a protest. Our hypothesis is that this fact reflects differences in knowledge: richer households have a better understanding of the protesting process, or they may be able to hire a tax agent to act on their behalf. We hypothesize that our information treatments may reduce the gap in protests between richer and poorer households, by providing poorer households with resources that in normal times are exclusive to the richer households.
- We have data on household characteristics (e.g., gender, race, partisan affiliation), and we'll explore whether the effects of our treatments on protests are heterogeneous across different households. We can use this heterogeneity to explore if there is evidence of discrimination in each step of the protest process (i.e., initiating a protest, and having a successful outcome conditional on protesting). This is motivated by evidence from other contexts (e.g., salary negotiations, academic publishing, engagement in tournaments) that females and ethnic minorities may have unequal access.
- Since we were providing households with information, it is possible that some of that information spilled over from treated to control households. If anything, this would be a source of attenuation bias for our estimated effect of information. In any case, we may be able to test whether there was such spillover by looking at the effects of the letters on adjacent households who did not receive a letter.
Randomization Method
Randomization done in office by a computer.
Randomization Unit
Household
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
79,322 households
These households are a subsample of the universe of all homeowners in Dallas County, Texas. We arrived to this subsample by applying a number of filters (e.g., focusing on single-family homes, excluding business properties). We would like to clarify that some households will need to be dropped from the subject pool due to reasons that we do not observe ex-ante (because the data is not posted in real-time) but we can measure ex-post such as:
- Some letters will not be deliverable (e.g., vacant addresses, letters returned to sender).
- Some households may have sold their properties before we shipped the letters. Since only the current owner can protest and pay property taxes, those households will have to be dropped from the sample.
- Some households may have already protested their taxes before we shipped the letters
Sample size: planned number of observations
79,322 households
Sample size (or number of clusters) by treatment arms
- We randomized 50,983 to receive a letter and 28,322 not to receive a letter.
- We cross-randomized 2/3's to see the information about averages.
- We cross-randomized 1/2 to see the additional row with tax rates.
- We cross-randomized 1/2 to see the additional message with aid for filing the protest
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Since there are multiple treatment arms, there are multiple Minimum Detectable Effect Sizes. We provide the effect of the aid message as illustration. Power calculations based on protest data from the previous year (i.e. "placebo" regressions using the previous year's protest outcome as dependent variable) suggest a minimum detectable effect size of 0.52 percentage points. We have more power for the letter vs. no-letter comparison. And we have a bit less power for the effects of the information on county averages included in the letter.
Supporting Documents and Materials
Documents
Document Name
Sample Envelope and Sample Letter
Document Type
other
Document Description
File
Sample Envelope and Sample Letter

MD5: 2e4e5aa5b5c1c8de0412bd11ca9ba5b2

SHA1: bdb5ed1cfbb53d8daf781be1b0e827029ecf3c4d

Uploaded At: June 15, 2020

Document Name
Survey Instrument (Supplemental Survey)
Document Type
survey_instrument
Document Description
This is the supplemental survey to be conducted on Amazon Mechanical Turk.
File
Survey Instrument (Supplemental Survey)

MD5: f3708235e3971e37b0d8238fcc95f09c

SHA1: e406052379a6af8eb660425286f553b2754de8a0

Uploaded At: June 10, 2020

Document Name
Survey Instrument (Letter Survey)
Document Type
survey_instrument
Document Description
This is what respondents saw when they went to the URL included in the letter.
File
Survey Instrument (Letter Survey)

MD5: 70bdcbcfb51733cfde6f4f6a4ffabe2b

SHA1: c60da4f603d549c5a7d45056d8fab339ffa80bc4

Uploaded At: June 10, 2020

IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
Office of Research Integrity and Outreach, University of Texas at Dallas
IRB Approval Date
2020-05-04
IRB Approval Number
20MR0126
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
No
Is data collection complete?
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports, Papers & Other Materials
Relevant Paper(s)
REPORTS & OTHER MATERIALS