Experimental Design Details
The subject pool was randomly allocated to:
- Letter: were sent a letter from the researchers. 50,983 letters were sent.
- No-Letter: were not sent a letter from the researchers. 28,322 households did not receive a letter.
The difference between these two treatment arms will give us a sense of whether the "bundle" of information included in the letter has an effect on the decision to protest taxes. This is not the core objective of the intervention however. The main purpose of the experiment is to understand the causal mechanisms. To do so, within the letter group we randomized some of the information included in the letter. We cross-randomized two main treatment arms:
- Information about the average taxes paid by other households: the table in the first page of the letter always includes information on the proposed values and property taxes of the recipient's household estimated using data provided by the county. We randomized whether the table includes additional information about the average values of those two variables in the whole county or not. Our regression model exploits treatment heterogeneity. In other words, our main interest is NOT to compare the average behavior between those who received the additional information on county averages or not. We also want to exploit the rich variation in signals given to the subjects, as in the disclosure-randomization design from Bottan and Perez-Truglia (2020). The RHS variable of interest will be the interaction between the dummy indicating if the individual received information about county averages and a variable indicating the intensity of the information (i.e., the difference between the recipient's and the county average).
- Aid message: All letters include information on how to protest on the first page. All letter recipients receive a link to our website that contains information on how to protest, but we anticipate that not all letter recipients will access the website. We randomized whether on the second page the letters include additional information on the protest process. More specifically, this aid-message provides a concrete example that the recipient could use to fill a protest. For this message, we identified a nearby home that was similar to that of the recipient and was recently sold for less than the proposed value of the recipient's home. We wrote a suggested argument in a way that the recipient could use directly in the protest form.
We also cross-randomized a more minor treatment arm:
- Tax amounts vs. Tax rates: one challenge with the identification is that we do not know if the recipients cares about the tax amount that they pay relative to the county average, or whether they care more about the tax rate that they face relative to the corresponding county average. Since the tax amounts and tax rates are not perfectly correlated to each other, we can introduce both interactions simultaneously in the regression to figure out what recipients react to. Moreover, we cross cross-randomized another feature of the table to explore this further: we randomized whether the table includes a third row with the tax rates (i.e., the ratio between the property tax and the proposed value). This is not providing any new information (it is just the ratio of two numbers already shown in the letter). However, the addition of this row may affect the framing/salience of tax rates and thus affect the way in which people react to the information provided to them (i.e. making tax rates more important than tax amounts).
Some important details about the analysis:
- As explained in more detail in the section "Planned Number of Clusters" below, some subject will need to be discarded from the sample (e.g. if they sold the property before we sent the letters).
-Since some individuals may have protested before we started sending letters, we might be able to do an event-study analysis of the effects of our intervention.
- We have panel data including all the outcome variables for the previous year. For that reason, we'll be able to conduct falsification tests using the pre-treatment outcomes as dependent variables.
- We have rich data on the subjects, including the full history of their protests and proxies for whether they proposed value is above or below market. We'll incorporate those as control variables to reduce the variance of the error term and thus increase statistical power.
Last, we have rich data on household and property characteristics to conduct an heterogeneity analysis. Find below some of the key sources of heterogeneity that we anticipate exploring:
- We constructed two proxies for whether the property is over-valued or under-valued by the tax agency (one measure based on home value estimates from Redfin, and another measure based on the number of valid examples we could find for the aid message). Due to the hassle cost of protesting, it is likely that individuals only protest whenever they think they have a decent shot at a successful protest. For this reason, we'll explore this heterogeneity. This heterogeneity may be specially important for the aid message: i.e., if the example that we suggested does not provide a "strong" argument, the household should be less likely to use it in a protest.
-We know whether the households have hired a tax agency to represent them or not. With these tax agencies, it is often the case that the agency is the one deciding when to protest and how to protest and then charge the household a fraction of the tax savings. For that reason, households with tax agencies may be able to be totally inattentive to the whole protesting process, meaning that our letters will fall on deaf ears. For that reason, we'll separate the effects on households who had tax agents in the past vs. those who never had a tax agent.
- Two of our comparisons (letter vs. no letter and aid message vs. no aid message) examines how the provision of graduated levels of information concerning the property tax protest process affects the decision to protest. In this regard, the county decided at the last minute not to send notifications to households whose proposed values stayed the same or declined (about a quarter of the sample). It is plausible that the effects of our information on the protest process is stronger for those households, because they may have forgotten about the opportunity to protest otherwise. We will test this hypothesis. - A key heterogeneity for the letter vs. no letter comparison and the aid-message sub-treatment is between richer and poorer households. One fact that motivated our research design is that richer households are substantially more likely to file a protest. Our hypothesis is that this fact reflects differences in knowledge: richer households have a better understanding of the protesting process, or they may be able to hire a tax agent to act on their behalf. We hypothesize that our information treatments may reduce the gap in protests between richer and poorer households, by providing poorer households with resources that in normal times are exclusive to the richer households.
- We have data on household characteristics (e.g., gender, race, partisan affiliation), and we'll explore whether the effects of our treatments on protests are heterogeneous across different households. We can use this heterogeneity to explore if there is evidence of discrimination in each step of the protest process (i.e., initiating a protest, and having a successful outcome conditional on protesting). This is motivated by evidence from other contexts (e.g., salary negotiations, academic publishing, engagement in tournaments) that females and ethnic minorities may have unequal access.
- Since we were providing households with information, it is possible that some of that information spilled over from treated to control households. If anything, this would be a source of attenuation bias for our estimated effect of information. In any case, we may be able to test whether there was such spillover by looking at the effects of the letters on adjacent households who did not receive a letter.