Experimental Design Details
Subjects are randomly assigned to one of three possible letter types (and to sub-treatments within some of these types). Samples of each letter type (and sub-types) are attached to this application. These letter types are identical in every respect except for the content of the table included in the middle of the first page:
- Present: the current median price of similar homes in the same ZIP Code.
- Future: the current median price of similar homes in the same ZIP Code, as well as a forecast for the price 1 year ahead. Within this letter type, subjects are randomized to one of three sub-treatments. Each sub-treatment corresponds to a different forecast model. Since the estimated future price change can differ across models, a given individual may see a different signals about the future depending on the model that was randomly chosen for them.
- Past: the current median price of similar homes in the same ZIP Code, in addition to information about past prices. Within this letter type, there are two sub-treatments: the past-1 sub-treatment includes information about the price 1 year ago; the past-2 sub-treatment includes information about the prices 1 and 2 years ago.
The main hypothesis of the study is that higher price expectations affect the transaction date and the transaction price (i.e., a seller who expects his or her house to appreciate more should be willing to wait a bit longer to sell the price for a higher price). We cannot manipulate house price expectations directly, but we do it indirectly through the information provision experiments:
- Within the future letter-type the hypothesis is that being randomly assigned to a higher forecast will increase the future home price expectations, resulting in houses that sell later and for a higher price. For a given individual, the three forecasting models result in forecasts of X%, Y% and Z%. The model that the individual is assigned to will determine the forecast that the individual receives. We will use the Instrumental-Variables regression method from Bottan and Perez-Truglia (2018) to exploit the exogenous variation created by the randomization of the forecast model.
- Within the past letter-type the hypothesis is that being shown higher prices changes in the past will increase the future home price expectations, resulting in houses that sell later and for a higher price. The idea is that giving sellers information about past price changes can influence expectations about the future because they extrapolate from past price changes to future price changes. There are two ways of analyzing this data. The most direct way is to use the randomization of sub-treatments. Consider an individual for whom the price changes were X% two years ago and Y% one year ago. If this individual is shown the past-1 letter, she will observe an average past change of Y% (i.e., over the last year); but if she is shown the past-2 letter, she will observe an average annual change of ((X+Y)/2)% (i.e., over the last two years). Thus, the randomization between these two sub-treatments creates exogenous variation in the information shown to the subject. There is an alternative way of analyzing the data, by using the comparison between the "current-type" and "past-type", exploiting heterogeneity in the past price changes in the ZIP code. However, we will only use this strategy if we do not have enough statistical power with the analysis of sub-treatments.
Note that these letter types (future and past) are testing the exact same hypothesis. The only reason why we included two ways of testing the same hypothesis is that individuals may trust one type of information more than the other (i.e., they may be more comfortable extrapolating from past prices than trusting "black-box" forecasts produced by researchers, or vice-versa). Thus, if the results are consistent between these two approaches, we will use an specification that pools them in order to gain statistical power.
The reason why we will send a large number of letters (60,000) is that we expect a majority of our letters to be ignored. For example, some letters may be lost in the mail, thrown away without being opened like most unsolicited mailing, recipients may not read the whole letter or they may read it but they may distrust the information. For all of these reasons, if we find the average effect to be close to zero, the next step would be to ask whether there is at least some groups of individuals who are influenced by the letters. We would use heterogeneity analysis to disentangle this.
The administrative records already contain some information about the subjects (e.g., whether the seller is living in the property or not, the number of days since the property was put on the market). Additionally, we will merge external data on other characteristics of the subjects, the properties and the local housing markets. For most individual characteristics, we will need to use geographic proxies (e.g., the average education at the census block level).
We would look at subgroups of the populations for which, ex-ante, we expect the letters may have a stronger influence. For example, it is possible that the seller's price expectations have a greater influence when the local market is a seller's market than when it is a buyer's market. Another example is that the letters may affect individuals who are selling their primary residencies less, because they may have less flexibility to delay the sale. We will also use the heterogeneity analysis in the auxiliary survey to guide the heterogeneity analysis in the field experiment. For example, if the auxiliary survey suggests that less educated individuals put more weight on the information we give them about the future, we will then test whether the letters had a stronger effect on the behavior of less educated sellers. When looking at these and other sources of heterogeneity, we'll be using standard methods for joint hypotheses testing. Additionally, we will also consider more modern machine learning methods.
Last, there is a secondary (less important) hypothesis: do sellers have systematic biases in their perceptions about past and/or future home prices? To shed light on this hypothesis, we will measure if there a difference in average behavior between individuals who receive the present, past and future letter types.
References
Bottan, Nicolas L. and Ricardo Perez-Truglia (2018), "Choosing Your Pond: Location Choices and Relative Income," NBER Working Paper No. 23615.