Experimental Design Details
Setting and sample. We are working with the City of Memphis and Shelby County, Tennessee, on an evaluation of their Emergency Rental and Utilities Assistance Program (ERAP). As a part of this program, tenants and landlords apply for emergency assistance to resolve rent delinquencies. We solicit participation in an online survey to approximately 4,000 landlords, property owners, and property managers (we call them “landlords” for short), using email and (perhaps) text messages. We offer to compensate landlords for their participation.
Our objective is to have as many landlords participate as possible. We cannot control the number who enroll, and we have a finite population from which we can solicit participants. Based on piloting, we expect perhaps 200-400 landlords to complete the survey experiment.
Overview of survey experiment. The goal of the experiment is to understand landlords’ behavior in the ERAP. In particular, many landlords do not participate in the ERAP, despite what one would think are attractive financial offers to repay tenants’ rent. In this survey, we first measure landlords’ behavior in a modified dictator game. We also measure landlords’ prior beliefs about the eviction process. We then provide a treatment group with truthful information about the percent of tenants served with monetary judgments who fully repay the judgment. We study the effect on beliefs and a number of behaviors. However, given our expected sample size, we are not certain to be powered to study the treatment effect on behaviors. As a result, our primary outcomes are the behavior in the dictator game and the prior beliefs.
Dictator game. We conduct two modified dictator games with each landlord participant. We randomize the order of the games. In each game, we elicit the landlord participant’s choice over a bundle of ($10 self, $10 opponent) and ($X self, $0 opponent). We elicit (bounds on) the value of $X at which landlords are indifferent over the bundles. In one game, the opponent is a tenant. In the other game, the opponent is a randomly selected landlord. When the opponent is a tenant, 2/3 of the landlords will see a particular tenant of theirs named. The other 1/3 will be instructed that the opponent is a tenant “chosen at random, among tenants whose landlords participate.” When the opponent is a landlord, the landlord is told their opponent is a random landlord who participated in the study.
We implement these choices for 10 total landlords in the study (including piloting), 5 from each game.
The purpose of the game is to see whether landlords’ altruistic preferences vary when the opponent is a tenant vs. a landlord. In particular, one hypothesis is that landlords are vengeful against their tenants, which is one reason that landlords have been reluctant to participate in the ERAP. This game gives an experimental measure of whether that is the case.
Prior beliefs. We measure a number of landlords’ beliefs about the eviction process. Beliefs about objective facts are incentivized using a scoring rule. We also measure beliefs about how the eviction process would work for their particular tenant. We do not incentivize these beliefs.
We elicit prior beliefs about:
1. The share of tenants who fully repay monetary evictions judgments
2. The number of monetary evictions judgments rendered between April 1 and June 30, 2021 (of roughly 3,500 filed)
3. The percent chance that their tenant would repay a monetary judgment, if they were granted one
4. The number of days to receive a judgment about their tenant.
We elicit uncertainty about items (1) and (3).
Intervention. 50% of participants receive information about the number of tenants out of 100 who fully repaid their claims.
Posterior beliefs. We ask landlords whether they wish to update their beliefs about: items (3) and (4) above.
Revealed outcomes. We then ask landlords a number of questions about how they would like to engage with the ERA program. See discussion of secondary outcomes below. The objective of these revealed outcomes is to study the relationship between beliefs about evictions and their engagement with the program.
Simple fairness outcome. We also include a simple fairness outcome. We inform landlords that there will be another lottery for a gift card. There is a 1% chance of winning the gift card, no matter how many people are enrolled. We run a separate lottery for tenants and for landlords. We ask whether the participant wishes to enroll their particular tenant in the tenant lottery, and their particular landlord in the landlord lottery.
Behavior in the modified dictator game with landlords (choice of bundle)
Behavior in the modified dictator game with tenants (choice of bundle)
Choice of whether to enter tenant or landlord in lottery for extra gift card
Accuracy of objective prior beliefs
Comparison between objective prior beliefs and beliefs about the tenant
Primary outcomes (explanation).
We will compare the indifference point $X when the opponent is a landlord to the value when the opponent is a tenant. Relatedly, we will also study the share of landlords whose indifference point is $20 (and/or the share whose indifference point is $10 or less) when the opponent is a landlord or a tenant.
We have data on the correct value for prior beliefs about objective facts. We will study how biased are landlords’ beliefs.
We will compare landlords’ beliefs about their own tenant to their beliefs about objective facts. For instance, we will generate measures of the distance between the objective fact and beliefs about the particular tenant.
We study whether receiving information affects beliefs about (1) the repayment probability for their particular tenant, and (2) the length of time it would take to receive a judgment for their particular tenant.
We include (2) as a placebo: we do not expect the treatment to affect beliefs about this ancillary outcome.
We study whether the information treatment affects the following program outcomes:
Index of: Choice of whether to receive offer for back rent (if offered to the landlord); choice of whether to receive a packet of informational materials about the program; choice of whether to refer tenants to the program; choice of whether to decline to sign any legal agreement; choice of whether to notify their tenant about opportunities to apply for additional funds. We will also study the individual outcomes.
The number of tenants referred if they refer
Haircut rate for future rent
We may study the effect of the information treatment on these outcomes directly, or instrument for beliefs using these outcomes. The purpose of the placebo above is to assist our goal of extracting the effect of beliefs about the repayment probability on behaviors with respect to the program. A null effect on the placebo will help confirm that the treatment did not affect other beliefs, and therefore render the exclusion restriction more likely to hold.
Heterogeneity in modified dictator game by landlord characteristics, including: gender, race (and whether tenant has the same race), years of experience, size of landlord (i.e., number of units owned), years of experience renting to current tenant
Heterogeneity in tenant characteristics, including: gender, race, back rent tenant owes/whether tenant has been paid by ERAP
Heterogeneity by uncertainty. We will study whether people update more if they are uncertain about the percent chance their own tenant would repay.
Behavior in dictator game, depending on whether the tenant is listed as a random tenant or the landlord’s particular tenant.
Links to administrative data. We intend to link landlords' behavior in the survey experiment to their decision of whether to accept the City/County's offer to repay back rent, using the government's administrative data. We may also construct related administrative measures like the length of time it took to accept the offer. The purpose of these administrative links would be to analyze whether landlords' beliefs and altruism (measured in the survey) can explain their behavior with respect to their tenants.
Secondary outcomes (explanation). These outcomes are secondary due to concerns about power, based on our expectations about the number of landlords who will participate (from piloting). They are not secondary because they are necessarily less important.
The “haircut rate” for future rent is a hypothetical choice about a future rent offer. We explain that the City/County might offer to subsidize the tenant’s future rent, but at a discount. We elicit the maximum discount (= haircut rate) that the landlord would accept for the future rent.
Instrumental variables design. We may also use the information treatment to obtain the elasticity of behaviors with respect to belief updates. We can use the information treatment to instrument for the belief update (about one’s own tenant’s chances of repaying the rent, or about the fact in general).
Attention check. We implement a simple attention check where we ask people to report that their favorite color is teal. Piloting suggests that this attention check would exclude a nontrivial amount of the (already limited) sample. We will implement the attention check and show results on a sample that passes it, but depending on power, we may still include people who fail the check.
Pilot. We collected data from about 20 landlords in a pilot. We may include the pilot data in the final sample to assist with power, but we will also show results excluding the pilot sample.
Reminders. We send reminders to induce landlord participation. On 9/16 or later, we will send a reminder only to landlords who partially completed the survey but then attrited. Of this sample, we may exclude landlords who had been exposed to treatment but not yet seen outcomes before attriting (since the effect may fade).