Demand for legal assistance

The survey experiment has two treatments and an active control group.

The first treatment (“information-only”) is a short, animated video that consists of two parts. First, it provides tenants with information on the effectiveness of attorneys in preventing evictions as measured by a recent RCT in Memphis. That RCT (AEARCTR-0010687) was conducted in partnership with the The Works Initiative (TWI), a nonprofit organization in Memphis, and randomized provision of attorneys to tenants who applied. We provide preliminary results from that study to respondents in this study by having respondents watch a short, animated video describing the study and its results. Second, the video also asks participants to take the survey seriously and reminds them that there are real stakes associated with their choices.

The second treatment (“trust”) additionally aims to build trust in TWI. It has the same two parts as the first treatment. It adds a third part, which has several components: (i) it informs tenants that TWI focuses on combating racial inequality, is led by community organizers, and serves primarily people of color; and (ii) it provides tenants with a positive testimonial from a tenant who was helped by TWI’s legal services program.

The (active) control group watches a short, animated video which only has the last part of each of the other videos (which asks tenants to take the survey seriously and reminds them that there are real stakes associated with their choices).

We have one “secondary” treatment. We experimentally relax budget constraints for some tenants by giving them a $500 bonus in the event that their WTA elicitation is implemented.

2023-07-21

2024-12-31

We aim to measure treatment effects of information and trust on demand for legal representation. Group 1 outcomes measure demand directly. Group 2 outcomes decompose the demand into mechanisms.

Group 1: direct measure of demand
Willingness to Accept (WTA) for cash versus an attorney from TWI

Group 2: demand mechanisms
Trust: Behavior in trust games against TWI attorneys
Beliefs: Predictions about TWI attorney effectiveness over the following three months

Demand explanation. We elicit WTAs using a Multiple Price List (MPL) technique. WTAs are incentivized using the strategy method. We elicit a WTA for a reference good, which we may additionally use as a control as in Dizon-Ross and Jayachandran (2022).

Trust explanation. Participants play Trust Games (Berg et al., 1995) against several “opponents.” In our Trust Game, the participant is endowed with $100. The participant can choose to share as much money as she wants with her opponent. When she shares money with the opponent, the money she shares is tripled. The opponent can then choose to share some amount of the (tripled) money back to the original participant. We have participants play with the following opponents: an eviction attorney from NPI, a personal injury attorney, a police officer, a doctor, another survey participant, and a landlord whose tenant applied for legal aid from TWI. The primary outcome is behavior against eviction attorneys. The behavior against other opponents is secondary (see below).

Beliefs explanation. We elicit both posterior and prior beliefs. Note that the treatment video presents backward-looking information, so it is relevant for predictions about the future, but participants need not report posterior beliefs that are exactly the information in the video. When we elicit prior beliefs, we ask respondents to predict eviction rates for tenants in the next three months who were randomly provided with an attorney and those who were not. When we elicit posterior beliefs, we ask them to predict effectiveness directly (i.e., the difference between evictions among those with and without lawyers, expressed as a percent of evictions among those without lawyers), and we remind them about what they predicted originally. We are primarily interested in effectiveness. We focus on the belief update (posteriors minus priors) as well as the levels of the posteriors, controlling for priors and distance from the information treatment.

A. Demand for legal assistance:
Share of participants who enter a lottery for free legal representation from TWI
Whether the participant refers anyone to TWI’s legal program
Number of referrals to TWI’s legal program
Whether participant intends to apply to TWI’s legal program, measured in the survey
Whether participant ultimately applies to TWI’s legal program, measured by linking the survey and the legal program

B. Behavior in trust games against: personal injury attorneys, police officers, doctors, another survey participant, and landlords whose tenants applied for legal assistance

C. Effect of budget constraints:
The effect of relaxing the budget constraint (a secondary treatment) on the reported WTA for lawyers versus cash.

D. We will analyze the following sources of heterogeneity as secondary:
- Heterogeneity based on existence of eviction notice. Some survey participants will have active eviction notices (self reported) or filings (which we can link from from court data). They may value attorneys differently and be more or less sensitive to treatments.
- Measurements of credit constraints. We measure credit constraints at baseline by asking standard questions about access to credit.
- Measurements of financial need. We will focus on: income and/or rent to income ratios, as well as measures of “bandwidth” to do financial planning, which we elicit with a free-response question about tenants’ worries and will analyze using natural-language processing. This heterogeneity could be one candidate explanation if we find trust and information do not affect demand: participants could have low bandwidth to seek out legal counsel even if they think it is effective and they trust it.
- Heterogeneity based on prior beliefs. We collect incentivized prior beliefs about the effectiveness of TWI attorneys over the next three months, before treatment. We use these beliefs to identify people whose beliefs are especially biased. That gives a secondary test for the role of information (see experimental details).

Secondary demand measures. We elicit several auxiliary measures that are intended to complement the willingness to accept measures. First, we ask people whether they would like to enter a free raffle for a TWI legal consultation. We will implement this raffle by providing them with a consultation with a TWI lawyer. Second, we ask whether the participant would like to refer anyone to the program. We focus on both whether the tenant refers anyone, as well as the number of people they refer. The referrals are incentivized, as we will actually have TWI contact them to apply for the program. Third, we ask whether the participant intends to apply to TWI’s program. Finally, we study whether the participant ultimately applies to the program, by linking the survey data to the TWI application data.

Trust Games. We collect behavior against the additional opponents for several reasons. First, collecting behavior against other opponents serves as a benchmark: we can compare the treatment effects of TGs against TWI lawyers to differences between (for instance) lawyers and landlords in the control group. Second, collecting TGs against doctors serves as an especially useful benchmark since lack of trust in the medical system is well-documented in this population (e.g., Alsan et al., 2019). Third, collecting a measure of trust against another type of lawyer measures whether the treatment affects generalized trust in the legal system versus trust in TWI in particular.

As noted in the treatments section, we have a secondary treatment which relaxes the budget constraint when eliciting the reported WTA. We test the effect of this treatment on the WTA cash versus lawyers. The purpose of this treatment is to test the extent to which tight budget constraints for the low-income participants in our study depress demand for legal services versus cash.

Intake. Memphis Inter-Faith Alliance (MIFA) is a nonprofit that provides rental and utility assistance to distressed tenants in Memphis, TN. They have agreed to link our survey on the ending screen of their online application for assistance. Tenants applying for MIFA aid will be offered the chance to participate in our survey: they will be told the survey will take approximately 15 minutes to complete and they will be compensated with a $15 gift card.

Design. Participants will first be asked a variety of background questions. The background questions will collect information about their housing situation, ask whether they have recently received an eviction notice, and elicit prior beliefs about the effect of an attorney in reducing eviction rates. Participants will then be randomly assigned one of three videos: a control video encouraging them to take the survey seriously and reminding them there are real stakes; an information-only treatment only; and a trust treatment. See above for details on the videos.

WTA. We now describe several details about the willingness to accept (WTA) elicitation. These are conducted as multiple price lists, incentivized using a Becker-Degroot-Marshak mechanism. Each tenant has two WTAs elicited: for an attorney who can represent them in the following year and for a reference good (an iPad). Tenants are randomized into reporting WTPs when we give them only $50 if they are selected (and then trade off more money versus the good), versus when we give them $500 if they are selected. The purpose of this elicitation is to test if materially relaxing budget constraints affects WTA. As a secondary outcome, we test the effect of relaxing the budget constraint on the reported WTA.

Purpose of treatments and intended tests. Our objective is to determine what drives demand for lawyers. The purpose of the information-only treatment is to shock beliefs about the efficacy of lawyers. The purpose of the trust treatment is to shock beliefs about the efficacy of lawyers and trust in the legal system. The difference between the information-only treatment and the control measures the effect of information. The difference in treatment effects between the trust and information-only treatment measures the effect of “trust.” Our focus is first on the WTA outcome, which we interpret as demand for lawyers. These are Intent-To-Treat estimates. We may also add demographic controls, using a principled control-selection procedure (e.g., Belloni et al., 2014).

In addition to establishing the ITT effects on demand across treatments, we directly capture the role of information and trust via treatment effects on belief updates and trust games, respectively. Finally, we further capture the role of information by eliciting prior and posterior beliefs, and measuring the effect of the information treatment relative to priors. For instance, we can study effects of information on demand among people with above-/below-median belief biases (or the instrumental-variables version of this specification). Thus, the experiment embeds multiple ways of testing for whether information and/or trust drive demand: via (1) the differential effects by treatment condition, and (2) the effects on outcomes intended to measure each channel directly.

We will primarily study the intent-to-treat of the two treatments versus control and each other. Additionally, for power, we may pool both treatments and compare them to control. Secondary tests will use the information-only treatments as instruments for beliefs.

Survey changes. We anticipate that the lawyers RCT will conclude on December 31. Therefore, on October 1, we will change the belief elicitation questions to be about the last three months of the program altogether (rather than the next three months going forward). In the fall, we will also need to take down the referrals outcome, as the program will no longer accept applicants.

References

Alsan, Marcella, Owen Garrick, and Grant Graziani. "Does diversity matter for health? Experimental evidence from Oakland." American Economic Review 109, no. 12 (2019): 4071-4111.

Belloni, Alexandre, Victor Chernozhukov, and Christian Hansen. "Inference on treatment effects after selection among high-dimensional controls." Review of Economic Studies 81, no. 2 (2014): 608-650.

Berg, Joyce, John Dickhaut, and Kevin McCabe. "Trust, reciprocity, and social history." Games and economic behavior 10, no. 1 (1995): 122-142.

Dizon-Ross, Rebecca, and Seema Jayachandran. "Improving Willingness-to-Pay Elicitation by Including a Benchmark Good." In AEA Papers and Proceedings, vol. 112, pp. 551-555. 2022.

Not available

Randomization done in Qualtrics

Individual

No

2,500 participants, excluding those who fail a general attention check.

2,500 participants, excluding those who fail a general attention check. There is a chance we will need to conclude the survey before we have 2,500 participants, because the partner may ask us to take the survey down before then.

About 833 participants in each of the two treatment arms + control. (Excluding those who fail a general attention check. Randomization probabilities equal across all arms.)

We consider power to detect a 0.15 standard deviation (SD) effect of information vs. control and a 0.15 SD effect of trust vs. information. The power to detect each is the same, since sample sizes are even across groups. A candidate outcome that we might standardize in this way could be Willingness to Accept. Simulations show that we are powered to detect a treatment effect of 0.15 SDs at power = 0.8 if we recruit at least 2200 participants. We register 2,500 participants to be safe.