The Effects of Legal Assistance on Evictions

There are two treatments and a control group.

The primary treatment group provides full legal assistance to tenants facing eviction. In particular, tenants who are randomized into this treatment will receive an offer for full legal representation in General Session court. The legal team constitutes a mix of experienced tenant lawyers and lawyers whom NPI recruited and trained to conduct “low-bono” legal work.

The secondary intervention provides light-touch (one-time) legal counseling and social work to tenants facing eviction. In particular, tenants in this group receive a one-time phone call from either law students or a social worker. The law students and social workers are trained by NPI to provide coaching about how to handle an eviction. We registered the intervention as “secondary” starting in v1.0, because faced uncertainty about whether NPI will hire enough social workers or law students to reach a large number of tenants. As of v1.3, it became clear that the nonprofit’s social workers will not reach a large share of the control group, and the ones they do reach may be called after the eviction proceedings. We continue to randomize social workers as a part of our impact evaluation/assistance we provide NPI. We may include this treatment in the paper as a secondary treatment. But we do not currently plan for it to be a focus.

The control group does not receive assistance. When they are informed via email that they were not selected for assistance, the email notification provides select other resources to which they can apply in Memphis, as well as some basic information about their rights in eviction court. This information is also provided to the attorneys and counselors as part of their training. Prior to v1.2, we had labeled this group as a “pure control,” because we view this information as light-touch and unlikely to make a large difference in tenant outcomes. Nothing about the control group has changed — just how we label it.

Given that there are two treatments, we discuss our intended comparisons below in the experimental design section.

2022-03-03

2023-12-31

There are three categories of outcomes where we will measure the treatment effect of provision of lawyers. Group 4 are outcomes that are collected prior to treatment where the raw means are still informative but cannot be affected by treatment (since collected before).

Group 1: Formal legal outcomes (registered in v1.0 after data collection began). The primary outcomes under this group are:
The share of eviction cases that result in judgments.
The share of eviction cases that were explicitly withdrawn and/or “nonsuited.”
The monetary amount owed in the eviction judgment.
Total time elapsed between filing and case resolution.

Group 2: Informal legal outcomes (preliminary in v1.0, preregistered in v1.1 before endline survey collection began). The primary outcomes under this group are:
Whether the participant moved (from the location where they had the eviction filing that triggered their application for legal assistance)
Whether the tenant paid a lower value of overdue rents than what the landlord initially asked for
The value of back rents that the tenant paid

Group 3: Financial outcomes (preliminary through v1.2).
Credit report outcomes, including: (i) credit score; (ii) collections balance; (iii) any positive balance on auto loan or lease; (iv) any open source of revolving credit. We choose these outcomes to be comparable to the main financial health outcomes in Collinson et al. (2022).

Group 4: Other comparisons or outcomes (preliminary in v1.0 and v1.1, preregistered in v1.2).
Willingness to pay for a lawyer.

[We elicit WTPs using a MPL technique. WTPs are incentivized using the strategy method.

In v1.2, we said we would residualize the WTPs by the WTP for the reference good; we realized that this practice may purge income effects, which are relevant in our setting. As a result, as of v1.3, we will use the “raw” WTPs but also show the effects of controlling for the WTP for the reference good.]

Group 1:
The share of cases where continuances are filed.
The share of cases where an eviction writ is filed.
The share of cases that went to trial.
The number of motions filed.
Total time spent in court.
The number of instances in which a judge interacts with the case [if available].
Whether the tenant had legal assistance (a “first stage”).
Payment by the local Memphis Shelby County Emergency Rental and Utilities Assistance Program.
Payment by another emergency assistance program, including MIFA, THDA, CSA, or others [added in v1.2]

Group 2:
Whether the participant was employed and their wages. Both are captured in the surveys (described below).
If ZIP location changed on the credit report. [Note that this is categorized as an informal legal outcome, even though only visible on credit reports, because it would imply that the tenant moved. It is therefore preliminary through v1.1, unlike other group 2 outcomes.]
Whether the tenant has agreed to pay back less than what the landlord asked for
The amount of back rents that the tenant agreed to pay back
Whether a payment plan or settlement was formed
If the participant stayed in a homeless shelter
Whether the landlord agreed to repairs
Whether the tenant appeared in court

Group 3:
Employment
Earnings
[Note that these are categorized as economic outcomes but are collected in the survey. They are therefore preregistered in v1.1 before data collection began.]

Group 4:
Willingness to pay for a social worker [moved from primary to secondary in v1.3]
Treatment effect of “budget unconstrained” elicitation (described below) on willingness to pay for lawyers.
Treatment effect of “budget unconstrained” elicitation (described below) on willingness to pay for counselors.
Treatment effect of “budget unconstrained” elicitation (described below) on willingness to pay for reference good.
Willingness to pay for a delay in court (hypothetical)
Trust in lawyers relative to other professions, measured via a Trust Game
Trust in general (Global Preference Survey elicitation)
The self-reported quality of the relationship between the tenant and landlord
Valence of free-response questions about the relationship between the tenant and the landlord
Incentivized beliefs about the treatment effects of lawyers [moved from primary to secondary in v1.3]
Levels of beliefs about lawyers (i.e., the “raw” beliefs about judgment rates with and without lawyers)

Heterogeneity:
1. Interaction with the local Emergency Rental and Utilities Assistance Program (ERAP). Prior to September 1, tenants could apply for the local ERAP, a government program that pays back rents of tenants facing eviction. The primary treatment was partially integrated with ERAP such that lawyers could use ERAP funds to pay back rents to landlords as a part of negotiations. This ended in September 2022. While we will pool the samples for power in our main analysis, an important source of heterogeneity will be the difference in treatment effects on primary outcomes, before and after the ERAP program closed. If we are sufficiently powered for the analysis, this could be a primary focus of the paper, but we have uncertainty about the number of treated units after ERAP closes.

2. Heterogeneity based on time until court date. A key reason for the program’s success could be that it gives lawyers time to negotiate with landlords before court. We will study if the program is more successful if lawyers have more time to negotiate. [registered in v1.3]

Other sources of treatment-effect heterogeneity are below. We invite applicants to participate in surveys at baseline. We intend to study whether the effect of treatment differs based on cuts of these characteristics below (preliminary through v1.1, preregistered in v1.2). We are unsure we will be powered to detect heterogeneity along these dimensions, which is why we register them as secondary.

3. Relationship with landlord. We ask tenants about their relationship with their landlord and examine treatment effect heterogeneity by whether the relationship is strong or weak.

4. Willingness to Pay for primary treatment (full legal services), secondary treatment (social workers/law students), or delays. We conduct willingness to pay exercises where we ask tenants (prior to randomization) their valuations between the indicated outcome and cash. The purpose of this exercise is to: (a) obtain an estimate of tenants’ valuations, which enters the welfare effects of the policy, and (b) determine whether tenants with high valuations have larger effects from the treatment. We also obtain the WTP for a reference good (9th generation iPad), which is a useful benchmark (Dizon-Ross and Jayachandran, 2022).

5. Measurements of credit constraints. We measure credit constraints at baseline by asking standard questions about access to credit.

6. Measurements of financial need, e.g. income and/or rent to income ratios.

7. Trust in lawyers/trust in general. We collect measures of trust in lawyers, via a Trust Game, and trust in general at baseline.

8. Baseline beliefs about the effects of lawyers. We collect these measures at baseline. We can test whether lawyers are more or less effective for participants who are optimistic about their effects. [registered in v1.3]

Design and intake. Tenants apply for the program at NPI’s website. Upon completion of the application, tenants are immediately directed to our baseline survey (more details below). Tenants are presently eligible to receive full legal assistance only if they already have received an eviction filing. After determining eligibility, tenants are randomized into receiving an offer for full assistance.

Tenants are eligible to receive one-time counseling even if they have not received an eviction filing. We randomly select some tenants to enter a queue for one-time counseling.

Intended comparisons. Because we have two treatments and a control, we preregister our “primary” and “secondary” comparisons.

We have uncertainty about the number of people who will be treated with the light-touch (secondary) treatment. This affects what we anticipate the primary comparisons in the experiment will be.

If that secondary treatment remains small relative to the control, then our main comparison will be between the primary treatment and the control. If the secondary treatment are a larger share of the sample, then our main comparison will be between the primary versus versus the secondary treatment pooled with the control.

ITT/IV. We will study the Intent to Treat effect of being offered a lawyer. There is incomplete compliance since some tenants are no longer eligible, do not reply to, or do not accept the offer of a lawyer (even though they must first apply for one). We will also use the random variation in the offer to instrument for the effect of lawyers on a given outcome.

Timing of outcomes. We will present results at sensible timeframes (e.g., 3, 6, and 12 months after filing or application). We will also present survival curves, in which we will test hypotheses using Wilcoxon tests or similar.

Randomization. NPI began providing the primary legal assistance treatment starting in March 2022. NPI asks us to randomize tenants into the sample based on the availability of the legal counsel. Because we are merely assisting the partner with randomization on a program that they intend to do otherwise, we do not have scope to stop the sample for piloting.

Pooling with Pilot sample. We have been piloting the process since March 2022 and initially registered in December 2022. There was (and remains) uncertainty about how large the secondary treatment will be and how large the sample will be. As a result, we have delayed preregistration until we had more information. Since the uncertainty has not yet been resolved, in the interest of transparency, we preregister our outcomes in December 2022 with the intention of pooling pilots with the main sample wherever possible, as our primary specification. We will also show all our estimates in separate exhibits that drop the pilot sample (who applied before the date of the preregistration), but we do not presently anticipate that these dropped estimates will be our main sample.

The only rationale we foresee that may change the assessment of what our main sample will be is the interaction with the local Emergency Rental and Utilities Assistance Program, as explained above. If it turns out that the access to funds from the local ERAP was critical and the program had very different levels of effectiveness when ERAP was available, that could motivate separating the pilots that had access to the local ERAP from the primary analysis sample.

Endline surveys and collection of informal outcomes. We intend to field phone or online surveys of tenants to measure the informal outcomes listed above. Separate from these surveys, lawyers or counselors can record many informal outcomes for treated individuals. There is a natural concern about differential attrition between the treated and control sample. To address this concern, we intend to field the survey among the treated sample as well. That will permit us to test for and/or adjust the estimates by comparing the treated group’s survey outcomes to the “ground truth” recorded by lawyers. The survey outcomes will also let us study whether the tenant is employed and their monthly income from employment.

We note that there are both baseline surveys and endline surveys. Treatment assignment occurs in a staggered fashion over more than a year. We launched the endline surveys on January 26, 2023 (v1.1). We launch the baseline surveys on March 2, 2023 (v1.2); they will clearly not cover all participants.

Links to financial data. Credit-report outcomes in outcomes in Group 3 above are preliminary. We will update this part of the preregistration when we have more information about whether these linkages are feasible. Broadly, we intend to purchase credit report credits from one of the three major credit companies in the United States and link them to the treatment and control group. We will examine standard outcomes in the literature, especially the ones emphasized in Collinson et al. (2022) for comparability.

Multiple hypothesis testing. We will appropriately adjust for multiple hypotheses within groups of outcomes above.

Baseline survey. After applying for the program, tenants are redirected to an online portal where they can take the baseline survey. Depending on response rates, we may additionally employ a surveyor to call tenants to take the survey on the phone. In the baseline survey, we conduct the following modules: tenants’ willingness to pay for lawyers, counselors, and a reference good (an iPad); beliefs about the share of tenants who receive eviction judgments (if there is a filing), if the tenants do and do not have lawyers; standard elicitations about credit constraints; information about the relationship between the tenant and landlord; whether the landlord or property manager recently changed; whether the tenant intends to go to court for their case; information about past interactions between the tenant and lawyers; and measures of trust that the tenant has for various occupations in society (measured via a Trust Game) as well as general trust in others (qualitative question).

These outcomes are collected pre-treatment. They serve as: (i) sources of heterogeneity; (ii) independently useful/interesting parameters for welfare calculations; (iii) comparisons to non-applicants when studying questions relating to selection and take-up; (iv) experimental tests of the effect of budget constraints on WTP for lawyers and counselors. We collect a generalized measure of attention in the baseline survey (“Select the number two”) and will drop tenants who fail this attention check. In our primary analysis, we will not use the pilot sample, but we may aggregate the primary sample with the ~25 pilot participants for power.

We now describe several details about the willingness to pay elicitation. These are conducted as multiple price lists, incentivized using a Becker-Degroot-Marshak mechanism. Each tenant has three WTPs elicited: for a lawyer, a counselor, and a reference good (an iPad). Tenants are randomized into reporting WTPs when we give them only $50 if they are selected (and then trade off more money versus the good), versus when we give them $500 if they are selected. The purpose of this elicitation is to test if materially relaxing budget constraints affects WTP. As a secondary outcome, we test the effect of relaxing the budget constraint on the reported WTP.

We also collect a hypothetical WTP to have an eviction delayed, to measure the potential effect on tenant well-being from court delays. We randomize the number of weeks that the eviction would be delayed, which allows us to trace a demand curve. This elicitation is hypothetical, so it is registered as secondary.

Attrition tests [added v1.3]. An important concern is that the endline surveys we conduct will have low take-up rates. We randomize whether we reach out to the tenant via a phone call from a surveyor or email (and only called if the surveyors have sufficient capacity). Among a subset of emailed tenants, we also randomize the payment ($8 or $15). We will use these randomizations as instruments for participation to study the effect of attrition.

References
Collinson, Robert, John Eric Humphries, Nicholas S. Mader, Davin K. Reed, Daniel I. Tannenbaum, and Winnie Van Dijk. “Eviction and Poverty in American Cities.” 2022.

Dizon-Ross, Rebecca, and Seema Jayachandran. "Improving Willingness-to-Pay Elicitation by Including a Benchmark Good." In AEA Papers and Proceedings, vol. 112, pp. 551-555. 2022.

Not available

Randomization is conducted in statistical software or via Javascript random number calls. In some pilots and some participants in early 2023 (which may be pooled with the main sample), treatment was based on the seconds at which a person applied the program.

Household

No

400-600 treated households (primary treatment). Unclear number of untreated households. 400-600 treated households in secondary treatment. For baseline survey, we expect between 200–600 participants.

400-600 treated households (primary treatment). Unclear number of untreated households. 400-600 treated households in secondary treatment. For baseline survey, we expect between 200–600 participants.

400-600 treated households (primary treatment). Unclear number of untreated households. 400-600 treated households in secondary treatment. Baseline survey: 50% in each treatment arm for budget constrained/unconstrained WTP elicitations (100–300 observations).

Judgments. We are powered to detect a 10 pp ITT effect on judgments at the 95% confidence level in a two-sided test with 0.8 power. Moves. We assume that we successfully recontact 50% of households, with no selection into recontacting. We are powered to detect a 15 pp ITT effect on moves at the 95% confidence level in a two-sided test with 0.8 power. Financial health. We consider the effect on an aggregate index of financial health. We are powered to detect a 0.25 SD increase on this index at the 95% level in a two-sided test with 0.8 power. We acknowledge that there is less power to detect plausible effects on the financial health outcome.