The Effects of Legal Assistance on Evictions

There are two treatments and a pure control group.

The primary treatment group provides full legal assistance to tenants facing eviction. In particular, tenants who are randomized into this treatment will receive an offer for full legal representation in General Session court. The legal team constitutes a mix of experienced tenant lawyers and lawyers whom NPI recruited and trained to conduct “low-bono” legal work.

The secondary intervention provides light-touch (one-time) legal counseling and social work to tenants facing eviction. In particular, tenants in this group receive a one-time phone call from either law students or a social worker. The law students and social workers are trained by NPI to provide coaching about how to handle an eviction. We pre-register the intervention as “secondary” because we continue to face uncertainty about whether NPI will sponsor social workers or law students to reach a large number of tenants.

Given that there are two treatments, we discuss our intended comparisons below in the experimental design section.

2022-03-03

2023-12-31

There are four categories of outcomes where we will measure the treatment effect of provision of lawyers. Group 4 are outcomes that are collected prior to treatment where the raw means are still informative but cannot be affected by treatment (since collected before).

Group 1: Formal legal outcomes. The primary outcomes under this group are:
The share of eviction cases that result in judgments.
The share of eviction cases that were explicitly withdrawn and/or “nonsuited.”
The back rents owed in the eviction judgment.
Total time elapsed between filing and case resolution.

Group 2: Informal legal outcomes (preliminary in v1.0). The primary outcomes under this group are:
Whether the participant moved due to the eviction.
Whether a payment plan was formed.
The value of back rents agreed to in the payment plan.

Group 3: Financial outcomes (preliminary in v1.0).
Whether the participant was employed and their wages. Both are captured in the surveys (described below).
Credit report outcomes, including: (i) credit score; (ii) collections balance; (iii) any positive balance on auto loan or lease; (iv) any open source of revolving credit. We choose these outcomes to be comparable to the main financial health outcomes in Collinson et al. (2022).

Group 4: Other comparisons or outcomes (preliminary in v1.0).
Willingness to pay for a lawyer.
Willingness to pay for a social worker.
Willingness to pay for a court delay.

Group 1:
The share of cases where continuances are filed.
The share of cases where an eviction writ is filed.
The share of cases that went to trial.
The number of motions filed.
Total time spent in court.
The number of instances in which a judge interacts with the case [if available].
Whether the tenant had legal assistance (a “first stage”).
Payment by the local Memphis Shelby County Emergency Rental and Utilities Assistance Program.

Group 2:
If ZIP location changed on the credit report. [Note that this is categorized as an informal legal outcome, even though only visible on credit reports, because it would imply that the tenant moved.]
If the participant stayed in a homeless shelter.
Whether the landlord agreed to repairs.
Whether the tenant appeared in court.

Group 3/Group 4: N/A.

Heterogeneity:
1. Interaction with the local Emergency Rental and Utilities Assistance Program (ERAP). Prior to September 1, tenants could apply for the local ERAP, a government program that pays back rents of tenants facing eviction. The primary treatment was partially integrated with ERAP such that lawyers could use ERAP funds to pay back rents to landlords as a part of negotiations. This ended in September 2022. While we will pool the samples for power in our main analysis, an important source of heterogeneity will be the difference in treatment effects on primary outcomes, before and after the ERAP program closed. If we are sufficiently powered for the analysis, this could be a primary focus of the paper, but we have uncertainty about the number of treated units after ERAP closes.

Other sources of heterogeneity are below. We intend to invite applicants to participate in surveys at baseline. We intend to study whether the effect of treatment differs based on cuts of these characteristics below (preliminary in version 1.0).

We have not piloted surveys of these forms of heterogeneity yet and preregister them with an acknowledgement that they may change after piloting.

2. Relationship with landlord. We will ask tenants about their relationship with their landlord and examine treatment effect heterogeneity by whether the relationship is strong or weak.

3. Willingness to Pay for primary treatment (full legal services), secondary treatment (social workers/law students), or delays. We will conduct willingness to pay exercises where we ask tenants (prior to randomization) their valuations between the indicated outcome and cash. The purpose of this exercise is to: (a) obtain an estimate of tenants’ valuations, which enters the welfare effects of the policy, and (b) determine whether tenants with high valuations have larger effects from the treatment. We also intend to obtain the WTP for a reference good, which is a useful benchmark.

4. Measurements of credit constraints. We intend to measure credit constraints at baseline by asking standard questions about access to credit. We also developed a version of the WTP exercise that asks WTP for a lawyer in a state of the world where the tenant is also given cash. The difference in WTPs (when the tenant does/does not have cash) is a measure of credit constraints.

5. Measurements of financial need, e.g. income and/or rent to income ratios.

Design and intake. Tenants apply for the program at NPI’s website. Tenants are presently eligible to receive full legal assistance only if they already have received an eviction filing. After determining eligibility, tenants are randomized into receiving an offer for full assistance.

Tenants are eligible to receive one-time counseling even if they have not received an eviction filing. As of the present, we randomly select some tenants to enter a queue for one-time counseling.

Intended comparisons. Because we have two treatments and a control, we preregister our “primary” and “secondary” comparisons.

We have uncertainty about the number of people who will be treated with the light-touch (secondary) treatment. This affects what we anticipate the primary comparisons in the experiment will be.

If that secondary treatment remains small relative to the pure control, then our primary comparison will be between the primary treatment and the pure control. If the secondary treatment becomes a larger share of the sample, then our primary comparison will be between the primary and secondary treatment pooled with or separated from the pure control.

ITT/IV. We will study the Intent to Treat effect of being offered a lawyer. There is incomplete compliance since some tenants are no longer eligible, do not reply to, or do not accept the offer of a lawyer (even though they must first apply for one). We will also use the random variation in the offer to instrument for the effect of lawyers on a given outcome.

Timing of outcomes. We will present results at sensible timeframes (e.g., 3, 6, and 12 months after filing). We will also present survival curves, in which we will test hypotheses using Wilcoxon tests or similar.

Randomization. NPI began providing the primary legal assistance treatment starting in March 2022. NPI asks us to randomize tenants into the sample based on the availability of the legal counsel. Because we are merely assisting the partner with randomization on a program that they intend to do otherwise, we do not have scope to stop the sample for piloting.

Pooling with Pilot sample. We have been piloting the process since March 2022. There was (and remains) uncertainty about how large the secondary treatment will be and how large the sample will be. As a result, we have delayed preregistration until we had more information. Since the uncertainty has not yet been resolved, in the interest of transparency, we preregister our outcomes now with the intention of pooling with the main sample wherever possible, as our primary specification. We will also show all our estimates in separate exhibits that drop the pilot sample (who applied before the date of the preregistration), but we do not presently anticipate that these dropped estimates will be our main sample.

The only rationale we foresee that may change the assessment of what our main sample will be is the interaction with the local Emergency Rental and Utilities Assistance Program, as explained above. If it turns out that the access to funds from the local ERAP was critical and the program had very different levels of effectiveness, that could motivate separating the pilots that had access to the local ERAP from the primary analysis sample.

Surveys and collection of informal outcomes. We intend to field phone or online surveys of tenants to measure the informal outcomes listed above. Separate from these surveys, lawyers or counselors can record many informal outcomes for treated individuals. There is a natural concern about differential attrition between the treated and control sample. To address this concern, we intend to field the survey among the treated sample as well. That will permit us to test for and/or adjust the estimates by comparing the treated group’s survey outcomes to the “ground truth” recorded by lawyers. The survey outcomes will also let us study whether the tenant is employed and their monthly income from employment.

Links to financial data. Credit-report outcomes in outcomes in Group 3 above are preliminary. We will update this part of the preregistration when we have more information about whether these linkages are feasible. Broadly, we intend to purchase credit report credits from one of the three major credit companies in the United States and link them to the treatment and control group. We will examine standard outcomes in the literature, especially the ones emphasized in Collinson et al. (2022) for comparability.

Multiple hypothesis testing. We will appropriately adjust for multiple hypotheses within groups of outcomes above.

Data collection at baseline. At baseline, we intend to invite tenants to participate in a survey. We will collect WTP data and other additional survey measures there.

References
Collinson, Robert, John Eric Humphries, Nicholas S. Mader, Davin K. Reed, Daniel I. Tannenbaum, and Winnie Van Dijk. “Eviction and Poverty in American Cities.” 2022.

Not available

Randomization is conducted in statistical software. In some pilots (which may be pooled with the main sample), treatment was based on the seconds at which a person applied the program.

Household

No

400-600 treated households (primary treatment). Unclear number of untreated households. 400-600 treated households in secondary treatment.

400-600 treated households (primary treatment). Unclear number of untreated households. 400-600 treated households in secondary treatment.

400-600 treated households (primary treatment). Unclear number of untreated households. 400-600 treated households in secondary treatment.

Judgments. We are powered to detect a 10 pp ITT effect on judgments at the 95% confidence level in a two-sided test with 0.8 power. Moves. We assume that we successfully recontact 50% of households, with no selection into recontacting. We are powered to detect a 15 pp ITT effect on moves at the 95% confidence level in a two-sided test with 0.8 power. Financial health. We consider the effect on an aggregate index of financial health. We are powered to detect a 0.25 SD increase on this index at the 95% level in a two-sided test with 0.8 power. We acknowledge that there is less power to detect plausible effects on the financial health outcome.