|
Field
Abstract
|
Before
Abstract
We conduct a randomized controlled trial (RCT) of the effect of providing attorneys to tenants facing eviction. We partner with a local nonprofit, Neighborhood Preservation, Inc., in Memphis, Tennessee. NPI has received independent grants to provide 200–600 lawyers to represent tenants with eviction filings in Shelby County General Sessions Court. We assist NPI with randomizing the provision of lawyers. We study the impact of lawyers on four groups of outcomes: (1) formal eviction outcomes, including judgments, writs, nonsuits, time in court, and money owed the landlord; (2) informal eviction outcomes collected via survey, including moves, and informal bargaining; (3) financial outcomes, including employment and wages (collected via survey), and other financial outcomes collected from merged credit reports, and (4) other outcomes collected at baseline (willingness to pay).
Disclaimer: The purpose of this iteration of the preregistration is to preregister: (i) the collection of outcomes in group (1), and (ii) the primary treatment (full legal services). The secondary treatment is still preliminary. We provide information based on our present understanding. The partner is sponsoring the provision of attorneys and we cannot delay the treatments while we get more information or pilot. Thus, we wanted to preregister the outcomes in (1) and the primary treatment as soon as possible. We are in the process of piloting outcomes in (2) and (4) and we are unsure that we will be able to implement the outcomes in (3) due to funding constraints. We will more fully register (2)--(4) when we have more information.
We emphasize that we have not begun collecting data from outcomes in Groups 2–4, so these are completely pre-registered. We have been able to observe some preliminary results from Group 1 with a pilot sample that we intend to pool with the main estimates. That is why we register these outcomes at this stage, even though Groups 2–4 are still preliminary.
|
After
We conduct a randomized controlled trial (RCT) of the effect of providing attorneys to tenants facing eviction. We partner with a local nonprofit, Neighborhood Preservation, Inc., in Memphis, Tennessee. NPI has received independent grants to provide 200–600 lawyers to represent tenants with eviction filings in Shelby County General Sessions Court. We assist NPI with randomizing the provision of lawyers. We study the impact of lawyers on three groups of outcomes: (1) formal eviction outcomes, including judgments, writs, nonsuits, time in court, and money owed the landlord; (2) informal eviction outcomes collected via endline survey, including moves, and informal bargaining; (3) financial outcomes collected via endline survey and merged credit reports. We also study a fourth group of outcomes: (4) other outcomes collected in baseline surveys (e.g., willingness to pay), though we do not study treatment effects on these outcomes since they are collected prior to treatment.
Disclaimer: The trial is registered in a staggered fashion. In v1.0, we registered outcomes in Group 1 and the primary treatment. We also gave our best prediction about registration of the secondary treatment and outcomes in Groups 2-3. In v1.1, we register outcomes in Group 2, prior to launching the endline surveys.
In all registrations, we provide information based on our present understanding. The partner is sponsoring the provision of attorneys and we cannot delay the treatments while we get more information or pilot.
We emphasize that the outcomes in Group 2 have not yet been collected, so these are completely preregistered. We have been able to observe some preliminary results from Group 1 with a pilot sample that we intend to pool with the main estimates. That is why we register these outcomes as soon as we could in v1.0.
Timeline:
v1.0 - initial registration, after treatment assignment for some participants and after some administrative outcomes collected. Before any surveys collected and credit reports purchased.
v1.1 - updated registration for outcomes collected in endline survey. Fully preregistered before endline surveys launched.
For primary outcomes:
Group 1 was registered after data collection began (v1.0).
Group 2 was pre-registered before endline data collection began (preliminary in v1.0, registered v1.1)
Groups 3 and 4 remain preliminary (through v1.1).
Note: We pre-register secondary survey outcomes before data is collected. Some secondary survey outcomes are in Group 3 and are indicated as such in the secondary outcomes section.
|
|
Field
Last Published
|
Before
January 03, 2023 05:15 PM
|
After
January 26, 2023 04:23 PM
|
|
Field
Intervention (Public)
|
Before
There are two treatments and a pure control group.
The primary treatment group provides full legal assistance to tenants facing eviction. In particular, tenants who are randomized into this treatment will receive an offer for full legal representation in General Session court. The legal team constitutes a mix of experienced tenant lawyers and lawyers whom NPI recruited and trained to conduct “low-bono” legal work.
The secondary intervention provides light-touch (one-time) legal counseling and social work to tenants facing eviction. In particular, tenants in this group receive a one-time phone call from either law students or a social worker. The law students and social workers are trained by NPI to provide coaching about how to handle an eviction. We pre-register the intervention as “secondary” because we continue to face uncertainty about whether NPI will sponsor social workers or law students to reach a large number of tenants.
Given that there are two treatments, we discuss our intended comparisons below in the experimental design section.
|
After
There are two treatments and a pure control group.
The primary treatment group provides full legal assistance to tenants facing eviction. In particular, tenants who are randomized into this treatment will receive an offer for full legal representation in General Session court. The legal team constitutes a mix of experienced tenant lawyers and lawyers whom NPI recruited and trained to conduct “low-bono” legal work.
The secondary intervention provides light-touch (one-time) legal counseling and social work to tenants facing eviction. In particular, tenants in this group receive a one-time phone call from either law students or a social worker. The law students and social workers are trained by NPI to provide coaching about how to handle an eviction. We pre-register the intervention as “secondary” because we continue to face uncertainty about whether NPI will sponsor social workers or law students to reach a large number of tenants. The light-touch counseling is given only to people who do not receive the primary treatment.
Given that there are two treatments, we discuss our intended comparisons below in the experimental design section.
|
|
Field
Primary Outcomes (End Points)
|
Before
There are four categories of outcomes where we will measure the treatment effect of provision of lawyers. Group 4 are outcomes that are collected prior to treatment where the raw means are still informative but cannot be affected by treatment (since collected before).
Group 1: Formal legal outcomes. The primary outcomes under this group are:
The share of eviction cases that result in judgments.
The share of eviction cases that were explicitly withdrawn and/or “nonsuited.”
The back rents owed in the eviction judgment.
Total time elapsed between filing and case resolution.
Group 2: Informal legal outcomes (preliminary in v1.0). The primary outcomes under this group are:
Whether the participant moved due to the eviction.
Whether a payment plan was formed.
The value of back rents agreed to in the payment plan.
Group 3: Financial outcomes (preliminary in v1.0).
Whether the participant was employed and their wages. Both are captured in the surveys (described below).
Credit report outcomes, including: (i) credit score; (ii) collections balance; (iii) any positive balance on auto loan or lease; (iv) any open source of revolving credit. We choose these outcomes to be comparable to the main financial health outcomes in Collinson et al. (2022).
Group 4: Other comparisons or outcomes (preliminary in v1.0).
Willingness to pay for a lawyer.
Willingness to pay for a social worker.
Willingness to pay for a court delay.
|
After
There are four categories of outcomes where we will measure the treatment effect of provision of lawyers. Group 4 are outcomes that are collected prior to treatment where the raw means are still informative but cannot be affected by treatment (since collected before).
Group 1: Formal legal outcomes (registered in v1.0 after data collection began). The primary outcomes under this group are:
The share of eviction cases that result in judgments.
The share of eviction cases that were explicitly withdrawn and/or “nonsuited.”
The back rents owed in the eviction judgment.
Total time elapsed between filing and case resolution.
Group 2: Informal legal outcomes (preliminary in v1.0, preregistered in v1.1 before endline survey collection began). The primary outcomes under this group are:
Whether the participant moved (from the location where they had the eviction filing that triggered their application for legal assistance)
Whether the tenant paid a lower value of overdue rents than what the landlord initially asked for
The value of back rents that the tenant paid
Group 3: Financial outcomes (preliminary in v1.0 and v1.1).
Credit report outcomes, including: (i) credit score; (ii) collections balance; (iii) any positive balance on auto loan or lease; (iv) any open source of revolving credit. We choose these outcomes to be comparable to the main financial health outcomes in Collinson et al. (2022).
Group 4: Other comparisons or outcomes (preliminary in v1.0 and v1.1).
Willingness to pay for a lawyer.
Willingness to pay for a social worker.
Willingness to pay for a court delay.
|
|
Field
Experimental Design (Public)
|
Before
Design and intake. Tenants apply for the program at NPI’s website. Tenants are presently eligible to receive full legal assistance only if they already have received an eviction filing. After determining eligibility, tenants are randomized into receiving an offer for full assistance.
Tenants are eligible to receive one-time counseling even if they have not received an eviction filing. As of the present, we randomly select some tenants to enter a queue for one-time counseling.
Intended comparisons. Because we have two treatments and a control, we preregister our “primary” and “secondary” comparisons.
We have uncertainty about the number of people who will be treated with the light-touch (secondary) treatment. This affects what we anticipate the primary comparisons in the experiment will be.
If that secondary treatment remains small relative to the pure control, then our primary comparison will be between the primary treatment and the pure control. If the secondary treatment becomes a larger share of the sample, then our primary comparison will be between the primary and secondary treatment pooled with or separated from the pure control.
ITT/IV. We will study the Intent to Treat effect of being offered a lawyer. There is incomplete compliance since some tenants are no longer eligible, do not reply to, or do not accept the offer of a lawyer (even though they must first apply for one). We will also use the random variation in the offer to instrument for the effect of lawyers on a given outcome.
Timing of outcomes. We will present results at sensible timeframes (e.g., 3, 6, and 12 months after filing). We will also present survival curves, in which we will test hypotheses using Wilcoxon tests or similar.
Randomization. NPI began providing the primary legal assistance treatment starting in March 2022. NPI asks us to randomize tenants into the sample based on the availability of the legal counsel. Because we are merely assisting the partner with randomization on a program that they intend to do otherwise, we do not have scope to stop the sample for piloting.
Pooling with Pilot sample. We have been piloting the process since March 2022. There was (and remains) uncertainty about how large the secondary treatment will be and how large the sample will be. As a result, we have delayed preregistration until we had more information. Since the uncertainty has not yet been resolved, in the interest of transparency, we preregister our outcomes now with the intention of pooling with the main sample wherever possible, as our primary specification. We will also show all our estimates in separate exhibits that drop the pilot sample (who applied before the date of the preregistration), but we do not presently anticipate that these dropped estimates will be our main sample.
The only rationale we foresee that may change the assessment of what our main sample will be is the interaction with the local Emergency Rental and Utilities Assistance Program, as explained above. If it turns out that the access to funds from the local ERAP was critical and the program had very different levels of effectiveness, that could motivate separating the pilots that had access to the local ERAP from the primary analysis sample.
Surveys and collection of informal outcomes. We intend to field phone or online surveys of tenants to measure the informal outcomes listed above. Separate from these surveys, lawyers or counselors can record many informal outcomes for treated individuals. There is a natural concern about differential attrition between the treated and control sample. To address this concern, we intend to field the survey among the treated sample as well. That will permit us to test for and/or adjust the estimates by comparing the treated group’s survey outcomes to the “ground truth” recorded by lawyers. The survey outcomes will also let us study whether the tenant is employed and their monthly income from employment.
Links to financial data. Credit-report outcomes in outcomes in Group 3 above are preliminary. We will update this part of the preregistration when we have more information about whether these linkages are feasible. Broadly, we intend to purchase credit report credits from one of the three major credit companies in the United States and link them to the treatment and control group. We will examine standard outcomes in the literature, especially the ones emphasized in Collinson et al. (2022) for comparability.
Multiple hypothesis testing. We will appropriately adjust for multiple hypotheses within groups of outcomes above.
Data collection at baseline. At baseline, we intend to invite tenants to participate in a survey. We will collect WTP data and other additional survey measures there.
References
Collinson, Robert, John Eric Humphries, Nicholas S. Mader, Davin K. Reed, Daniel I. Tannenbaum, and Winnie Van Dijk. “Eviction and Poverty in American Cities.” 2022.
|
After
Design and intake. Tenants apply for the program at NPI’s website. Tenants are presently eligible to receive full legal assistance only if they already have received an eviction filing. After determining eligibility, tenants are randomized into receiving an offer for full assistance.
Tenants are eligible to receive one-time counseling even if they have not received an eviction filing. As of the present, we randomly select some tenants to enter a queue for one-time counseling.
Intended comparisons. Because we have two treatments and a control, we preregister our “primary” and “secondary” comparisons.
We have uncertainty about the number of people who will be treated with the light-touch (secondary) treatment. This affects what we anticipate the primary comparisons in the experiment will be.
If that secondary treatment remains small relative to the pure control, then our primary comparison will be between the primary treatment and the pure control. If the secondary treatment becomes a larger share of the sample, then our primary comparison will be between the primary and secondary treatment pooled with or separated from the pure control.
ITT/IV. We will study the Intent to Treat effect of being offered a lawyer. There is incomplete compliance since some tenants are no longer eligible, do not reply to, or do not accept the offer of a lawyer (even though they must first apply for one). We will also use the random variation in the offer to instrument for the effect of lawyers on a given outcome.
Timing of outcomes. We will present results at sensible timeframes (e.g., 3, 6, and 12 months after filing). We will also present survival curves, in which we will test hypotheses using Wilcoxon tests or similar.
Randomization. NPI began providing the primary legal assistance treatment starting in March 2022. NPI asks us to randomize tenants into the sample based on the availability of the legal counsel. Because we are merely assisting the partner with randomization on a program that they intend to do otherwise, we do not have scope to stop the sample for piloting.
Pooling with Pilot sample. We have been piloting the process since March 2022. There was (and remains) uncertainty about how large the secondary treatment will be and how large the sample will be. As a result, we have delayed preregistration until we had more information. Since the uncertainty has not yet been resolved, in the interest of transparency, we preregister our outcomes now with the intention of pooling with the main sample wherever possible, as our primary specification. We will also show all our estimates in separate exhibits that drop the pilot sample (who applied before the date of the preregistration), but we do not presently anticipate that these dropped estimates will be our main sample.
The only rationale we foresee that may change the assessment of what our main sample will be is the interaction with the local Emergency Rental and Utilities Assistance Program, as explained above. If it turns out that the access to funds from the local ERAP was critical and the program had very different levels of effectiveness, that could motivate separating the pilots that had access to the local ERAP from the primary analysis sample.
Surveys and collection of informal outcomes. We intend to field phone or online surveys of tenants to measure the informal outcomes listed above. Separate from these surveys, lawyers or counselors can record many informal outcomes for treated individuals. There is a natural concern about differential attrition between the treated and control sample. To address this concern, we intend to field the survey among the treated sample as well. That will permit us to test for and/or adjust the estimates by comparing the treated group’s survey outcomes to the “ground truth” recorded by lawyers. The survey outcomes will also let us study whether the tenant is employed and their monthly income from employment.
We note that there are both baseline surveys and endline surveys. Treatment assignment occurs in a staggered fashion over more than a year. We launched the endline surveys on January 26, 2023 (v1.1). We intend to launch the baseline surveys after this; they will clearly not cover all participants.
Links to financial data. Credit-report outcomes in outcomes in Group 3 above are preliminary. We will update this part of the preregistration when we have more information about whether these linkages are feasible. Broadly, we intend to purchase credit report credits from one of the three major credit companies in the United States and link them to the treatment and control group. We will examine standard outcomes in the literature, especially the ones emphasized in Collinson et al. (2022) for comparability.
Multiple hypothesis testing. We will appropriately adjust for multiple hypotheses within groups of outcomes above.
Data collection at baseline. At baseline, we intend to invite tenants to participate in a survey. We will collect WTP data and other additional survey measures there.
References
Collinson, Robert, John Eric Humphries, Nicholas S. Mader, Davin K. Reed, Daniel I. Tannenbaum, and Winnie Van Dijk. “Eviction and Poverty in American Cities.” 2022.
|
|
Field
Secondary Outcomes (End Points)
|
Before
Group 1:
The share of cases where continuances are filed.
The share of cases where an eviction writ is filed.
The share of cases that went to trial.
The number of motions filed.
Total time spent in court.
The number of instances in which a judge interacts with the case [if available].
Whether the tenant had legal assistance (a “first stage”).
Payment by the local Memphis Shelby County Emergency Rental and Utilities Assistance Program.
Group 2:
If ZIP location changed on the credit report. [Note that this is categorized as an informal legal outcome, even though only visible on credit reports, because it would imply that the tenant moved.]
If the participant stayed in a homeless shelter.
Whether the landlord agreed to repairs.
Whether the tenant appeared in court.
Group 3/Group 4: N/A.
Heterogeneity:
1. Interaction with the local Emergency Rental and Utilities Assistance Program (ERAP). Prior to September 1, tenants could apply for the local ERAP, a government program that pays back rents of tenants facing eviction. The primary treatment was partially integrated with ERAP such that lawyers could use ERAP funds to pay back rents to landlords as a part of negotiations. This ended in September 2022. While we will pool the samples for power in our main analysis, an important source of heterogeneity will be the difference in treatment effects on primary outcomes, before and after the ERAP program closed. If we are sufficiently powered for the analysis, this could be a primary focus of the paper, but we have uncertainty about the number of treated units after ERAP closes.
Other sources of heterogeneity are below. We intend to invite applicants to participate in surveys at baseline. We intend to study whether the effect of treatment differs based on cuts of these characteristics below (preliminary in version 1.0).
We have not piloted surveys of these forms of heterogeneity yet and preregister them with an acknowledgement that they may change after piloting.
2. Relationship with landlord. We will ask tenants about their relationship with their landlord and examine treatment effect heterogeneity by whether the relationship is strong or weak.
3. Willingness to Pay for primary treatment (full legal services), secondary treatment (social workers/law students), or delays. We will conduct willingness to pay exercises where we ask tenants (prior to randomization) their valuations between the indicated outcome and cash. The purpose of this exercise is to: (a) obtain an estimate of tenants’ valuations, which enters the welfare effects of the policy, and (b) determine whether tenants with high valuations have larger effects from the treatment. We also intend to obtain the WTP for a reference good, which is a useful benchmark.
4. Measurements of credit constraints. We intend to measure credit constraints at baseline by asking standard questions about access to credit. We also developed a version of the WTP exercise that asks WTP for a lawyer in a state of the world where the tenant is also given cash. The difference in WTPs (when the tenant does/does not have cash) is a measure of credit constraints.
5. Measurements of financial need, e.g. income and/or rent to income ratios.
|
After
Group 1:
The share of cases where continuances are filed.
The share of cases where an eviction writ is filed.
The share of cases that went to trial.
The number of motions filed.
Total time spent in court.
The number of instances in which a judge interacts with the case [if available].
Whether the tenant had legal assistance (a “first stage”).
Payment by the local Memphis Shelby County Emergency Rental and Utilities Assistance Program.
Group 2:
Whether the participant was employed and their wages. Both are captured in the surveys (described below).
If ZIP location changed on the credit report. [Note that this is categorized as an informal legal outcome, even though only visible on credit reports, because it would imply that the tenant moved. It is therefore preliminary through v1.1, unlike other group 2 outcomes.]
Whether the tenant has agreed to pay back less than what the landlord asked for
The amount of back rents that the tenant agreed to pay back
Whether a payment plan or settlement was formed
If the participant stayed in a homeless shelter
Whether the landlord agreed to repairs
Whether the tenant appeared in court
Group 3:
Employment
Earnings
[Note that these are categorized as economic outcomes but are collected in the survey. They are therefore preregistered in v1.1 before data collection began.]
Group 4: N/A
Heterogeneity:
1. Interaction with the local Emergency Rental and Utilities Assistance Program (ERAP). Prior to September 1, tenants could apply for the local ERAP, a government program that pays back rents of tenants facing eviction. The primary treatment was partially integrated with ERAP such that lawyers could use ERAP funds to pay back rents to landlords as a part of negotiations. This ended in September 2022. While we will pool the samples for power in our main analysis, an important source of heterogeneity will be the difference in treatment effects on primary outcomes, before and after the ERAP program closed. If we are sufficiently powered for the analysis, this could be a primary focus of the paper, but we have uncertainty about the number of treated units after ERAP closes.
Other sources of treatment-effect heterogeneity are below. We intend to invite applicants to participate in surveys at baseline. We intend to study whether the effect of treatment differs based on cuts of these characteristics below (preliminary in version 1.0 and v1.1).
We have not piloted surveys of these forms of heterogeneity yet and preregister them with an acknowledgement that they may change after piloting.
2. Relationship with landlord. We will ask tenants about their relationship with their landlord and examine treatment effect heterogeneity by whether the relationship is strong or weak.
3. Willingness to Pay for primary treatment (full legal services), secondary treatment (social workers/law students), or delays. We will conduct willingness to pay exercises where we ask tenants (prior to randomization) their valuations between the indicated outcome and cash. The purpose of this exercise is to: (a) obtain an estimate of tenants’ valuations, which enters the welfare effects of the policy, and (b) determine whether tenants with high valuations have larger effects from the treatment. We also intend to obtain the WTP for a reference good, which is a useful benchmark.
4. Measurements of credit constraints. We intend to measure credit constraints at baseline by asking standard questions about access to credit. We also developed a version of the WTP exercise that asks WTP for a lawyer in a state of the world where the tenant is also given cash. The difference in WTPs (when the tenant does/does not have cash) is a measure of credit constraints.
5. Measurements of financial need, e.g. income and/or rent to income ratios.
|