Experimental Design
Design and intake. Tenants apply for the program at NPI’s website. Tenants are presently eligible to receive full legal assistance only if they already have received an eviction filing. After determining eligibility, tenants are randomized into receiving an offer for full assistance.
Tenants are eligible to receive one-time counseling even if they have not received an eviction filing. As of the present, we randomly select some tenants to enter a queue for one-time counseling.
Intended comparisons. Because we have two treatments and a control, we preregister our “primary” and “secondary” comparisons.
We have uncertainty about the number of people who will be treated with the light-touch (secondary) treatment. This affects what we anticipate the primary comparisons in the experiment will be.
If that secondary treatment remains small relative to the pure control, then our primary comparison will be between the primary treatment and the pure control. If the secondary treatment becomes a larger share of the sample, then our primary comparison will be between the primary and secondary treatment pooled with or separated from the pure control.
ITT/IV. We will study the Intent to Treat effect of being offered a lawyer. There is incomplete compliance since some tenants are no longer eligible, do not reply to, or do not accept the offer of a lawyer (even though they must first apply for one). We will also use the random variation in the offer to instrument for the effect of lawyers on a given outcome.
Timing of outcomes. We will present results at sensible timeframes (e.g., 3, 6, and 12 months after filing). We will also present survival curves, in which we will test hypotheses using Wilcoxon tests or similar.
Randomization. NPI began providing the primary legal assistance treatment starting in March 2022. NPI asks us to randomize tenants into the sample based on the availability of the legal counsel. Because we are merely assisting the partner with randomization on a program that they intend to do otherwise, we do not have scope to stop the sample for piloting.
Pooling with Pilot sample. We have been piloting the process since March 2022. There was (and remains) uncertainty about how large the secondary treatment will be and how large the sample will be. As a result, we have delayed preregistration until we had more information. Since the uncertainty has not yet been resolved, in the interest of transparency, we preregister our outcomes now with the intention of pooling with the main sample wherever possible, as our primary specification. We will also show all our estimates in separate exhibits that drop the pilot sample (who applied before the date of the preregistration), but we do not presently anticipate that these dropped estimates will be our main sample.
The only rationale we foresee that may change the assessment of what our main sample will be is the interaction with the local Emergency Rental and Utilities Assistance Program, as explained above. If it turns out that the access to funds from the local ERAP was critical and the program had very different levels of effectiveness, that could motivate separating the pilots that had access to the local ERAP from the primary analysis sample.
Surveys and collection of informal outcomes. We intend to field phone or online surveys of tenants to measure the informal outcomes listed above. Separate from these surveys, lawyers or counselors can record many informal outcomes for treated individuals. There is a natural concern about differential attrition between the treated and control sample. To address this concern, we intend to field the survey among the treated sample as well. That will permit us to test for and/or adjust the estimates by comparing the treated group’s survey outcomes to the “ground truth” recorded by lawyers. The survey outcomes will also let us study whether the tenant is employed and their monthly income from employment.
We note that there are both baseline surveys and endline surveys. Treatment assignment occurs in a staggered fashion over more than a year. We launched the endline surveys on January 26, 2023 (v1.1). We intend to launch the baseline surveys after this; they will clearly not cover all participants.
Links to financial data. Credit-report outcomes in outcomes in Group 3 above are preliminary. We will update this part of the preregistration when we have more information about whether these linkages are feasible. Broadly, we intend to purchase credit report credits from one of the three major credit companies in the United States and link them to the treatment and control group. We will examine standard outcomes in the literature, especially the ones emphasized in Collinson et al. (2022) for comparability.
Multiple hypothesis testing. We will appropriately adjust for multiple hypotheses within groups of outcomes above.
Data collection at baseline. At baseline, we intend to invite tenants to participate in a survey. We will collect WTP data and other additional survey measures there.
References
Collinson, Robert, John Eric Humphries, Nicholas S. Mader, Davin K. Reed, Daniel I. Tannenbaum, and Winnie Van Dijk. “Eviction and Poverty in American Cities.” 2022.