Helping Parents Earn More: A Randomized Evaluation of Holistic Wrap-around Services for Parents and their Children

The Elevating Families program provides participants with a bundle of services meant to alleviate generational poverty. Specifically, parent participants work with a Mobility Mentor to develop a Goal Action Plan that addresses five key areas: family stability, health and well-being, financial management, education and training, and employment and career. In order to help motivate their progress and recognize their achievements, participants receive earned incentives upon the completion of their goals. In addition to meetings with Mobility Mentors, participants will have access to free childcare services during mentoring sessions and classes on money management, career development, and parenting skills at the Elevating Families center on CCNN’s new family services campus.

In the first 4 weeks of program enrollment, participants and their Mobility Mentors are required to have completed their first Bridge to Self-Sufficiency: this a holistic evaluation of the barriers and advantages that each participant has in the five focus areas and includes a long-term goal setting component. Roughly two weeks later, they then draft their first Goal Action Plan, a detailed outline of how a participant plans to meet the goals they set in their Bridge to Self-Sufficiency. The action plan includes transitionary goals, pillars, action steps, dates, and incentive amounts. Meetings or contacts between Mobility Mentors and participants then occur every 2 weeks and continue throughout the program, with participants reviewing their progress via a new Bridge to Self-Sufficiency after 6 months. As participants make significant progress towards their goals and are able to demonstrate self-sufficiency, participation is titrated as participants move towards greater independence. When a participant household has been in the program for 12 months, CCNN and program participants begin making a plan for concluding supportive services and transitioning to life without Elevating Families. Participants then follow this plan for the next 12 months, after which point they have completed the program. However, even after they have exited, participants are still given access to limited support from their Mobility Mentors.

2023-11-12

2027-11-12

For enrolled parents our key outcomes are annual earnings observed in Year 3 (defined as the third year after program enrollment) and a labor market index, which includes: an indicator for any positive earnings in Year 3, an indicator for earning over $20,000 in Year 3, an indicator for earning over $35,000 in Year 3, and annual earnings in Year 3.

For children of enrolled parents, our primary outcomes are a school Index, which includes GPA, attendance rate, and indicators for chronically absenteeism, on-time grade progression, any detention, and any suspension of expulsion. We will also create a CPS involvement index, which includes indicators for any call, any referral, an investigation, and a substantiated investigation.

Our primary outcomes will include the four variables described above, two pertaining to enrolled parental participants and two relating to their children. All standardized indices will be computed using the control group mean and standard deviation as described in Kling et al. (2007). Exact variables included in these indices may be subject to change based on what data are available.

Citations:
Kling, J. R., J. B. Liebman, and L. F. Katz (2007). Experimental analysis of neighborhood effects. Econometrica 75 (1), 83–119.

For enrolled parents we will examine:
1.) Additional Labor Market Outcomes: We will report each component of the labor market index as well as a few others (if data are available), such as Household income, as measured through an endline survey, and indicators for moving into higher-paying occupation or moving into a high-growth sector (e.g. health care, IT, etc.).

2.) Financial Health and Credit Index: We plan to purchase panel data from Experian that will provide information on how a participant’s financial health evolves through the program. If the following variables are available and of high quality, we will combine them into a single standardized financial health and credit index. We will adapt the indices used by Miller and Soo (2021) to measure credit access and delinquency. Our index will be composed of two distinct indices – a credit index and a delinquency index – that will also be reported separately in supplemental tables. The exact variables that enter these indices may change as we learn more about the available data from Experian. For the credit index, we will include credit score estimated using VantageScore, total available credit on all accounts in last 3 months, and total balance on all accounts in last 3 months. For the delinquency index we will use amount past due on trades presently 30 days delinquent reported in the last 6 months, amount of debt past due held by third-party collection agencies, total number of public record bankruptcies and tax liens, total number of trades presently satisfactory that were ever 30 or more days delinquent or derogatory, excluding collections.

3.) Health & Well-being Index: Provided sufficient funding, the follow-up survey will collect the following health and well-being measures, which were included on the baseline survey. These will be combined into a singular standardized health and well-being index, generated from the following survey questions: participant displays major depression symptoms (i.e. has a score of 10 or higher on the PHQ-8 instrument), participant displays symptoms of generalized anxiety disorder (i.e. scores a 3 or higher on the GAD-2 instrument), participant reports being generally not too happy, and participant reports having fair or poor health when asked about their general health.

4.) Public Benefit Expenditures: While we would like to include information on usage of public benefits, it is unclear what data will be available from the state. Ideally, this measure would include the value of all public benefits received by participants, including SNAP, TANF, rental support and the value of in-kind transfers, such as public housing.

We will also report disaggregated schooling and CPS outcomes for children of parents enrolled in the study.

Citations:
Miller, S. and C. K. Soo (2021). Do neighborhoods affect the credit market decisions of low-income borrowers? evidence from the moving to opportunity experiment. The Review of Financial Studies 34 (2), 827–863.

Our primary specification estimates the impact of the offer to enroll in Elevating Families or intention-to-treat (ITT) effects of the program on outcomes. The basic specification is

yi = βTreatmenti + γyi0 + αs(i) + εi (1)

where yi is an outcome for enrolled participant i, Treatmenti indicates whether participant i was randomly assigned to the treatment group, yi0 is the outcome of participant i at baseline, and αs(i) are strata fixed effects (i.e., randomization-pair indicators). Within each randomization block, we plan to randomize between pairs of parents that are sequentially matched on employment status, reported income, months worked in the past 12 months, and household income at baseline. The coefficient of interest – β – estimates the average difference in outcomes between treatment and control groups, controlling for baseline outcome levels. This is our preferred specification provided that treatment and control groups do not meaningfully differ (by chance) in observable baseline characteristics.

We will also estimate treatment effects conditional on control vector Xi′ to account for any sampling variation in the composition of treatment and control groups:

yi =βTreatmenti +Xi′δ+γyi0 +αs(i) +εi (2)

Xi′ includes household-level controls that will be selected from the following variables collected in the baseline survey: gender, educational attainment, age, race, ethnicity, home language, parental status, marital status, disability status, employment status, employment history, household income, individual earnings, individual weekly hours worked, and felony arrest history. As described in Chetty et al. (2018), we will select baseline covariates using “the state-of-art approach to penalize overfitting such as LASSO (at present) or the preferred machine learning approach for covariate selection available at the time the analysis is done”.

In addition to the reduced-form estimates obtained by Equations 1 and 2, we are also interested in estimating the causal impact of the program on those who received it, or treatment-on-treated (TOT) effects. To this end, we will instrument for different measures of program take-up using treatment status. Using 2SLS, we will estimate the system:

yi = βDi + γy0i + X′iδ + δs(i) + ϵi (3)

Di = πTreatmenti + ηy0i + X′iν + ζs(i) + υi (4)

where Di is a measure of program enrollment and/or engagement. These exact engagement measure will depend on the data that are collected by CCNN. Similar to Engle et al. (2022), we would ideally measure enrollment as the participant having completed a bridge assessment and set at least one goal with their mentor. Similarly, engagement could be measured as the fraction of months the participant was marked as “actively engaged” throughout the two-year program period. Under reasonable assumptions, β^Di captures the causal impact of engagement with the program on outcome yi. Note that this parameter equals the intent-to-treat parameter (β^ITT ) divided by the regression-adjusted take-up rate (π).


Citations and Footnotes:
Engle, L., Katz, L., & Tebes, J. (2022). Pre-Analysis Plan for “Supporting Pathways Out of Poverty: A Randomized Evaluation of AMP Up Boston”
Chetty, R., S. DeLuca, L. F. Katz, and C. Palmer (2018). Creating moves to opportunity in seattle and king county randomized controlled trial pre-analysis plan.
This approach relies on the assumption that there was no average effect of being offered program enrollment on those who did not take up the program and that the control group was not affected by losing the lottery.

Not available

Subjects will be randomized using random number generation on Stata once a month. We will match enrolled participants based on earnings at baseline. For those without any earnings reported, we will match on reported household income. Random assignment will occur within these matched pairs, such that half are assigned to treatment and half to control.

Unit of randomization: Households. Program staff will designate one parent as the primary (potential) recipient and this parent will enter the lottery. Both parents will be eligible to receive support if the primary parent is randomized into the treatment group.

No

600 households

600 households

300 households in treatment, 300 households in control

The intended sample size is 600 households over 2 years of enrollment, with 300 households assigned to the treatment group (offered a spot in Elevating Families) and 300 assigned to the control group (not offered a spot in Elevating Families). All power calculations below assume a 70% take-up rate and that 20% of the outcome variance is explained by a lagged dependent variable (and other control variables). 1.) Assuming a mean household earnings of $8,664 among the control group, we are powered to detect a $1,996 increase in earnings among households offered the program (ITT), and $2,852 increase in earnings among those who take-up the program (TOT.) 2.) Assuming an employment rate of 50% among the control group, we are powered to detect a 10.2 percentage point increase in employment among those who are offered the program (ITT) and a 14.6 percentage point increase in employment among those who take-up the program (TOT). 3.) Assuming 20% of the control group households earning above $35,000 at baseline, we are powered to detect a 8.2 percentage point increase in the proportion of households that fall above this cutoff among households offered the program (ITT), and a 11.7 percentage point increase among those who take-up the program (TOT). 4.) Assuming that families, on average, have 1.5 school-aged children per household, we are powered to detect a 0.25 SD improvement in a standardized index of child outcomes among children living in households who take-up the program (TOT). Among children living in households offered the program (ITT), we are powered to detect a .175 SD improvement. Outcomes that will enter this index may include school engagement, academic performance, and CPS involvement. Citations: Data Interaction for Nevada Accountability Portal. (n.d.). Chronic Absenteeism. Nevada Department of Education. http://nevadareportcard.nv.gov/di/report/reportcard_1?report=reportcard_1&scope=e33.y19&organization=c17129&fields=309%2C310%2C311%2C313%2C318%2C320&hiddenfieldsid=309%2C310%2C311%2C313%2C318%2C320&scores=1456&num=160&page=1&pagesize=20&domain=chronicabsenteeism&