x

We are happy to announce that all trial registrations will now be issued DOIs (digital object identifiers). For more information, see here.
The Effects of Youth Letters of Recommendation
Last registered on March 01, 2019

Pre-Trial

Trial Information
General Information
Title
The Effects of Youth Letters of Recommendation
RCT ID
AEARCTR-0002771
Initial registration date
March 21, 2018
Last updated
March 01, 2019 2:22 PM EST
Location(s)

This section is unavailable to the public. Use the button below to request access to this information.

Request Information
Primary Investigator
Affiliation
Wharton School - University of Pennsylvania
Other Primary Investigator(s)
PI Affiliation
University of Michigan
Additional Trial Information
Status
On going
Start date
2016-09-01
End date
2022-09-01
Secondary IDs
Abstract
Summer Youth Employment Programs (SYEPs) are city-run programs that provide youth with paid work during the summer. These programs have been shown to significantly improve important youth outcomes including criminality, incarceration, and mortality. However, researchers have failed to find positive average effects on other important outcomes, such as future employment or college attendance. We aim to evaluate whether a minor program change, offering SYEP participants with personalized "letters of recommendation" (i.e., letters than can be shown to potential future employers, teachers, or guidance counselors), can improve outcomes for participating youth.
External Link(s)
Registration Citation
Citation
Heller, Sara and Judd Kessler. 2019. "The Effects of Youth Letters of Recommendation." AEA RCT Registry. March 01. https://doi.org/10.1257/rct.2771-2.0.
Former Citation
Heller, Sara, Judd Kessler and Judd Kessler. 2019. "The Effects of Youth Letters of Recommendation." AEA RCT Registry. March 01. https://www.socialscienceregistry.org/trials/2771/history/42280.
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details
Interventions
Intervention(s)
This trial studies the impact of providing personalized letters of recommendation to youth participants of the New York City Summer Youth Employment Program (SYEP). SYEP provides youth with up to six weeks of employment at public and private organizations throughout New York City. We survey summer supervisors about youth performance. Each youth in the treatment group who is rated highly enough by their supervisor is sent a letter with personalized information from their supervisor about the youth's strengths. Youth in the control group are excluded from receiving a letter of recommendation through our program but are not prevented from requesting letters of recommendation from their employers directly.
Intervention Start Date
2016-09-01
Intervention End Date
2018-09-01
Primary Outcomes
Primary Outcomes (end points)
Employment and earnings in the formal sector
Primary Outcomes (explanation)
Our main outcomes of interest are measures of success in the formal labor market: employment and earnings. We plan to measure these through whichever data source we can obtain permission to use, likely the NYS Department of Labor Unemployment Insurance records.
Secondary Outcomes
Secondary Outcomes (end points)
Pending data availability: college enrollment and attainment; high school enrollment, performance, and attainment; crime outcomes, such as arrests and convictions; job application and letter use rates among a subset of study youth
Secondary Outcomes (explanation)
Since letters may also be used in college applications, we aim to match youth to National Student Clearinghouse college matriculation records. Given that letters may also affect youths' beliefs about themselves and their labor market prospects, we may also seek NYC Department of Education records and NYS arrest/incarceration records, but our use of these outcomes depends on whether we find initial effects on our main outcomes and whether we are able to obtain the data.
We are also aiming to measure an intermediate outcome: how letter receipt changes youth job seeking and job application behavior. A subset of youth have been invited to apply to a job that we are offering, so we can measure application rates and the use of letters of recommendation in job applications.
Experimental Design
Experimental Design
The administrators of New York City's Summer Youth Employment Program (SYEP) select which summer employers (i.e., supervisors) will be part of the study. With the technical assistance of the research team, those supervisors will be sent a link to a survey instrument in which they will be shown the names of randomly selected youth who worked for them during the summer. After supervisors complete the survey, the research team will generate and send letters of recommendation to all youth who are rated highly enough by their supervisors to receive positive letters. We will subsequently link youth to existing administrative data on employment, and perhaps education and crime outcomes, in order to test whether the offer of a letter improves outcomes over a 3-year follow-up period. Data from a pilot of these measures conducted with 2016 SYEP participants will be combined with data from the full-scale trial of 2017 SYEP participants. Additionally, participants of the 2017 SYEP program will be invited to apply for a job, which will allow us to evaluate how the letter of recommendation affects the job application process directly.
Experimental Design Details
Not available
Randomization Method
Participants of the 2016 and 2017 NYC Summer Youth Employment Programs were randomly assigned to treatment or control groups. The research team conducted the lottery using Stata.
Randomization Unit
The randomization was conducted at the individual youth level.
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
No clustering, see planned number of observations response below.
Sample size: planned number of observations
70,000-80,000 individual youth who were SYEP participants, subject to data cleaning and de-duplication across study years. Additionally, in order to compare our study population to the broader SYEP applicant pool, we may collect data on the broader population of all youth who applied to SYEP in 2016 and 2017, even if they did not participate in SYEP. This broader group would be on the order of 200,000 individual youth. Additionally, we expect some non-compliance driven by factors such as whether supervisors received our survey recruitment emails and clicked on the survey invitation link. Whether a supervisor clicks on the survey invitation link should not differ between treatment and control youth on average, since survey recruitment emails give no indication of which youth were randomized to treatment or control. As such, we may separately analyze subsets where compliance is likely to be higher, but treatment is still random, such as youth who had a supervisor who clicked on a survey invitation link.
Sample size (or number of clusters) by treatment arms
There is only one treatment arm and one control arm. The treatment and control arms each have 35,000-40,000 individual SYEP participants, subject to data cleaning and de-duplication across study years. Additionally, in order to compare our study population to the broader SYEP applicant pool, we may collect data on the broader population of all youth who applied to SYEP in 2016 and 2017, even if they did not participate in SYEP.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
University of Pennsylvania Institutional Review Board
IRB Approval Date
2016-08-05
IRB Approval Number
825434
Analysis Plan

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information