|
Field
Abstract
|
Before
The Excel Centers (TEC) are tuition-free charter public high schools that award high school diplomas to adults and provide child care, housing and transportation assistance, college credit, and industry-recognized certification courses to help students overcome barriers and continue their education. Through a randomized controlled trial, we will investigate the impact of two behavioral interventions on enrollment and persistence in TEC, with a possible follow-up study examining the impact of increased enrollment and graduation on long-term educational, financial, and labor market outcomes.
The first intervention involves a one-on-one orientation with a life coach before the start of classes. During these orientations, the coach first leads the applicant through a mindfulness breathing exercise before engaging in mental imagery. This involves visualizing their graduation and dream job, as well as writing a letter to their future selves on the day of their graduation. Such exercises have proven effective in other contexts, especially among populations with a history of trauma (Ashraf et al., 2022). The goal of this intervention is to alter “what comes to mind” for students (Gennaioli and Shleifer, 2010): where students previously might think of past difficulties in school, now they may think of their coach and their visualized future (e.g. career or educational goals). By also facilitating a close bond with coaches, this intervention could cascade to increased student engagement and persistence in the program. The second intervention is an advertising video that seeks to reduce perceived barriers of program attendance: depending on the recipient’s age, it features either a single mother or middle-aged man navigating barriers to success at TEC to eventually earn their diplomas.
The research team will enroll 5,000 individuals into the study over 18 months, with enrollees being randomly assigned into one of four groups: (i) being offered only the video, (ii) being offered only the one-on-one orientation, (iii) being offered both the video and the orientation, and (iv) being offered only TEC’s standard services, which do not include either the video or the one-on-one orientation.
Citations:
Ashraf, N., Bryan, G., Delfino, A., Holmes, E., Iacovone, L., & Pople, A. (2022). Learning to see the world’s opportunities: The impact of mental imagery on entrepreneurial action. (Working Paper). Innovations for Poverty Action. https://poverty-action.org/sites/default/files/2023-07/learning-to-see-a-world-of-opportunities-working-paper-1.pdf
Gennaioli, Nicola, and Andrei Shleifer. "What comes to mind." The Quarterly journal of economics 125, no. 4 (2010): 1399-1433.
|
After
The Excel Centers (TEC) are tuition-free charter public high schools that award high school diplomas to adults and provide child care, housing and transportation assistance, college credit, and industry-recognized certification courses to help students overcome barriers and continue their education. Through a randomized controlled trial, we will investigate the impact of two behavioral interventions on enrollment and persistence in TEC, with a possible follow-up study examining the impact of increased enrollment and graduation on long-term educational, financial, and labor market outcomes.
The first intervention involves a one-on-one orientation with a life coach before the start of classes. During these orientations, the coach first leads the applicant through a mindfulness breathing exercise before engaging in mental imagery. This involves visualizing their graduation and dream job, as well as writing a letter to their future selves on the day of their graduation. Such exercises have proven effective in other contexts, especially among populations with a history of trauma (Ashraf et al., 2022). The goal of this intervention is to alter “what comes to mind” for students (Gennaioli and Shleifer, 2010): where students previously might think of past difficulties in school, now they may think of their coach and their visualized future (e.g. career or educational goals). By also facilitating a close bond with coaches, this intervention could cascade to increased student engagement and persistence in the program. The second intervention is an advertising video that seeks to reduce perceived barriers of program attendance: depending on the recipient’s age, it features either a single mother or middle-aged man navigating barriers to success at TEC to eventually earn their diplomas.
The research team will enroll 5,000 individuals into the study over 18 months, with enrollees being randomly assigned into one of four groups: (i) being offered only the video, (ii) being offered only the one-on-one orientation, (iii) being offered both the video and the orientation, and (iv) being offered only TEC’s standard services, which do not include either the video or the one-on-one orientation.
Amendment 12/10/2025:
After observing preliminary results from the video intervention, the research team, in collaboration with TEC, has decided to randomize a new set of informational messages. Once their applications are approved, consenting applicants to TEC will be randomized into one of four messaging groups: the first group is a pure control group, which will not receive any messages; the second will receive a standard welcome message; the third group will receive a message that draws on Brough et al. (2024) to provide information on the average earnings effects of the program five years after application; the fourth group will receive information about additional support services available to enrolled students, such as transportation assistance and childcare. 40% of the prospective sample will be assigned to the pure control group and 20% of the sample will be assigned to each of the other three messaging interventions. These messaging interventions will be implemented for roughly 3-9 months to build up a sufficient sample size to evaluate their effectiveness. Then, the research team will present information on orientation participation rates to inform the refinement of future messages and/or the development of new messages.
Citations:
Ashraf, N., Bryan, G., Delfino, A., Holmes, E., Iacovone, L., & Pople, A. (2022). Learning to see the world’s opportunities: The impact of mental imagery on entrepreneurial action. (Working Paper). Innovations for Poverty Action. https://poverty-action.org/sites/default/files/2023-07/learning-to-see-a-world-of-opportunities-working-paper-1.pdf
Gennaioli, Nicola, and Andrei Shleifer. "What comes to mind." The Quarterly journal of economics 125, no. 4 (2010): 1399-1433.
|
|
Field
Last Published
|
Before
May 29, 2024 10:19 AM
|
After
December 10, 2025 02:03 PM
|
|
Field
Intervention (Public)
|
Before
Historically, following registration, TEC invited participants to a group orientation. Given the group setting, these orientations allowed little flexibility in scheduling and few opportunities for participants’ individual concerns to be addressed. As part of the study, we will randomize students to attend either the traditional group orientation or a one-on-one orientation with their life coach. Life coaches are dedicated Excel Center staff that students can turn to should they need counseling or resources to address both academic and non-academic barriers. Importantly, coaches often live in the communities they serve and closely reflect the racial and socio-economic backgrounds of their students. During these orientations, students and their life coaches will review the student’s high school records, availability for classes, employment status, and childcare needs. Through a barriers assessment, the life coach works with the student to create a plan that addresses any factors that could impede their ability to attend classes, such as housing instability, legal/justice issues, mental health concerns, or transportation issues. Small amounts of financial assistance are available to address these barriers.
After the barriers assessment, applicants are guided through a breathing-based mindfulness exercise that encourages them to “slow down” and meaningfully engage with the two behavioral exercises that follow (Khaneman, 2011). The first behavioral exercise is a mental imagery exercise where applicants visualize their graduation from TEC and starting their dream job. The second exercise builds on practices that TEC has previously found effective: students write a letter to their future self to open on the day of their graduation. Finally, coaches end the orientation with a positive affirmation that acknowledges the student’s courage to make this commitment to their education and expresses the coach’s belief in them. This intervention builds on promising behavioral interventions in psychology and behavioral economics. Exercises like visualization, mindfulness, and writing a letter to one’s future self have been found to impact education-related and economic outcomes in a wide range of settings. In our context, past negative experiences in school may similarly generate psychological barriers that reduce engagement in adult educational programs. While extensive prior work in psychology has documented that positive mental imagery and letter-to-future-self exercises are associated with improved goal-setting and academic effort (Oyserman et al., 2006; Blouin-Hudon et al., 2017; Yuta et al., 2021), these interventions have yet to be rigorously evaluated as the means for improving enrollment and persistence in educational services.
Soon after application, applicants are also randomized into an intervention group that either does or does not receive an email and a text message that includes one of two “de-stigmatizing” videos. The first video, which is sent if an applicant is below the age of 25, accompanies a young mother as she attends an Excel Center, overcoming her doubts about her place in the program and eventually earning her diploma. The second video, which is sent to those over 25, features a middle-aged man who overcomes his stigma about age to attend classes, graduate, and attain his dream job. These videos were designed by TEC’s marketing team and reflect the two most common student profiles.
Citations:
Blouin-Hudon, E. M. C. & Pychyl, T. A. (2017). A Mental Imagery Intervention to Increase Future Self-Continuity and Reduce Procrastination. Applied Psychology: An International Review, 66(2), 326–352. https://iaap-journals.onlinelibrary.wiley.com/doi/epdf/10.1111/apps.12088
Khaneman, Daniel. Thinking, fast and slow. Macmillan, 2011.
Oyserman, D., Bybee, D., & Terry, K. (2006). Possible selves and academic outcomes: How and when possible selves impel action. Journal of Personality and Social Psychology, 91(1), 188–204. https://doi.org/10.1037/0022-3514.91.1.188
Yuta, C. & Wilson, A. (2021). Conversation with a future self: A letter-exchange exercise enhances student self-continuity, career planning, and academic thinking. Self and Identity, 20(5), 646-671. https://www.tandfonline.com/doi/full/10.1080/15298868.2020.1754283
|
After
Historically, following registration, TEC invited participants to a group orientation. Given the group setting, these orientations allowed little flexibility in scheduling and few opportunities for participants’ individual concerns to be addressed. As part of the study, we will randomize students to attend either the traditional group orientation or a one-on-one orientation with their life coach. Life coaches are dedicated Excel Center staff that students can turn to should they need counseling or resources to address both academic and non-academic barriers. Importantly, coaches often live in the communities they serve and closely reflect the racial and socio-economic backgrounds of their students. During these orientations, students and their life coaches will review the student’s high school records, availability for classes, employment status, and childcare needs. Through a barriers assessment, the life coach works with the student to create a plan that addresses any factors that could impede their ability to attend classes, such as housing instability, legal/justice issues, mental health concerns, or transportation issues. Small amounts of financial assistance are available to address these barriers.
After the barriers assessment, applicants are guided through a breathing-based mindfulness exercise that encourages them to “slow down” and meaningfully engage with the two behavioral exercises that follow (Khaneman, 2011). The first behavioral exercise is a mental imagery exercise where applicants visualize their graduation from TEC and starting their dream job. The second exercise builds on practices that TEC has previously found effective: students write a letter to their future self to open on the day of their graduation. Finally, coaches end the orientation with a positive affirmation that acknowledges the student’s courage to make this commitment to their education and expresses the coach’s belief in them. This intervention builds on promising behavioral interventions in psychology and behavioral economics. Exercises like visualization, mindfulness, and writing a letter to one’s future self have been found to impact education-related and economic outcomes in a wide range of settings. In our context, past negative experiences in school may similarly generate psychological barriers that reduce engagement in adult educational programs. While extensive prior work in psychology has documented that positive mental imagery and letter-to-future-self exercises are associated with improved goal-setting and academic effort (Oyserman et al., 2006; Blouin-Hudon et al., 2017; Yuta et al., 2021), these interventions have yet to be rigorously evaluated as the means for improving enrollment and persistence in educational services.
Soon after application, applicants are also randomized into an intervention group that either does or does not receive an email and a text message that includes one of two “de-stigmatizing” videos. The first video, which is sent if an applicant is below the age of 25, accompanies a young mother as she attends an Excel Center, overcoming her doubts about her place in the program and eventually earning her diploma. The second video, which is sent to those over 25, features a middle-aged man who overcomes his stigma about age to attend classes, graduate, and attain his dream job. These videos were designed by TEC’s marketing team and reflect the two most common student profiles.
Amendment 12/10/2025: Informational Interventions
Preliminary analyses of the data from the video intervention indicate that study applicants are sensitive to the tone and content of information provided by TEC after their application. The video intervention was found to reduce orientation attendance and course enrollment. Since the videos described how past participants were able to overcome significant personal barriers to graduate TEC, they may have unintentionally primed applicants to consider the barriers they face and discouraged them from enrolling in TEC.
Contingent on consent to participate in research, applicants to TEC will now be randomized into one of three text message streams with 20% probability each: research-based, logistics-based, and placebo. The remaining 40% of applications will be randomized into the control group, which will not receive the above messages.
The research team will monitor the effect of each messaging intervention on short-term outcomes at TEC, such as orientation attendance, enrollment, course completion, and graduation. As the enrollment numbers for these interventions increase and their impact is determined, the research team will remove the less effective messages, increasing the treatment probability of the remaining messages, or introducing new messages that build on the findings. All new messages and changes in the randomization probability will be integrated into this registry entry. The text of the current messages is given below:
(20%) T1 (Research): “Welcome to the Excel Center community! Did you know that the average Excel Center graduate sees their earnings increase by 40% in just five years-and many see even bigger gains? We'll be in touch shortly to schedule your orientation.”
(20%) T2 (Logistics): “Welcome to the Excel Center community! Did you know that the Excel Center provides childcare, transportation and other assistance for orientation and beyond? We will be in touch shortly to schedule your orientation.”
(20%) Control (Placebo): “Welcome to the Excel Center community! We're excited to help you begin your journey toward earning your diploma. We will be in touch shortly to schedule your orientation and share more details about what to expect.”
(40%) Pure control: no message
Citations:
Blouin-Hudon, E. M. C. & Pychyl, T. A. (2017). A Mental Imagery Intervention to Increase Future Self-Continuity and Reduce Procrastination. Applied Psychology: An International Review, 66(2), 326–352. https://iaap-journals.onlinelibrary.wiley.com/doi/epdf/10.1111/apps.12088
Khaneman, Daniel. Thinking, fast and slow. Macmillan, 2011.
Oyserman, D., Bybee, D., & Terry, K. (2006). Possible selves and academic outcomes: How and when possible selves impel action. Journal of Personality and Social Psychology, 91(1), 188–204. https://doi.org/10.1037/0022-3514.91.1.188
Yuta, C. & Wilson, A. (2021). Conversation with a future self: A letter-exchange exercise enhances student self-continuity, career planning, and academic thinking. Self and Identity, 20(5), 646-671. https://www.tandfonline.com/doi/full/10.1080/15298868.2020.1754283
|
|
Field
Planned Number of Observations
|
Before
5,000 applicants
|
After
5,000 applicants
|
|
Field
Sample size (or number of clusters) by treatment arms
|
Before
1,250 receive only the video
1,250 receive only the one-on-one orientation
1,250 receive both the video and the one-on-one orientation
1,250 receive standard services (neither the video nor the one-on-one orientation)
|
After
1,250 receive only the video
1,250 receive only the one-on-one orientation
1,250 receive both the video and the one-on-one orientation
1,250 receive standard services (neither the video nor the one-on-one orientation)
Amendment 12/10/2025:
We will enroll TEC applicants into the text message streams for roughly 3-9 months to build up a sufficient sample size to evaluate their effectiveness. Then, the research team will present information on orientation participation rates to inform the refinement of future messages and/or the development of new messages.
|
|
Field
Power calculation: Minimum Detectable Effect Size for Main Outcomes
|
Before
The intended sample size is 5,000 individuals enrolled over 18 months. About 1,250 people will be assigned to each of the four study groups
Our primary analysis will pool together the groups who received each intervention, for example, by comparing all those who were offered an individual orientation to all those who were offered a group orientation. These comparisons will include our full sample of approximately 5,000 people. Given this sample size and conventional power assumptions, we estimate the minimum detectable difference in outcomes between either intervention group and the control group. We are powered to detect a 3.76 percentage point increase in orientation attendance (control mean = 50%), a 3.68 percentage point increase in attending the first day of class (control mean = 40%), and a 2.3 percentage point increase in graduation (control mean = 10%).
To gauge statistical power for long-term outcomes, which may be the focus of a follow-up paper, we explore various effect sizes on TEC enrollment. If the interventions collectively lead to a 30% increase in enrollment, we are powered to detect a 6.9 percentage point increase in receipt of a HS diploma among those who are induced to enroll (but otherwise would not have). Likewise, we are powered to detect a $1,419 or 27% improvement in earnings. If the interventions collectively lead to a 20% increase in TEC enrollment, we are powered to detect a 10.3 percentage point increase in receipt of a HS diploma and a $2,127 or 41% increase in earnings among those who additionally enroll. That is, if the behavioral interventions meaningfully increase enrollment, we are powered to detect small changes in receipt of a HS diploma and earnings gains of similar size to those observed in Brough et al. (forthcoming).
|
After
The intended sample size is 5,000 individuals enrolled over 18 months. About 1,250 people will be assigned to each of the four study groups
Our primary analysis will pool together the groups who received each intervention, for example, by comparing all those who were offered an individual orientation to all those who were offered a group orientation. These comparisons will include our full sample of approximately 5,000 people. Given this sample size and conventional power assumptions, we estimate the minimum detectable difference in outcomes between either intervention group and the control group. We are powered to detect a 3.76 percentage point increase in orientation attendance (control mean = 50%), a 3.68 percentage point increase in attending the first day of class (control mean = 40%), and a 2.3 percentage point increase in graduation (control mean = 10%).
To gauge statistical power for long-term outcomes, which may be the focus of a follow-up paper, we explore various effect sizes on TEC enrollment. If the interventions collectively lead to a 30% increase in enrollment, we are powered to detect a 6.9 percentage point increase in receipt of a HS diploma among those who are induced to enroll (but otherwise would not have). Likewise, we are powered to detect a $1,419 or 27% improvement in earnings. If the interventions collectively lead to a 20% increase in TEC enrollment, we are powered to detect a 10.3 percentage point increase in receipt of a HS diploma and a $2,127 or 41% increase in earnings among those who additionally enroll. That is, if the behavioral interventions meaningfully increase enrollment, we are powered to detect small changes in receipt of a HS diploma and earnings gains of similar size to those observed in Brough et al. (forthcoming).
Amendment 12/10/2025:
Our primary analysis will pool together the groups randomized into the various treatment arms (video, orientation, and messages). These comparisons will include a sample of approximately 22,000 people. Given this sample size and conventional power assumptions, we estimate the minimum detectable difference in outcomes between either intervention group and the control group. We are powered to detect a 1.75 percentage point increase in orientation attendance (control mean = 35%), a 1.70 percentage point increase in course enrollment (control mean = 31%), and a 1.57 percentage point increase in course enrollment (control mean = 24%). These means come from preliminary data on short term outcomes that was provided by TEC.
To gauge statistical power for long-term outcomes, we explore various effect sizes on TEC graduation. If the interventions collectively lead to an 8 percentage point increase in graduation, we are powered to detect a $2,754 or 53% improvement in earnings, assuming a $5,200 control mean. Under the same conditions and assuming a 40% employment rate among the control group, we are powered to detect a 22.49 percentage point increase in employment (56% increase over control mean) in employment.
If the interventions instead increase graduation by 12 percentage points, then we will be powered to detect the effects found in Brough et al. (2024). Specifically, under the previously given sample size (22,000 participants) and control means ($5,200 for earnings and 40% employment rate), we will be able to detect earnings effects greater than $1,836 (35% increase over control mean) and employment effects greater than 15 percentage points (37% increase over control mean).
In the future, the research team will also investigate additional instruments in the late design, such as the time gap between study participants' date of application and their potential start date for classes.
|