Seeing is Believing? A Cognitive View of Program Take-up

Last registered on May 29, 2024


Trial Information

General Information

Seeing is Believing? A Cognitive View of Program Take-up
Initial registration date
May 21, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
May 29, 2024, 10:19 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.


There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

University of Notre Dame

Other Primary Investigator(s)

Additional Trial Information

On going
Start date
End date
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
The Excel Centers (TEC) are tuition-free charter public high schools that award high school diplomas to adults and provide child care, housing and transportation assistance, college credit, and industry-recognized certification courses to help students overcome barriers and continue their education. Through a randomized controlled trial, we will investigate the impact of two behavioral interventions on enrollment and persistence in TEC, with a possible follow-up study examining the impact of increased enrollment and graduation on long-term educational, financial, and labor market outcomes.

The first intervention involves a one-on-one orientation with a life coach before the start of classes. During these orientations, the coach first leads the applicant through a mindfulness breathing exercise before engaging in mental imagery. This involves visualizing their graduation and dream job, as well as writing a letter to their future selves on the day of their graduation. Such exercises have proven effective in other contexts, especially among populations with a history of trauma (Ashraf et al., 2022). The goal of this intervention is to alter “what comes to mind” for students (Gennaioli and Shleifer, 2010): where students previously might think of past difficulties in school, now they may think of their coach and their visualized future (e.g. career or educational goals). By also facilitating a close bond with coaches, this intervention could cascade to increased student engagement and persistence in the program. The second intervention is an advertising video that seeks to reduce perceived barriers of program attendance: depending on the recipient’s age, it features either a single mother or middle-aged man navigating barriers to success at TEC to eventually earn their diplomas.

The research team will enroll 5,000 individuals into the study over 18 months, with enrollees being randomly assigned into one of four groups: (i) being offered only the video, (ii) being offered only the one-on-one orientation, (iii) being offered both the video and the orientation, and (iv) being offered only TEC’s standard services, which do not include either the video or the one-on-one orientation.


Ashraf, N., Bryan, G., Delfino, A., Holmes, E., Iacovone, L., & Pople, A. (2022). Learning to see the world’s opportunities: The impact of mental imagery on entrepreneurial action. (Working Paper). Innovations for Poverty Action.

Gennaioli, Nicola, and Andrei Shleifer. "What comes to mind." The Quarterly journal of economics 125, no. 4 (2010): 1399-1433.
External Link(s)

Registration Citation

Tebes, Jonathan. 2024. "Seeing is Believing? A Cognitive View of Program Take-up." AEA RCT Registry. May 29.
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details


Historically, following registration, TEC invited participants to a group orientation. Given the group setting, these orientations allowed little flexibility in scheduling and few opportunities for participants’ individual concerns to be addressed. As part of the study, we will randomize students to attend either the traditional group orientation or a one-on-one orientation with their life coach. Life coaches are dedicated Excel Center staff that students can turn to should they need counseling or resources to address both academic and non-academic barriers. Importantly, coaches often live in the communities they serve and closely reflect the racial and socio-economic backgrounds of their students. During these orientations, students and their life coaches will review the student’s high school records, availability for classes, employment status, and childcare needs. Through a barriers assessment, the life coach works with the student to create a plan that addresses any factors that could impede their ability to attend classes, such as housing instability, legal/justice issues, mental health concerns, or transportation issues. Small amounts of financial assistance are available to address these barriers.

After the barriers assessment, applicants are guided through a breathing-based mindfulness exercise that encourages them to “slow down” and meaningfully engage with the two behavioral exercises that follow (Khaneman, 2011). The first behavioral exercise is a mental imagery exercise where applicants visualize their graduation from TEC and starting their dream job. The second exercise builds on practices that TEC has previously found effective: students write a letter to their future self to open on the day of their graduation. Finally, coaches end the orientation with a positive affirmation that acknowledges the student’s courage to make this commitment to their education and expresses the coach’s belief in them. This intervention builds on promising behavioral interventions in psychology and behavioral economics. Exercises like visualization, mindfulness, and writing a letter to one’s future self have been found to impact education-related and economic outcomes in a wide range of settings. In our context, past negative experiences in school may similarly generate psychological barriers that reduce engagement in adult educational programs. While extensive prior work in psychology has documented that positive mental imagery and letter-to-future-self exercises are associated with improved goal-setting and academic effort (Oyserman et al., 2006; Blouin-Hudon et al., 2017; Yuta et al., 2021), these interventions have yet to be rigorously evaluated as the means for improving enrollment and persistence in educational services.

Soon after application, applicants are also randomized into an intervention group that either does or does not receive an email and a text message that includes one of two “de-stigmatizing” videos. The first video, which is sent if an applicant is below the age of 25, accompanies a young mother as she attends an Excel Center, overcoming her doubts about her place in the program and eventually earning her diploma. The second video, which is sent to those over 25, features a middle-aged man who overcomes his stigma about age to attend classes, graduate, and attain his dream job. These videos were designed by TEC’s marketing team and reflect the two most common student profiles.


Blouin-Hudon, E. M. C. & Pychyl, T. A. (2017). A Mental Imagery Intervention to Increase Future Self-Continuity and Reduce Procrastination. Applied Psychology: An International Review, 66(2), 326–352.
Khaneman, Daniel. Thinking, fast and slow. Macmillan, 2011.

Oyserman, D., Bybee, D., & Terry, K. (2006). Possible selves and academic outcomes: How and when possible selves impel action. Journal of Personality and Social Psychology, 91(1), 188–204.

Yuta, C. & Wilson, A. (2021). Conversation with a future self: A letter-exchange exercise enhances student self-continuity, career planning, and academic thinking. Self and Identity, 20(5), 646-671.
Intervention Start Date
Intervention End Date

Primary Outcomes

Primary Outcomes (end points)
Our primary outcomes at the applicant-level will be (individual outcomes listed on each line):

Indicator for attending an orientation within one year after application
Indicator for being enrolled in a class at TEC within one year after application
Indicator for earning any credit from the TEC within one year after application
Indicator for the receipt of a high school diploma from TEC within one year after application
Primary Outcomes (explanation)
The TEC academic year consists of five 8-week terms. Term of application refers to the nearest term after an application. “Within one year after application” refers to the five academic terms after an applicant’s application date. We will primarily focus on the following time horizons: the term of application, one year post-application, and two year post-application (provided sufficient coverage). However, to understand dynamics, we will also report outcomes for each term after application, up to three years post-application (provided sufficient coverage). Additionally, we will mark a student as having completed enrollment in a given term if they have an enrollment spell that overlaps with the start and end dates of a term.

Secondary Outcomes

Secondary Outcomes (end points)
Our secondary take-up outcomes at the applicant-level will be (first line of each grouping is the category, followed by specific outcomes):

Student interactions with life coaches (grouping):
Cumulative count of separate interactions with life coach (including orientation)
Cumulative count of separate calls or text conversations
Cumulative count of in-person meetings

Enrollment at TEC, such as (grouping):
Cumulative count of terms enrolled in classes at TEC
Cumulative count of terms with any credits earned

Class attendance at TEC, such as (grouping):
Indicator for first-day attendance in at least one of the applicant’s enrolled courses in the term of application
Proportion of class days marked as attended out of total number of enrolled class days
Cumulative count of total days marked as attended across all terms

Metrics for academic performance at TEC, such as (grouping):
Grade point average
Cumulative count of credits earned
Cumulative count of math credits earned at TEC
Cumulative count of English credits earned at TEC
Indicator for received any credit towards post-secondary degree or certificate

Usage of non-academic services from TEC, such as (grouping):
Indicator for usage of any non-academic services from TEC
Indicator for usage of TEC’s housing services
Indicator for usage of TEC’s mental health services
Indicator for usage of TEC’s childcare services
Indicator for usage of TEC’s transportation services
Indicator for usage of other TEC services
Estimated cost of all TEC services used

As a potential follow-up study, we would also like to investigate the effects of the behavioral interventions on a set of long-term outcomes. Depending on data availability, these outcomes could include (structured as above, but with groupings and sub-groupings):

Labor market outcomes (grouping):
Total quarterly earnings
Indicator for employment
Count of quarters employed
Length of applicant’s longest spell of employment

Education and professional training outcomes (grouping):

Progress towards and receipt of a professional certificate (sub-grouping):
Indicator for receipt of a professional certificate in a given field
Count of credits towards a professional certificate

Post-secondary institution attendance and degree receipt (sub-grouping):
Indicator for receiving a degree in a given field
Indicator for receipt of credits toward a degree in a given field
Count of credits towards a degree in a given field
Indicator for attendance at a 2 year institution
Indicator for attendance at a 4 year institution
Indicator for graduation from a 2 year institution
Indicator for graduation from a 4 year institution

Utilization of Public Benefits (grouping):
Indicator for receiving public housing support (housing vouchers, rental assistance, etc.)
Expenditures received related to housing (housing vouchers, rental assistance, etc.)
Indicator for enrollment in Supplemental Nutritional Assistance Program (SNAP)
Expenditures received from SNAP
Indicator for enrollment in Temporary Assistance for Needy Families (TANF)
Expenditures received from TANF

Credit and delinquency outcomes (grouping):

Credit Index (sub-grouping):
Credit score estimated using VantageScore
Total available credit on all accounts
Total balance on all accounts

Delinquency Index (sub-grouping):
Amount past due on trades presently 30 days delinquent reported in the last 6 months
Amount of debt past due held by third-party collection agencies
Total number of public record bankruptcies and tax liens
Total number of trades presently satisfactory that were ever 30 or more days delinquent or derogatory excluding collections

Criminal justice involvement (grouping):
Indicator for any criminal charge
Indicator for any felony charge
Indicator for any misdemeanor charge
Indicator for any violation charge
Cumulative count of criminal charges
Cumulative count of felony charges
Cumulative count of misdemeanor charges
Cumulative count of violation charges
Secondary Outcomes (explanation)
Similar to our primary outcomes, we will focus on TEC outcomes measured in the term of application, one year post-application, and two years post-application (provided sufficient coverage). However, we will also report outcomes for each academic term post-application in figures to illustrate engagement dynamics.

For the follow-up study on long-term outcomes, we will report outcomes in an event-study framework for each year post-application relative to the year prior to application. Where possible (e.g. quarterly earnings and employment), we will report outcomes at the quarterly level.

GCSC tracks and has provided numerous categorizations for student-life coach interactions. After we receive and clean pilot data, we will pre-specify more granular variables that better capture various dimensions of student-coach bonds.

The construction of labor market variables will follow that of Brough et al. (forthcoming); the measure of total quarterly earnings will be inflation-adjusted to 2024 using the CPI-U Midwest. It will also be winsorized at the 99th percentile, with missing values taking on the implicit zero value.

Employment sectors will be categorized based on their NAICS codes; the construction of the categories will follow that section B.1.1 of the appendix to Brough et al. (forthcoming).

Professional certificate and degree categories, as well as credits towards them, will follow that of tables B-1 and B-2 of the appendix to Brough et al. (forthcoming).

For the criminal justice outcomes, the data will come from the Indiana Office of Courts Services and Marion County Sheriff’s Office.

All standardized indices will be computed using the control group mean and standard deviation as described in Kling et al. (2007). Exact variables included in these indices may be subject to change based on what data are available.


Brough, R., Phillips, D. C., & Turner P. S. (Forthcoming). High Schools Tailored to Adults Can Help Them Complete a Traditional Diploma and Excel in the Labor Market. American Economic Journal: Economic Policy.

Kling, J. R., J. B. Liebman, and L. F. Katz (2007). Experimental analysis of neighborhood effects. Econometrica 75 (1), 83–119.

Experimental Design

Experimental Design
In order to be eligible for the study, applicants must be an Indiana resident, not have previously earned their high school diploma, not be a sex offender, and be at least 18 years of age. With the exception of the age restriction, these eligibility criteria exactly follow those of TEC.

Early evidence from a pilot study of these interventions indicates that English language learners may react differently to the behavioral interventions than their peers: life coaches have stated that this group is largely seeking English classes, rather than a high school diploma. As such, it may be the case that the intervention is not a good fit for these students, since they are not facing the same barriers that the intervention seeks to address. Therefore, the research team plans to (1) study English language learners as a sub-population, and (2) based on data from the initial pilot decide with the GW team whether to include this population in the roll-out of the intervention to all ten schools.

The study process begins with potential students filling out an online application form. As part of this form, they are presented with a shortened consent form and given the option of opting out of research participation. Eligible applicants are then randomized with ¼ probability into four groups: a pure control group that receives standard services (i.e., a group orientation and no video message), a group that receives only the video intervention, a group that receives only the one-on-one orientation, and a group that receives both the video and the one-on-one orientation interventions . Additionally, we will randomize all past and existing students of TEC to one of these four groups at the start of the study; this randomization will be stratified based on whether or not their primary language is English and if they have previously enrolled and not just applied for a term at TEC. If these students return to TEC after time away, they will be offered the take-up interventions based on this initial randomization.

We expect nearly full compliance for the video intervention, as it will be texted to the cell phone provided in the application. However, there may not be full compliance for the one-on-one orientation: some individuals may repeatedly miss their scheduled one-on-one orientation dates, in which case they are reverted to the group orientation.

For the analysis, our primary specification estimates the impact of the offer of each intervention, or intention-to-treat (ITT) effects, on outcomes. Since the offer is randomly assigned, any differences in outcomes between groups can be attributed to the intervention offer. The basic specification is (we encase subscripts in parentheses for clarity):

y(ist)= β(1)*Video(it) + β(2)*Orientation(it) + τ(st) + γX(it) + ε(ist)

Where y(ist) is one of our primary take-up outcomes (e.g., TEC enrollment or graduation) for applicant i in application term t, where s denotes the applicant’s randomization strata. Video(it) indicates whether or not participant i was sent the video intervention by term t. Similarly, Orientation(it) indicates whether participant i had been assigned to the one-on-one orientation by term t. τ(st) are strata-time fixed effects, which will include a set of application term indicators and two indicators used in the stratified randomization of prior applicants. Lastly, X(it) is a vector of control variables, such as demographics (age, race, gender) and pre-study variables captured in the application form or historical program data. The coefficients of interest – β(1) and β(2) – respectively estimate the average difference in outcomes between each intervention group (video or one-on-one orientation) and the control group, controlling for baseline characteristics.

Using historical (out-of-sample) data, we will predict a given applicant’s likelihood of enrolling, persisting, and graduating from the program. We anticipate that students who enroll in remedial classes and students who are first-time applicants will be less likely to enroll, persist, and graduate from TEC. We will explore heterogeneity by predicted persistence as well as by key predictors of persistence (e.g. enrolled in remedial classes, English Language Learners, etc.) to understand how the program interacts with the barriers applicants face at baseline. Does the program nudge those on the margin of persisting at baseline? And/or does it facilitate persistence among those least likely to persist at baseline?

Given that our main outcomes relate to take-up, we will also report differences in average characteristics of those who enrolled, persisted or graduated across Treatment and Control groups using a LATE framework (Angrist and Imbens, 1996). This approach instruments for program enrollment, persistence (e.g. any receipt of credit), or program graduation with treatment status, and the dependent variable is a characteristic. Characteristics may include race, age, gender, previous applicant status, predicted academic level upon entry, or English Language Learner status. We will pre-specify the characteristics to be included based on pilot data and will adjust for multiple hypothesis testing using randomization inference. This will provide rich information on whether the interventions differentially benefit those who face the most significant barriers.

We will also explore heterogeneity by remedial class status and predicted program completion. GCSC has indicated that students who enroll in remedial courses, particularly for English and math, are at a higher risk of exiting from TEC due to the longer time to completion for diplomas. Therefore, strong treatment effects from the one-on-one intervention may be indicative of its ability to reframe the importance of a high school diploma for this population. Additionally, if English Language Learners are included in the main analysis, we will explore effects for this sub-sample of students, as they face unique barriers relative to the full sample.

Finally, as part of our analysis, we would like to explore treatment effect heterogeneity using machine learning methods, such as LASSO (Chernozhukov et al., 2018; Davis & Heller, 2017). These will allow us to ascertain which sub-populations in the study were most affected by the treatment. The benefit of this approach is that it does not require the researchers to know a priori which characteristics or interactions of characteristics are related to underlying heterogeneity; the algorithm identifies these characteristics while penalizing over-fitting.


Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association 91(434), 444-455.

Chernozhukov, V., Demirer, M., Duflo, E., & Fernandez-Val, I. (2018). Generic machine learning inference on heterogeneous treatment effects in randomized experiments, with an application to immunization in India. (Working Paper No. w24678). National Bureau of Economic Research.

Davis, J. & Heller, S. B. (2017). Using causal forests to predict treatment heterogeneity: An application to summer jobs. American Economic Review 107(5), 546–50.
Experimental Design Details
Not available
Randomization Method
New TEC applicants will be randomly assigned into a treatment group using random number generation in Goodwill Indy’s student information system. Previous applicants who are returning to TEC will be randomized via STATA using a stratified design: the strata are based on whether the applicant’s primary language is English and whether the applicant had previously completed a term at TEC.
Randomization Unit
Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters
N/A: the treatment is not clustered
Sample size: planned number of observations
5,000 applicants
Sample size (or number of clusters) by treatment arms
1,250 receive only the video
1,250 receive only the one-on-one orientation
1,250 receive both the video and the one-on-one orientation
1,250 receive standard services (neither the video nor the one-on-one orientation)
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
The intended sample size is 5,000 individuals enrolled over 18 months. About 1,250 people will be assigned to each of the four study groups Our primary analysis will pool together the groups who received each intervention, for example, by comparing all those who were offered an individual orientation to all those who were offered a group orientation. These comparisons will include our full sample of approximately 5,000 people. Given this sample size and conventional power assumptions, we estimate the minimum detectable difference in outcomes between either intervention group and the control group. We are powered to detect a 3.76 percentage point increase in orientation attendance (control mean = 50%), a 3.68 percentage point increase in attending the first day of class (control mean = 40%), and a 2.3 percentage point increase in graduation (control mean = 10%). To gauge statistical power for long-term outcomes, which may be the focus of a follow-up paper, we explore various effect sizes on TEC enrollment. If the interventions collectively lead to a 30% increase in enrollment, we are powered to detect a 6.9 percentage point increase in receipt of a HS diploma among those who are induced to enroll (but otherwise would not have). Likewise, we are powered to detect a $1,419 or 27% improvement in earnings. If the interventions collectively lead to a 20% increase in TEC enrollment, we are powered to detect a 10.3 percentage point increase in receipt of a HS diploma and a $2,127 or 41% increase in earnings among those who additionally enroll. That is, if the behavioral interventions meaningfully increase enrollment, we are powered to detect small changes in receipt of a HS diploma and earnings gains of similar size to those observed in Brough et al. (forthcoming).

Institutional Review Boards (IRBs)

IRB Name
The University of Notre Dame Institutional Review Board
IRB Approval Date
IRB Approval Number