Impact Evaluation of Girls Inc.’s EmpowerHub

The EmpowerHub program is delivered by trained program coordinators and is broken into three sections – Strong (health and wellness), Smart (academic readiness), Bold (civic responsibility and leadership). The program is designed to help deliver consistent programming across each program site. EmpowerHub's strategy encompasses a multifaceted approach that includes reinforcing academic behaviors, fostering perseverance, cultivating positive mindsets, enhancing learning strategies, and developing social skills, all integral to shaping well-rounded individuals and re-engaging students in their future.
The Girls Inc.’s Experience involves the interplay of people, environment, and programming to create a holistic developmental space to address these areas. EmpowerHub’s comprehensive strategy also utilizes programming to emphasize developing non-cognitive skills affecting academic pursuits. This program focuses on non-cognitive factors such as academic behaviors, mindsets, perseverance, learning strategies, and social skills. EmpowerHub’s curricular interventions are strategically designed to develop non-cognitive skills shown by research to be critical in enhancing students' cognitive and emotional realms, thereby improving academic outcomes.
Programmatic components include mentorship from program coordinators, activities in STEM and arts, support in building academic and life skills through goal setting and leadership training, and other engaging sessions.
EmpowerHub operates within the school setting, taking place during advisory periods, with a focus on addressing middle school girls' disengagement and academic underperformance. The goal is to provide participants at each partnering location with 35-50 hours of high-impact programming throughout the academic year. EmpowerHub program will be available to a cohort of 25-40 6th-grade girls, at each of the seven participating IPS middle schools.

2024-09-16

2027-05-31

Standardized GPA; Behavioral index consisting of incident reports, chronic absenteeism, grade repetition, and homework completion

We will analyze the following academic and behavioral outcomes:
Standardized GPA: cumulative GPA, normalized by the control group mean and standard deviation at the end of year 1 within a given school
We will combine non-cognitive outcomes into a standardized summary index or alternative summary aggregated index (see, e.g. Rose et al, 2022), behavioral index:
Incident reports: Number of reported school incidents, suspensions, and detentions, and an indicator for whether an incident occurred during the school year
Chronic absenteeism: indicator for whether participant has been chronically absent
Grade repetition: indicator for whether the student did not progress to the next grade on time
Homework completion: cumulative rate of homework completion as reported by school records
All of these outcomes will be measured at the end of a given school year.

Aggregated and disaggregated standardized test scores, chronic absenteeism, grade repetition, homework completion, GPA by subject category, school attendance rate, full attendance, disciplinary outcomes, participation in other extracurricular and athletic activities, and survey-based social outcomes

Aggregated and disaggregated standardized test scores: standardized test scores in ILearn (math and ELA)
Chronic absenteeism: indicator for whether participant has been chronically absent
Grade repetition: indicator for whether the student did not progress to the next grade on time
Homework completion: cumulative rate of homework completion as reported by school records
GPA by subject category: standardized GPA broken into subject category
Participation in other extracurricular and athletic activities: indicator for whether student participates in other extracurricular and athletic activities outside of EmpowerHub
School attendance rate: fraction of school days attended
Full attendance: indicator for attending 94% or more of school days
Disciplinary outcomes: number of school suspensions and detentions, and an indicator for whether the participant had any suspensions or detentions during the school year
Incident reports: Number of reported school incidents, and an indicator for whether an incident occurred during the school year
Social outcomes: collected at the end of the academic year, constructed from student responses to a survey administered in schools by IPS with questions on self-efficacy and belonging. We are collaborating with IPS on the survey content, so outcome construction is dependent upon data availability and granularity.

The Girls Inc. team iteratively designed the study throughout weekly calls with the research team. The following elements describe the procedures beginning with recruitment and study enrollment.
Study recruitment will run in July and August of school years 2024-2025 through 2026-2027. The Girls Inc. recruitment team will attend school-based back-to-school events and present during the in-school lunch periods at the seven Indianapolis Public School (IPS) middle schools in which Girls Inc. will begin offering EmpowerHub programming to 6th grade girls starting in the fall of 2024. Those eligible children who provide assent and whose parents provide parental consent will be enrolled in the study.
In September of each year of the study, the research team will randomize study participants into two equal groups, treatment and control, by school. Depending on program capacity constraints in subsequent years, the randomization ratio will change as the number randomized into treatment will be reduced to match the number of open spots left by attriters. The treatment group will begin EmpowerHub programming soon after randomization, and the control group will continue attending standard in-school programming (i.e., an advisory period and lunch). EmpowerHub participants will be invited to continue in the EmpowerHub program through 8th grade.
Through a data sharing partnership with Girls Inc. and IPS, the research team will receive academic outcomes for study participants from the PowerSchool and Schoology platforms, as well as other administrative data collected by the school system. The research team will also receive data from an annual survey of all students across the seven involved schools, using only the responses of study participants in any analyses for the study. We are in the process of coordinating with school administrators to potentially expand existing survey content.
Program effects will be estimated by an intent-to-treat design where standard deviation changes in grades and test scores will be regressed on a vector of observed characteristics, and a treatment group dummy. We will also report treatment-on-treated effects using a two-stage least squares regression approach and treatment assignment as the instrument. Heterogeneous impacts can be obtained by estimating the baseline regression with additional controls for groups and group-by-treatment interaction effects. The sub-groups outlined below are subject to change based upon data availability as we continue to formalize data sharing relationships with relevant schools and other stakeholders.
One-year outcomes will be collected and reported for the first enrollment cohort in summer 2025, and preliminary results will be updated and reported for each subsequent year of the study. Three-year results will be available for the first enrollment cohort in summer of 2027, and for the full sample in the summer of 2029.

We plan to investigate the following sub-groups:
- Baseline cognitive measures index: We will use grades and standardized test scores prior to randomization to predict baseline cognitive/academic performance.
- Baseline non-cognitive measures index: We will use disciplinary outcomes, attendance, homework completion, and grade repetition prior to randomization to predict baseline non-cognitive traits.
- To measure the level of poverty and family resources at baseline, we will create a family resources index: This index will consist of family structure, family income, parent/guardian education level, family homelessness status, all measured at baseline.
- Racial congruence: We will use student and program coordinator race to investigate the impact of same-race program coordinators on participant outcomes.

Not available

Computer (STATA)

individual

No

1,050 individuals

1,050 individuals

1,050 individuals

We plan to enroll a study sample of approximately 1,050 individuals, with approximately 40% of these assigned to the treatment group (offered a spot in EmpowerHub), and expect a program take-up rate of around 80%. Because full attendance (attending 94% or more of school days) is readily available rather than chronic absenteeism (missing 10% or more of school days) for the schools we are studying, we report full attendance for our power calculations. Based on school district data, 47.3% of students at the seven participating schools attend school for at least 94% of school days. We are powered to detect a 10.42 percentage point change in full attendance, or a 22% increase. For standardized outcomes in test scores and grades, we are powered to detect a 0.158 standard deviation change between the control group and the treatment group.

Impact Evaluation of Girls Inc. EmpowerHub Analysis Plan