Middle School Parent Engagement Text Messaging Study--Grade-Specific Program Analysis Plan

Last registered on July 14, 2024

View Trial History

Pre-Trial

Trial Information

General Information

Title

Middle School Parent Engagement Text Messaging Study--Grade-Specific Program Analysis Plan

RCT ID

AEARCTR-0009762

Initial registration date

October 31, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

October 31, 2022, 3:53 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated

July 14, 2024, 6:14 PM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Name

Kalena Cortes

Affiliation

Texas A&M University

Contact Primary Investigator

Other Primary Investigator(s)

PI Name

Brian Holzman

PI Affiliation

Texas A&M University

Contact Investigator

Additional Trial Information

Status

On going

Start date

2020-09-17

End date

2026-08-31

Keywords

Education

Additional Keywords

JEL code(s)

I21, I24, I26, I29

Secondary IDs

National Science Foundation Award Number: 1918016

Prior work

This trial does not extend or rely on any prior RCTs.

Abstract

Parent involvement at both home and school during the middle school years is positively associated with a range of behavioral and academic outcomes. Yet, due to structural changes in school environments and the increasing autonomy of teenagers, parent involvement often declines during the formative middle school years. Therefore, parents face unique challenges in their ability to both monitor their middle school child’s academic progress and be engaged in their child’s life.

This project covers a set of randomized controlled trials (RCTs) of Texts4Teens, a text messaging program for parents of middle school that aims to increase parental involvement by decreasing the behavioral barriers to engaged parenting.

This document describes grade-specific text message interventions administered during three academic years: 2020-2021, 2021-2022, and 2022-2023. Below are the school districts that participated in the study these three years (pseudonyms):

• 2020-2021: District 1, District 2, and District 3
• 2021-2022: District 1, District 2, District 3, District 4, District 5, District 6, and District 7
• 2022-2023: District 1, District 2, District 3, District 4, District 5, District 6, and District 7

Our middle school parenting program, Texts4Teens, is a text-based intervention that seeks to positively influence parent-teenage day-to-day interactions by addressing potential barriers to parents’ home and school involvement with their child. Texts4Teens’ goal is to narrow achievement gaps before children enter high school and beyond. The text messages include facts and tips on: 1) How to best support and guide their child’s academic trajectory as they transition from middle school to high school; 2) How to best support their child’s social and emotional learning (SEL) skills; and 3) How knowledge, skills, and abilities of their child are grow-able. Parents can choose to receive texts in English or Spanish. Every week, we will send middle school parents in the treatment group three texts about a particular academic topic and an SEL skill or activity for their child. Specifically,

--> The INSPIRE texts will be sent on Mondays. These texts are designed to motivate parents on a weekly academic topic related to their middle school child (e.g., asking questions in class, learning from mistakes, sharing schoolwork in progress, talking about report cards, and talking to teachers).
--> The EXCEL texts will be sent on Wednesdays. These texts address the academic topic of the week and aim to: (1) develop specific social and emotional learning (SEL) skills for their child (i.e., growth mindset, self-efficacy, self-management, self-awareness, and social awareness, responsible decision-making), (2) promote their child’s school engagement (i.e., emotional, behavioral, or cognitive), (3) encourage a parenting practice (i.e., autonomy support, monitoring, or warmth) and/or (4) foster a type of parental involvement (i.e., home-based, school-based, or academic socialization).
--> The BOND texts will be sent on Fridays and are also related to the academic topic of the week. These texts typically reinforce the week’s aim from the EXCEL text by fostering positive parent-child interactions (e.g., affective interactions).

External Link(s)

Program website

Registration Citation

Citation

Cortes, Kalena and Brian Holzman. 2024. "Middle School Parent Engagement Text Messaging Study--Grade-Specific Program Analysis Plan." AEA RCT Registry. July 14. https://doi.org/10.1257/rct.9762-1.1

Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Experimental Details

Interventions

Intervention(s)

Our middle school parenting program, Texts4Teens, is a text-based intervention that seeks to positively influence parent-teenage day-to-day interactions by addressing potential barriers to parents’ home and school involvement with their child. Texts4Teens’ goal is to narrow achievement gaps before children enter high school and beyond. The text messages include facts and tips on: 1) How to best support and guide their child’s academic trajectory as they transition from middle school to high school; 2) How to best support their child’s social and emotional learning (SEL) skills; and 3) How knowledge, skills, and abilities of their child are grow-able. Parents can choose to receive texts in English or Spanish. Every week, we will send middle school parents in the treatment group three texts about a particular academic topic and an SEL skill or activity for their child. Specifically,

--> The INSPIRE texts will be sent on Mondays. These texts are designed to motivate parents on a weekly academic topic related to their middle school child (e.g., asking questions in class, learning from mistakes, sharing schoolwork in progress, talking about report cards, and talking to teachers).
--> The EXCEL texts will be sent on Wednesdays. These texts address the academic topic of the week and aim to: (1) develop specific social and emotional learning (SEL) skills for their child (i.e., growth mindset, self-efficacy, self-management, self-awareness, and social awareness, responsible decision-making), (2) promote their child’s school engagement (i.e., emotional, behavioral, or cognitive), (3) encourage a parenting practice (i.e., autonomy support, monitoring, or warmth) and/or (4) foster a type of parental involvement (i.e., home-based, school-based, or academic socialization).
--> The BOND texts will be sent on Fridays and are also related to the academic topic of the week. These texts typically reinforce the week’s aim from the EXCEL text by fostering positive parent-child interactions (e.g., affective interactions).

Intervention Start Date

2020-10-12

Intervention End Date

2023-06-19

Primary Outcomes

Primary Outcomes (end points)

- Student academic and behavioral outcomes (using student-level administrative data from each district)
- Perceived levels of parental engagement and students’ development of social and emotional skills (using end-of-year parent survey data & texted prompts throughout the texting program)
- Parent and student educational aspirations and expectations? (using parent survey data)

Primary Outcomes (explanation)

Some items related to socioemotional learning will be combined into indices using item response theory (e.g., using individual items from the parent survey data)

Secondary Outcomes

Secondary Outcomes (end points)

Data from the texting platform will be used to examine opt-out rates. Open-ended text responses from the parent survey, as well as parents' texted responses to the texts and prompts, will be used to examine parent perceptions of the program.

Secondary Outcomes (explanation)

Experimental Design

To assign parents to one of the texting groups, we will employ a multi-site person-level randomized controlled trial design, blocking on, for example, middle school site, grade level, and texting language, English or Spanish (Spybrook, Bloom, Congdon, Hill, Martinez, & Raudenbush, 2011). Specifically, within the districts’ middle schools, grades, and texting languages, we will randomly select half of participating parents to join the control texting group and half to join the INSPIRE-EXCEL-BOND texting group.

This study builds on an earlier text message RCT in District 1 in spring 2019 (pilot) and 2019-2020. That study used a general, non-differentiated curriculum that gave students in grades 6 through 8 the same text messages. During this phase of the study, we learned that parent participants commented that they would prefer grade-specific text messages. Therefore, this study delivers grade-specific messages to parents of middle school children.

District 1 parents who participated in the earlier general program and who enrolled in the grade-specific program in 2020-2021 were assigned to their earlier treatment condition. District 1, District 2, and District 3 parents who participated in the grade-specific program in 2021-2022 were assigned to their earlier treatment condition; see Figure 1 for an illustration. In 2022-2023, we, again, assigned re-enrollees from those three districts to their earlier treatment condition. This design allows us to examine whether multiple years of treatment has effects on student outcomes, relative to zero years of treatment.

For District 4, District 5, District 6, and District 7, parents who re-enroll will be re-randomized. The re-randomization of these districts will allow us to causally estimate dosage effects. Specifically, there will be four groups of parents across the 2021-2022 and 2022-2023 school years: control-control (CC), control-treatment (CT), treatment-control (TC), and treatment-treatment (TT); see Figure 2 for an illustration. This design will allow us to test whether one year of treatment has effects on student outcomes over zero years of treatment, as well as whether two years of treatment has effects on student outcomes over zero and one years of treatment (a double-dosage effect).

It should be noted that for District 2 and District 6, there are intermediate and middle schools. Intermediate schools serve grades 5 and 6, while middle schools serve grades 7 and 8. For these two districts only, we include 5th grade students in the randomization. Sixth grade students in District 2 and District 6 are unique in that they have already been exposed to one full year of intermediate/middle school, whereas 6th grade students in other school districts have not been exposed to that experience.

Experimental Design Details

Not available

Randomization Method

Randomization done in Stata

Randomization Unit

Individual-level randomization within school-by-grade blocks (or school-by-grade-by texting language blocks, if there are sufficient numbers of Spanish-speaking parents in a district)
- Grade includes grade levels 5, 6, 7, or 8 (some districts include grade 5 in a middle or intermediate school)
- Texting language includes English or Spanish text messages (chosen by parent during enrollment)

Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters

2020-2021 (same-year effects sample sizes)

District 1: 2,324 students, 46 blocks (school-by-grade-by texting language fixed-effects)
District 2: 2,106 students, 12 blocks (school-by-grade fixed-effects)
District 3: 2,888 students, 33 blocks (school-by-grade fixed-effects)

2021-2022 (same-year effects sample sizes)

District 1: 1,133 students, 47 blocks (school-by-grade-by texting language fixed-effects)
District 2: 943 students, 12 blocks (school-by-grade fixed-effects)
District 3: 1,246 students, 33 blocks (school-by-grade fixed-effects)
District 4: 2,819 students, 34 blocks (school-by-grade-by texting language fixed-effects)
District 5: 4,976 students, 66 blocks (school-by-grade-by texting language fixed-effects)
District 6: 3,510 students, 36 blocks (school-by-grade-by texting language fixed-effects)
District 7: 609 students, 18 blocks (school-by-grade fixed-effects)

2022-2023 (same-year effects sample sizes) xxxxxxxxxxxxxx

District 1: 1,133 students, 47 blocks (school-by-grade-by texting language fixed-effects)
District 2: 943 students, 12 blocks (school-by-grade fixed-effects)
District 3: 1,246 students, 33 blocks (school-by-grade fixed-effects)
District 4: 2,819 students, 34 blocks (school-by-grade-by texting language fixed-effects)
District 5: 4,976 students, 66 blocks (school-by-grade-by texting language fixed-effects)
District 6: 3,510 students, 36 blocks (school-by-grade-by texting language fixed-effects)
District 7: 609 students, 18 blocks (school-by-grade fixed-effects)
District 8: 609 students, 18 blocks (school-by-grade fixed-effects)

2023-2024 (same-year effects sample sizes)

District 4: 1,426 students, 33 blocks (school-by-grade-by texting language fixed-effects)
District 5: xxx4,976 students, 66 blocks (school-by-grade-by texting language fixed-effects)
District 6: xxx3,510 students, 36 blocks (school-by-grade-by texting language fixed-effects)

Sample size: planned number of observations

Sample size (or number of clusters) by treatment arms

2020-2021 (same-year effects sample sizes)

District 1
- Treatment: 1,169
- Control: 1,155
- Total: 2,324
- Blocks: 46

District 2
- Treatment: 1,051
- Control: 1,055
- Total: 2,106
- Blocks: 12

District 3
- Treatment: 1,445
- Control: 1,443
- Total: 2,888
- Blocks: 33

2020-2021 Academic Year
District Treatment Control Total # Blocks
District 1 1,431 1,409 2,840 46
District 2 1,193 1,203 2,396 12
District 3 1,598 1,605 3,203 33
Total 4,222 4,217 8,439 91

2021-2022

District 1
- Treatment: 569
- Control: 564
- Total: 1,133
- Blocks: 47

District 2
- Treatment: 472
- Treatment: 471
- Total: 943
- Blocks: 12

District 3
- Treatment: 622
- Control: 624
- Total: 1,246
- Blocks: 33

District 4
- Treatment: 1,410
- Control: 1,409
- Total: 2,819
- Blocks: 34

District 5
- Treatment: 2,488
- Control: 2,488
- Total: 4,976
- Blocks: 66

District 6
- Treatment: 1,753
- Control: 1,757
- Total: 3,510
- Blocks: 36

District 7
- Treatment: 302
- Control: 307
- Total: 609
- Blocks: 18

2021-2022

District 1
- Treatment: 569
- Control: 564
- Total: 1,133
- Blocks: 47

District 2
- Treatment: 472
- Treatment: 471
- Total: 943
- Blocks: 12

District 3
- Treatment: 622
- Control: 624
- Total: 1,246
- Blocks: 33

District 4
- Treatment: 1,410
- Control: 1,409
- Total: 2,819
- Blocks: 34

District 5
- Treatment: 2,488
- Control: 2,488
- Total: 4,976
- Blocks: 66

District 6
- Treatment: 1,753
- Control: 1,757
- Total: 3,510
- Blocks: 36

District 7
- Treatment: 302
- Control: 307
- Total: 609
- Blocks: 18

2023-2024

District 4
- Treatment: 713
- Control: 713
- Total: 1,426
- Blocks: 33

District 5
- Treatment: 724
- Control: 724
- Total: 1,448
- Blocks: 35

District 6
- Treatment: 1,068
- Control: 1,051
- Total: 2,119
- Blocks: 72

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

We used the PowerUp! tool for power analyses (Shiny App here: https://powerupr.shinyapps.io/index/), allocating 50% of students to the treatment and 50% to the control. Below are the minimum detectable effect sizes (SD units). 2020-2021 (same-year effects sample sizes) District 1: 46 blocks, harmonic mean across blocks = 15.83980 --> MDES = 0.170 SD District 2: 12 blocks, harmonic mean across blocks = 172.3907 --> MDES = 0.101 SD District 3: 33 blocks, harmonic mean across blocks = 72.02920 --> MDES = 0.094 SD 2021-2022 (same-year effects sample sizes) District 1: 47 blocks, harmonic mean across blocks = 11.03975 --> MDES = 0.201 SD District 2: 12 blocks, harmonic mean across blocks = 69.04140 --> MDES = 0.159 SD District 3: 33 blocks, harmonic mean across blocks = 29.79569 --> MDES = 0.146 SD District 4: 34 blocks, harmonic mean across blocks = 17.98736 --> MDES = 0.185 SD District 5: 66 blocks, harmonic mean across blocks = 56.84348 --> MDES = 0.075 SD District 6: 36 blocks, harmonic mean across blocks = 32.24948 --> MDES = 0.134 SD District 7: 18 blocks, harmonic mean across blocks = 31.09235 --> MDES = 0.194 SD

Supporting Documents and Materials

IRB

Institutional Review Boards (IRBs)

IRB Name

Texas A&M University Institutional Review Board

IRB Approval Date

2019-03-01

IRB Approval Number

IRB2019-0035

IRB Name

National Bureau of Economic Research Institutional Review Board

IRB Approval Date

2019-07-22

IRB Approval Number

19_291

Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information