Experimental evaluation of the integration of Sesame Workshop’s Watch, Play, Learn and the Semillas de Apego intervention to foster maternal mental health and early childhood development among conflict-affected communities in Colombia

Last registered on July 28, 2023

Pre-Trial

Trial Information

General Information

Title
Experimental evaluation of the integration of Sesame Workshop’s Watch, Play, Learn and the Semillas de Apego intervention to foster maternal mental health and early childhood development among conflict-affected communities in Colombia
RCT ID
AEARCTR-0011834
Initial registration date
July 24, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
July 28, 2023, 1:59 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Primary Investigator

Affiliation
Universidad de los Andes

Other Primary Investigator(s)

PI Affiliation
Boston College
PI Affiliation
Universidad de los Andes

Additional Trial Information

Status
On going
Start date
2023-04-03
End date
2023-12-20
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Over half a million children between 0 to 5 years of age have been affected by armed conflict and forced displacement in Colombia, while many others have been forced to migrate with their families from Venezuela. While conflict and displacement can bring about devastating effects for young children, healthy ad secure emotional attachments between children and caregivers can help regulate the socioemotional responses to stress and adversities and lay the foundations for proper early childhood development.
In this trial, we will evaluate the impact of integrating Sesame Workshop’s Watch, Play, Learn (WPL) socioemotional content within the Semillas de Apego program (SA). The program is a community and group-based psychosocial intervention focused on restoring mental health and promoting healthy child-parent attachments of caregivers of young children in conflict-affected settings. Between 2018 and 2020, the impact evaluation of SA was implemented in Tumaco, Colombia with a sample of around 1,400 primary caregivers of children ages 2-5. The results of the impact evaluation demonstrate that the program had sizeable and statistically significant effects on caregiver mental health, the style and quality of the child-caregiver relationship, and on child mental health and development.
This new trial evaluates the integration of the WPL socioemotional contents within the SA program to strengthen socio-emotional learning and regulation among the children of the caregivers participating in the SA group sessions. Specifically, the SA and WPL enhanced model integrates 23 socioemotional videos and 3 visual aids from WPL into the SA curriculum. While the SA standard approach targets children indirectly by supporting their caregivers, the enhanced model aims to reach children directly through the delivery of these contents and to strengthen caregivers’ capacities to connect with their children through 1) improved understanding of children’s social-emotional development; 2) specific age-appropriate contents that enable caregivers to engage with their children’s in social-emotional learning ; 3) increased avenues for socioemotional co-regulation (this is, the improved socioemotional regulation of children will have positive effects on their caregivers, which will then create positive feedbacks on children themselves).
In this study, we will analyze whether the enhanced model brings about additional impacts on caregivers’ mental health, caregiver-child relationships, and children’s developmental outcomes. Since we are limited by the study’s sample and underpowered, this trial should be considered as a feasibility study with the hope of a future well-powered impact evaluation. In parallel, we will conduct a process evaluation based on the principles of implementation science to understand the processes and factors that enable or hinder the appropriate integration of the two programs.
The standard and enhanced SA models were implemented in Tumaco and Jamundí, two municipalities in Colombia heavily affected by violence and poverty during the first cohort of 2023 (April – July). Over a time span of 7 months, we will follow the implementation of the enhanced program with 20 groups of on average 16 participants each, all of them mothers or primary caregivers of children 3 to 5, thereby reaching a total of 320 participants and their children.
The impact evaluation is based on a randomized control trial in which we will assign eligible subjects to two experimental arms: the standard program, which serves as the control, and the enhanced model, which in this case will be the treatment group. We randomly assigned the implementation groups (n=19) to the treatment (n=9) and control arms (n=10). In this case, each implementation group corresponds to a different neighborhood, meaning that random assignment was conducted at the neighborhood level.
Participants in both experimental groups participated in 15 group-led sessions over the period of 3 months. Participants in the treatment (enhanced model) group will also have access to WPL videos starting from the 7th session when the program changes focus from the caregiver to the child and their relationship.
We will assess whether the enhanced program brings about differentiated positive and sequential impacts on the following dimensions, relative to the standard program: (i) primary caregiver’s mental health, (ii) child-parent relationship, (iii) quality of the child-parent emotional bond, (iv) children’s mental health, and (v) children’s cognitive and socioemotional development. For this purpose, data will be collected by an independent firm at baseline and two post-intervention assessments: immediately after the intervention and 4 months after the implementation has concluded to understand the short-run impacts of the enhanced model and whether these impacts persist over time.
External Link(s)

Registration Citation

Citation
Moya, Andres, Maria Piñeros-Leaño and Maria Jose Torres Herrera. 2023. "Experimental evaluation of the integration of Sesame Workshop’s Watch, Play, Learn and the Semillas de Apego intervention to foster maternal mental health and early childhood development among conflict-affected communities in Colombia." AEA RCT Registry. July 28. https://doi.org/10.1257/rct.11834-1.0
Experimental Details

Interventions

Intervention(s)
The SA program is implemented through 15-weekly sessions among groups of 16 participants, who are mothers or primary caregivers of children 3 to 5. The implementation of the SA program is based on a community-based model, where each implementation group is led by 2 non-professional community agents, also known as facilitators, who have previously participated in a three-week intensive training or have been participants of the SA program in the past. Overall, there are 8 facilitators, arranged into 4 dyads. In each municipality, one dyad will lead the group assigned to the standard model while the other dyad will lead the sessions of the groups assigned to the enhanced program. There will be 10 groups in the WPL intervention and 10 groups in the control arm, with 160 participants in each group.
Two to three weeks before the program started, community meetings were conducted to share the information on the program, its objectives, structure, and eligibility criteria, and to invite caregivers to participate. To facilitate participation and promote adherence to the program, we held during the day and time convened with the participants, provided incentives, including a 10 USD voucher for the participants every time they attend 3 sessions and a monthly phone plan to those participants that remain in the program.
Only the treatment group received the WPL content. The WPL content was conceptually integrated into each session starting from session 7, which is when SA sessions’ objectives emphasize understanding early childhood development milestones and healthy child-parent relationships to promote Early Childhood Development (ECD) (weeks 8-12) and building support networks and child-rearing teams (weeks 12-15). The education team at SW and the technical team at Universidad de los Andes organized the WPL videos and visual aids into five groups. Group 1 introduces big emotions; Group 2 addresses fear and dealing with challenging situations; Group 3 helps caregivers relate the child's behavior to big emotions and identify alternative strategies to deal with these emotions; Group 4 involves child-rearing teams in identifying the child's emotions and strategies; and Group 5 reinforces breathing techniques. At the end of each session, facilitators linked the discussions that emerged during the session to the WPL videos.
The WPL content was delivered through social media (Facebook and WhatsApp). On FB, the contents were disseminated on private groups within the SA general page, one private group per intervention group. Private groups allow control of the enrollment of caregivers, which is key for the validity of the experimental evaluation, and in particular to prevent contamination to caregivers in the control group. In the private groups, caregivers in the treatment arms received private posts with WPL contents after each weekly session is completed in addition to the standard SA contents. Additionally, they received messages through private group chats and had the chance to interact with other members in the same group. Participants in the control group only received SA content via group chats and the public posts on the SA FB general page. Since the success of this strategy depended on access to internet, participants in both groups received pre-paid phone plans.
On WhatsApp, participants in the treatment arm received a message with a WPL trailer for that week’s videos and a link inviting to watch the videos in the corresponding FB group. Additionally, by the end of each week, the participants in the treatment arm received a full WPL video, in case they were not able to join FB for any particular reason. In this sense, WhatsApp allow for greater reach as it gives access to those participants without FB access. Facebook’s large advantage is that it provides data analytics to understand the participant’s access and use of the WPL content.
Intervention Start Date
2023-04-10
Intervention End Date
2023-07-28

Primary Outcomes

Primary Outcomes (end points)
We identify five different constructs as primary outcomes for evaluation. We will conduct two rounds of post-intervention data collection that will take place immediately after implementation and four (4) months after the 15-week program has concluded. This will enable us to observe the short and medium-run impacts of the intervention. The second post-intervention will be conditioned to the results of access and use of the Sesame Workshop content in the first post-intervention evaluation.
The five primary outcomes and the corresponding hypothesis that guide this evaluation are explained below. For each outcome, we will pool data from the following psychometric scales and construct a composite index for each family of outcomes, following Kling et al. (2020) based on the standardized inverse-covariance weighted average of the different items in each dimension. This estimation method reduces measurement error in any individual scale or assessment and reduces the dimensions of the analysis, which increases the statistical power. For children’s outcomes, we will also standardize by age. The primary outcomes are the following:
1. Primary caregiver’s mental health: Participation in the intervention arm of the evaluation will generate a reduction in the number of emotional symptoms reported by caregivers one month after the intervention has been completed (first post-intervention assessment) and noticeably four months after the end of the intervention (second post-intervention assessment). This outcome will be measured with five sub-scales of an adapted version of the Symptom Checklist-90-R (Derogatis, 1994). This self-reported measure asks caregivers to report on several indicators of emotional well-being and mental health experienced during the last 30 days.
2. Child rearing practices: Participation in the intervention arm of the evaluation will generate an increment in the number and type of reported child-rearing practices implemented by caregivers right after the end of the intervention (first post-intervention assessment). These effects will persist four months after the end of the intervention (second post-intervention assessment). Outcomes will be measured with a questionnaire in which caregivers report whether they engaged in any of the six stimulating activities with their children in the previous week: (1) reading stories or looking at books with images; (2) telling stories; (3) singing songs; (4) playing with child; (5) taking the child outside; and (6) spending time in physical activities with child. Additionally, they report the type of disciplines used during child rearing practice. To analyze this construct as a composite measure we will compute a summary score ranging from zero (no engagement in any activity) to six (engagement in the six activities), as previously done in several studies (e.g., Bornstein & Putnick, 2012; Cabrera et al., 2011; Jeong et al., 2017; Jeong et al.,2016; Sunet al., 2016) and also weigh in the corresponding score of the disciplines.
3. Healthy child-parent emotional bonds: Participation in the intervention arm of the evaluation will lower the stress in the child parent relationship (first post-intervention assessment) and will generate a positive effect in the quality of the attachment relationship between caregiver and children four months after the end of the intervention (second post-intervention assessment). Outcome will be measured with three different instruments: the parenting stress-index (PSI, Hasket et al, 2006), the Being a Mother (BaM, Matthey, 2011) and the child parent relationship (CPR, Discroll and Pianta, 2011). The PSI is a measure focused on three major domains of stress: child characteristics, parent characteristics and situational/demographic life stress. Additionally, during the BaM questionnaire, participants will be asked to answer 13 items on child and adult experience and relationship closeness, which they will respond to using a 4-point Likert scale. Finally, CPR asks the caregiver 30 items which capture the quality of the child-parent relationship. We will compute a summary score for the three scales.
4. Children’s mental health: Participation in the intervention arm of the evaluation will generate a positive effect in the caregivers’ report of child mental health in the medium run, or four months after the end of the intervention (second post-intervention assessment). Outcome will be measured with an adapted instrument of the Trauma Symptom Checklist for Young Children (TSCYC, Briere, 2005) to describe child-levels of trauma and abuse-related symptomatology.
5. Child’s emotional development: Participation in the intervention arm of the evaluation will generate a positive effect in children’s’ cognitive and socioemotional development in the medium run, or four months after the end of the intervention (second post-intervention assessment). Measured with the Strengths and Difficulties Questionnaire (ASQ-SE). We will focus our report on a composite score of the five different subscales within this questionnaire.
Primary Outcomes (explanation)
Based on dynamic auto-complementarity in early childhood development and parental inputs, we expect the WPL and SA integration to have multiplying effects, with the two channels reinforcing each other. As caregivers are better able to respond to children's progress, investments should become more productive improving emotional coregulatory process between caregiver and child and better social and emotional outcomes for the child. This should then map out into caregiver’s perceptions of the effectiveness of their inputs and strategies and further promote their capacity and willingness to respond to children’s development, thereby creating virtuous cycles and reinforcing effects.

Secondary Outcomes

Secondary Outcomes (end points)
The secondary outcomes of the evaluation are mainly exploratory and consist of analyzing each individual scale and subscales. We will calculate individual continuous z scores and risk thresholds for each subscale and scale. We will adjust the signs of each scale so that positive scores denote improvements and standardize the scales according to the standard deviation of the control group by wave. The secondary outcomes in each primary outcome are:
1. Primary caregiver’s mental health: We focus on the following Symptom Checklist-90-R’s sub-scales: (i) Anxiety, (ii) Depression, (iii) Phobic Anxiety, (iv) Hostility and (v) Sensitivity.
2. Child rearing practices: We will focus on three subscales: the six stimulating activities, the negative disciplines and the positive disciplines.
3. Healthy child-parent emotional bonds: Since there are three different instruments for this outcome, we have three different types of subscales. We will analyze the individual sub-scales for the following domains of the PSI: parental distress, parent-child dysfunctional interaction, and difficult child. BaM will provide insights into the following subscales: emotional closeness, child experience and adult experience. CPR will assess the subscales: emotional closeness, conflict resolution and dependence.
4. Children’s mental health: The subscales of the Trauma Symptom Checklist for Young Children (TSCYC, Briere, 2005) are: anger/agression, anxiety, depression, dissociation, sex concerns, post-traumatic stress arousal, post-traumatic stress avoidance, post-traumatic stress intrusion and post-traumatic stress total.
5. Child’s emotional development: We will assess each child on five different SDQ subscales including emotional symptoms, conduct problems, hyperactivity and inattention, peer relationship problems and prosocial behavior.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The experimental evaluation of the enhanced model will be conducted as a Cluster-Randomized Control Trial (C-RCT). The eligible population for the study is all families in Tumaco and Jamundí, Colombia whose children’s ages range between 3 to 5 years of age.
Random assignment to the treatment was conducted at the group level (n=20). The groups were assigned by neighborhoods to minimize possible spillovers and to facilitate the program’s implementation. We could not cluster our random assignment at the facilitator level (randomizing which pair of facilitators has the WPL content), because we would have few clusters, and this hurts our power. For this reason, we will randomize across groups, meaning that certain facilitators will introduce the WPL in some groups. Participants are, thus, blind to treatment prior to the implementation.
Originally, 10 groups were going to be assigned to receive the enhanced model and the other 10 groups would have been assigned to the control arm. However, once the implementation started, only 9 groups were selected to the treatment arm due to geographical and contextual violence reasons. Additionally, the groups were expected to have 16 participants, but they actually have on average 13 participants, totaling 227 participants in the pilot.
Experimental Design Details
Randomization Method
Randomization using Stata commands
Randomization Unit
Intervention groups (n=20)
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
20 intervention groups
Sample size: planned number of observations
320 participants
Sample size (or number of clusters) by treatment arms
10 intervention groups each into treatment and control arms; 160 participants per treatment arm.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
With 20 intervention groups overall and 160 participants in each treatment arm, we estimated 80% of power to detect effect size of 0.4 standard deviations at minimum. See Figure 1. The minimal detectable effect size (MDES) is based on estimates a 0.05 probability of type I error, assigning 10 groups and half of the individual-level sample (160) to the treatment, and an intra-class correlation coefficient (ICC) of 0.03. Graph 1: Power analysis for planned scenario
IRB

Institutional Review Boards (IRBs)

IRB Name
Comité de Ética, Universidad de los Andes
IRB Approval Date
2023-02-20
IRB Approval Number
Protocol #1668 2023
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials