Experimental evaluation of Semillas de Apego, a group-based program to foster maternal mental health and early childhood development among violence exposed communities in Colombia.

Semillas de Apego is a group-based psychosocial program for victimized caregivers with children 2 to 5 in Colombia, a country devastated by violence. The program’s builds upon scientific evidence on (i) the way in which violence hinders early childhood development and erodes mothers’ mental health and their capacity to form nurturing relationships with their children, and (ii) the effectiveness of promoting healthy child-parent attachments to mitigate the effects of toxic stress on toddlers (Lieberman and Van Horn, 2011).
Semillas de Apego aims to foster the healthy child-parent attachments that promote appropriate affect regulation and healthy emotional development in the midst of adverse circumstances. For this purpose, the program first provides tools so that victimized caregivers can better understand their emotional and behavioral reactions to violence and can process their own trauma. Then, the program focuses on allowing a proper understanding of the child’s development trajectories and how they are silently being affected by the experience of adversities (such as violence exposure). Finally, the curriculum works towards fostering child-rearing good practices.
The program will be implemented through 15-weekly sessions among groups of 14 to 16 participants, who are mothers or primary caregivers of children 2 to 5. The sessions follow a detailed curriculum, which was developed in partnership between the Child Trauma Research Program at the University of California, San Francisco and our team at Universidad de los Andes. The original curriculum was first designed and adapted to the Colombian context between 2014 and 2015, based on the group-based Child Parent Psychotherapy curriculum (Reyes and Lieberman, 2010). In 2015 we conducted a small pilot test to assess the program’s validity. Based on the lessons that followed from this pilot test, we made further adaptations to the curriculum and developed a training manual for facilitators. Finally, between January and February 2018, we conducted a final adaptation to incorporate cultural and social practices that are common in Tumaco, Colombia, the implementation site of this trial.
Semillas de Apego will be delivered through the 18 Early Childhood Development Centers (ECDC) that are run by Genesis Foundation, our implementation partner, in the urban area of Tumaco. Currently, these ECDC serve over 2,000 children 0 to 5. Assignment into the program will follow a cluster-based randomization. In particular, we will first randomly select the Centers in which Semillas de Apego will implemented, and then we will randomly assign among eligible subjects within the treated and control centers to either arm. . We discuss this in more detail below.
Participants will be selected two to three weeks prior to the starting date of the program. Selection will be based using administrative data from Genesis Foundation. We will use this information to randomly assign a total of 640 mothers or primary care givers into the program and 640 into a control group. Participants will be assigned to a 16-member group intervention group in the ECDC closest to their residence. Groups will meet once per week for approximately 3 hours per session.
Subjects assigned to the control group, and those in centers assigned to the intervention but not randomly selected to participate in the program, will continue benefiting from the services available to all families affiliated with child development centers in the municipality of the intervention.
Due to local operation and implementation restrictions, the program will be delivered in four sequential cohorts of approximately 160 participants in the intervention arm and 160 participants in the control arm per cohort. The total number of intervention groups will vary per cohort starting with 9 groups in each of the two cohorts of 2018 and increasing to 12 groups for each of the two cohorts in 2019. After the implementation of these four sequential cohorts, we will reach 640 participants in the intervention arm, and 640 subjects in the control arm.
The implementation of the program will be led by a team of six social workers who participated in a three-week intensive training. The team of trained social workers will be assigned into three dyads. Each dyad will lead the implementation of the program among 3 to 4 groups per cohort and complete implementation and debriefing logs on the progress of the session and the participants. The implementation team, in addition, will be supervised by three expert coaches on a weekly basis. The supervision scheme aims to guarantee the quality and fidelity of the intervention but also to provide reflexive support to mitigate burn-out and symptoms of vicarious trauma.
To ensure adherence, we will implement the following actions: First, sessions will be held during a day and time convened with the participants. Second, community health workers will make weekly phone calls to follow-up on each participant’s needs and progress and encourage attendance. We followed this strategy during the implementation pilot and observed that 94% of the mothers who participated in the program completed all the sessions.

2018-04-10

2019-12-15

We identify five different constructs as primary outcomes for the program. According to the program’s theory of change, we should observe effects in these different constructs in a sequential way. In particular, we expect that the program will bring about positive effects in some dimensions during the implementation of the 15-week program; for instance, in the maternal mental health and child rearing practices domains. Additionally, we expect that effects on other dimensions, such as early childhood development, will build up and become salient at the second follow-up assessment. For this reason, we will conduct two rounds of follow-up data collection that will take place one (1) month and twelve (12) months after the 15-week program has concluded, . This will enable us to observe the short and medium run impacts of the intervention.
The five primary outcomes and the hypothesis that guide this evaluation are:
1) Primary caregiver’s mental health: Participation in the intervention arm of the evaluation will generate a reduction in the number of emotional symptoms reported by caregivers one month after the intervention has been completed (first follow-up) and one year after the end of the intervention (second follow-up)
2) Child rearing practices: Participation in the intervention arm of the evaluation will generate an increment in the number and type of reported child-rearing practices implemented by caregivers right after the end of the intervention (first follow-up). These effects will persist one year after the end of the intervention (second follow-up).
3) Healthy child-parent emotional bonds: Participation in the intervention arm of the evaluation will lower the stress in the child parent relationship (first follow-up) and will generate a positive effect in the quality of the attachment relationship between caregiver and children one year after the end of the intervention (second follow-up).
4) Children’s mental health: Participation in the intervention arm of the evaluation will generate a positive effect in the caregivers’ report of child mental health in the medium run; 12 months after the end of the intervention (second follow-up).
5) Child’s cognitive, social, and emotional development. Participation in the intervention arm of the evaluation will generate a positive effect in children’s’ cognitive and socioemotional development in the medium run; 12 months after the end of the intervention (second follow-up).

Building upon the program’s theory of change and evidence from the CPP clinical trials (Lieberman and Van Horn, 2011), we will observe caregivers and children’s outcomes throughout two follow-ups after the end of the intervention. We will include reduced versions of instruments designed and adapted to address all five primary outcomes at each moment in the data collection (baseline, first and second follow-up). Nevertheless, we formulate differential hypotheses of the effect of the program on different constructs at different moments.
Below, we describe the schedule of data collection that we will implement for each cohort. We detail the psychometric scales and assessment instruments that will be included at each phase.
Baseline Assessment:
For each cohort, we will administer a detailed household survey prior to the start of the intervention. The survey includes nine different modules that aim to characterize subjects and their households according to the: (1) demographic composition of the household, (2) education attainment of the primary caregivers and head of the household, (3) labor participation and primary source of income for the household, (4) child composition of the household and basic health indicators, (5) parental locus of control, (6) reported household-level exposure to violence, (7) primary caregiver time-use; (8) living arrangements and physical conditions of the house, and (9) contact information for future assessment.
In addition, we will administer the following scales and instruments to assess the baseline level of our primary outcomes:
1) Primary caregiver’s mental health: Measured with four sub-scales of the Symptom Checklist-90-R (Derogatis, 1994). This self-reported measure asks caregivers to report on several indicators of emotional well-being and mental health experimented during the last 30 days. In our evaluation, we focus on the sub-scales providing indicators for the extent of symptoms for the following psychopathologies: (i) Anxiety, (ii) Depression, (iii) Phobic Anxiety, and (iv) Interpersonal Sensitivity.
2) Child rearing practices: Measured with a questionnaire in which caregivers report whether they engaged in any of the following six stimulating activities with their children in the previous week: (1) reading stories or looking at books with images; (2) telling stories; (3) singing songs; (4) playing with child; (5) taking the child outside; and (6) spending time in physical activities with child. To analyze this construct as a composite measure we will computed a summary score ranging from zero (no engagement in any activity) to six (engagement in the six activities), as previously done in several studies (e.g., Bornstein & Putnick, 2012; Cabrera et al., 2011; Jeong et al., 2017; Jeong et al., 2016; Sun et al., 2016).
3) Healthy child-parent emotional bonds: Measured with the parenting stress-index (PSI, Adibin, 2012), a measure focused on three major domains of stress: child characteristics, parent characteristics and situational/demographic life stress. We will analyze a composite measure of this scale and the individual sub-scales for the following domains: (i) parental distress, (ii) parent-child dysfunctional interaction, and (iii) difficult child.
4) Children’s mental health: Measured with two separated scales reported by the caregiver. First, we will employ an adapted measure of the Trauma Symptom Checklist for Young Children (TSCYC, Briere,2005) to describe child-levels of trauma and abuse-related symptomatology. Second, we will employ the brief version of the Infant Toddler Social Emotional Assessment (BITSEA, Carter & Briggs-Gowan, 2004) to screen for social, emotional, and behavioral problems in our population. For both scales, we will focus on composite scores, yet we will provide detailed findings for each subscale.
5) Children’s cognitive, social, and emotional development. Measured at baseline with an adapted version of the Preschool Self-Regulatory Assessment (PSRA, Smith-Donald, Raver, Hayes & Richardson, 2007). This tool will provide independent scores of self-regulations in emotional, attentional, and behavioral domains. We will focus our report on a composite score, but also provide detailed findings for each subscale.
Additionally, at baseline and the two follow-up assessments we will record child height and weight. With this information, we will construct height and weight z-scores for each age cohort, and compare with WHO international scales. These anthropometric measures, in addition, become an additional indicator of the children’s development at baseline.


First Follow-Up: 1 month after end of intervention
For each cohort, we will re-assess all our participants in the aforementioned dimensions one month after the end of the treatment implementation. At this moment, we will also update key demographic and socioeconomic information to assess key socioeconomic changes. Following the program’s theory of change, at this phase of the data collection, our primary outcomes include the primary caregiver’s mental health, the caregiver´s report of child rearing practices, and the caregiver´s report of stress-index in parenting interactions (PSI, Adibin, 2012). We expect to observe positive effects at end line for the first two primary dimensions. For the latter, we may observe worse self-reported outcomes (higher levels of stress in the child-parent relation) in the treatment group relative to the control group. This, however would not be indicative of a negative effect of the intervention, but rather an improved understanding by the participants on the way in which trauma and adversities undermine the child-parent relationship.
Secondary outcomes at phase of data collection includes and observational measure of healthy child-parent emotional bonds, caregiver´s reports of children’s mental health (TSCYC and BITSEA), and direct assessment of social and emotional development of the child measured with the PSRA.
Second follow-up: 12 months after end of intervention
A year after the end of the implementation of the program for each cohort, we will again re-assess all our participants using the different scales and instruments described above to assess the key dimensions. Additionally, to better understand the change in the quality of the child-parent bond and children’s cognitive and socioemotional development, we will also administer the following instruments.
1) Healthy child-parent emotional bonds: Measured with an observation measured designed for this trial. This measure will describe the quality of the relationship between children and caregivers in the child-development center. An overall score will be produced and analyzed from observers’ ratings to items targeting: (a) the quality of the interaction, (b) the cognitive stimulation provided in the situation, and (c) the emotional support provided during the situation.
1) Children´s cognitive and social-emotional development: Measured with the international Development Learning Assessment (IDELA, Pisani, Borisova & Dowd, 2015). This direct assessment tool, will describe the: (i) motor development, (ii) emergent language and literacy, (iii) emergent numeracy and problem solving, and (iv) socio-emotional skills of children.

Owing to the sequential follow-up design we have designed for these trial, we characterize unexpected differences in our child and relational outcomes at the first-follow up as secondary outcomes. As such, we postulate that positive differences in: (i) Quality emotional relationships measured by observation of the interaction, (ii) child´s mental health, measured with the TSCYC and BITSEA, and (iii) child´s cognitive and socioemotional development, measured with the PSRA. While unpredicted according to our theory of change, positive differences could be observed at this phase of data collection. If present, we will report these findings as secondary outcomes.

The experimental evaluation of Semillas de Apego will be conducted as a Cluster-Randomized Control Trial (C-RCT). The eligible population for the study are all families served by Genesis Foundation’s ECDC in Tumaco, Colombia and whose children’s ages range between 2 to 4 years of age. For example, for the first cohort, the children should have been born between August 1st, 2014 to April 1st, 2016.
Random assignment to the treatment will be conducted at the ECDC level (n=19). We eliminated one of these EDCD because it did not provide the infrastructure required to run the group sessions – a private space. However, this center is not different than the other 18 ECDC over a range of observable characteristics. Out of the remaining 18 EDCD, we conducted a random assignment based on the ECD’s size, age-range, gender and anthropometrics of eligible children, the size of the center, and a composite socio-economic status score. In doing so, 9 ECDC to were selected to run the implementation, while the other 9 remaining centers were assigned to the control arm.
Within each center assigned to the treatment arm in each cohort, the total number of families invited to participate in the program, and the number of intervention groups, will be based on the number of children served by the center relative to the total number of children served by the 18 EDCD in Tumaco. Then, we will distribute the treated ECDC and intervention groups across the 4 cohorts, ensuring that each ECDC has at least one intervention group per year. Hence, larger ECDC in the treatment arm will have one or more intervention groups in each of the 4 cohorts, whereas the smaller ECDC will have at least one intervention group in 2018 and another one in 2019.
For each treated ECDC in each cohort, we will randomly order eligible subjects within each center to be invited to participate in the intervention and invite then to participate according to this ordered list. Invitations will be carried out by the team of six social workers starting three weeks prior to the start of the intervention. Eligible subjects will be contacted at the EDCD or by phone and will be informed of the objectives and structure of the program, the schedule of the group sessions, and the potential benefits, including small incentives (super market cash cards or other incentives for up to US$8) for completing each phase of the data collection.
Eligible subjects will also be informed that their decision to participate or not in Semillas de Apego and the data collection does not have any consequence on access to Genesis Foundation’s programs in the municipality, including their children’s access to the ECDC, or their participation in standard community groups and activities. Once eligible subjects accept to participate, they will be contacted by the data collection firm, which will coordinate the date and time of the baseline data collection.
We will follow a similar strategy in the EDCD assigned to the control arm. We will first calculate the number of families that were invited to participate in the program based on the number of children served by the center relative to the total number of children served by the 18 EDCD in Tumaco. Then, we will distribute the ECDC across the 4 cohorts, ensuring that in each ECDC we assemble control groups at least once per year. Hence, larger ECDC in the control arm will have one or more control groups in each of the 4 cohorts, whereas the smaller ECDC will have at least one control group in 2018 and another one in 2019.
Then, we will randomly order eligible subjects within each center to be invited to participate in the different phases of data collection. Invitations will be carried out by the data collection team three weeks prior to the start of the intervention. Eligible subjects will be contacted at the EDCD or by phone and will be informed of the objectives and structure of the data collection and the potential benefits, including small incentives (super market cash cards for up to US$8) for completing the data collection.
Eligible subjects will also be informed that their decision to participate in the data collection does not have any consequence on access to Genesis Foundation’s programs in the municipality, including their children’s access to the ECDC, or their participation in standard community groups and activities.
All randomization procedures will be conducted using the RANDOMIZE module (Kennedy & Mann, 2015), available in Stata 15. In doing so, we achieve balance on key characteristics at each phase of the randomization procedure.

All randomization procedures will be conducted using the RANDOMIZE module (Kennedy & Mann, 2015), available in Stata 15.

1st stage/level: Early Childhood Development Center. 2nd stage/level: individual - primary caregiver

Yes

18 (9 per treatment arm).

40 groups of 16 participants in treatment and 640 participants in the control arm: n = 1280

9 Early Childhood Development Centers

With 1280 participating households, nested in 18 child development centers, half of which are randomly assigned to an implementation cohort of the program, we estimate to have 80% of power to detect effect sizes ranging from 0.262 to 0.2221 at minimum. This range of minimal detectable effect sizes (MDES), is based on estimates in which we define a 0.05 probability of type I error, assign 9 clusters and half of the individual-level sample (640) to the treatment, and employing two-tailed tests in all our estimations. For all our estimations, we assume an Intra-class correlation coefficient (ICC) of 0.04, based on pilot estimations and a review of the literature. All power calculations were conducted with Power-Up! (Dong & Maynard, 2013), and validated using EGAP’s Online Power Calculation Tool. The smaller estimated MDES assumes that no covariates will be used in the estimation models, which would account for 30% of outcome variability explained by demographic covariates. Considering the range of effect sizes observed in the pilot evaluation, we consider that this study is adequately powered to detect smaller effects sizes that those previously reported. Assuming an attrition rate of 8%, the effective sample, we still be powered to detect effect sizes of 0.27. Telephonic check-ups and smalls incentives will be used to minimize attrition. Note: Security conditions have worsened in Tumaco, leading to some uncertainty regarding our ability to implement the program in 2019. In this extreme case, the trial would only be run over the two 2018 cohorts, with 640 participating households, nested in 18 child development centers, half of which are randomly assigned to an implementation cohort of the program. In such case, we estimate to have 80% of power to detect effect sizes ranging from 0.341 to 0.405 at minimum. This range of minimal detectable effect sizes (MDES), is based on the same estimates as above: 0.05 probability of type I error, 9 clusters and half of the individual-level sample (320) assigned to the treatment, two-tailed statistical tests in our estimations and an Intra-class correlation coefficient (ICC) of 0.04. If we account for covariates, and assuming an attrition rate of 8%, we be powered to detect effect sizes of 0.404.