Optimal Design of Digital Early Childhood Development Interventions: A Multifactorial Randomized Control Trial

Afini (i.e., the intervention) is an interactive “virtual assistant” that uses a chatbot to recommend simple, fun, age-appropriate activities that parents and other caregivers can complete with young children to support child development. Afini includes 1,500 activities that have been delivered to 70,000 users in Latin America. Afini’s activity recommendations are evidence-based and reflect principles of behavioral economics (e.g., by providing scientific rationale for the activities). Although they are designed to be “culturally neutral,” Afini activities can be adapted to reflect materials, games, and concepts that are most relevant to a given context or program. Activities focus largely on supporting children’s language, cognitive, motor, and social-emotional development.

We will test the benefits of Afini as an add-on to the Criança Feliz national home visiting program (relative to a control group that will keep receiving the Criança Feliz program as usual) and the relative benefits of specific Afini features, focusing on tradeoffs for cost-effectiveness. Specifically, we will examine:
1. delivery through a standalone app (a lower-cost alternative that does not rely on third-party fees) vs. WhatsApp messenger (an existing provider trusted by families, which may improve engagement);
2. sending alerts (i.e., unsolicited activity recommendations) once (less expensive) vs. twice (more intensive) per week;
3. customizing content to caregiver preference (may improve engagement) vs. children’s developmental needs (may lead to greater child-level impact).

Afini seeks to promote young children’s healthy development by supporting these day-to-day interactions that serve as the basis of nurturing care. Our theory of change is that parents (or other caregivers in the home) will engage with Afini and complete recommended activities with children. These processes will lead to improvements in parents’ knowledge, provision of stimulation, and mental health, which will improve the quality of their interactions with children as well as children’s ECD outcomes (i.e., motor, cognitive, language, and social-emotional skills). We hypothesize that Afini will benefit families already receiving home visiting because, unlike Criança Feliz (which does not currently have a defined curriculum), Afini provides concrete, specific, and customized activity recommendations designed specifically for vulnerable families (e.g., minimal text, few required materials). Furthermore, Afini can be used at families’ convenience, improving accessibility.

Afini (i.e., the intervention) is an interactive “virtual assistant” that uses a chatbot to recommend simple, fun, age-appropriate activities that parents and other caregivers can complete with young children to support child development. Afini includes 1,500 activities that have been delivered to 70,000 users in Latin America. Afini’s activity recommendations are evidence-based and reflect principles of behavioral economics (e.g., by providing scientific rationale for the activities). Although they are designed to be “culturally neutral,” Afini activities can be adapted to reflect materials, games, and concepts that are most relevant to a given context or program. Activities focus largely on supporting children’s language, cognitive, motor, and social-emotional development.

We will test the benefits of Afini as an add-on to the Criança Feliz home visiting program (relative to a control group that will keep receiving the Criança Feliz program as usual) and the relative benefits of specific Afini features, focusing on tradeoffs for cost-effectiveness. Specifically, we will examine:
1. delivery through a standalone app (a lower-cost alternative that does not rely on third-party fees) vs. WhatsApp messenger (an existing provider trusted by families, which may improve engagement);
2. sending alerts (i.e., unsolicited activity recommendations) once (less expensive) vs. twice (more intensive) per week;
3. customizing content to caregiver preference (may improve engagement) vs. children’s developmental needs (may lead to greater child-level impact).

Afini seeks to promote young children’s healthy development by supporting these day-to-day interactions that serve as the basis of nurturing care. Our theory of change is that parents (or other caregivers in the home) will engage with Afini and complete recommended activities with children. These processes will lead to improvements in parents’ knowledge, provision of stimulation, and mental health, which will improve the quality of their interactions with children as well as children’s ECD outcomes (i.e., motor, cognitive, language, and social-emotional skills). We hypothesize that Afini will benefit families already receiving home visiting because, unlike Criança Feliz (which does not currently have a defined curriculum), Afini provides concrete, specific, and customized activity recommendations designed specifically for vulnerable families (e.g., minimal text, few required materials). Furthermore, Afini can be used at families’ convenience, improving accessibility.

2023-04-15

2024-05-31

1. Child Development: We will use the Caregiver-Reported Early Development Instruments - Long Form (CREDI-LF) with a full sample and the Global Scales of Early Development - Long Form (GSED-LF) with a subsample.

The CREDI-LF is a parent-reported tool to measure child development using simple yes/no questions that has been used and validated in multiple countries, including Brazil, and demonstrates adequate psychometric properties to measure with precision motor, cognitive, language, and social-emotional development.

The GSED-LF is a direct assessment of child development, and we use it to complement the parent-report approach of the CREDI. The GSED was developed by a large team of researchers led by the WHO (including Dr. McCoy), designed to be culturally-neutral, and validity evidence has been collected from multiple countries. While it covers multiple domains, it produces a single overall development score.

1. Caregiver mental health: The Depression, Anxiety and Stress Scale - 21 Items (DASS-21)
2. Parents’ early stimulation: Family Care Indicators (FCI) and Parenting Styles and Dimensions Questionnaire (PSDQ)
3. Parents’ discipline behaviors: Conflict Tactics Scales: Parent-child Version (CTSPC)
4. Parent self-efficacy: Self-efficacy for Parenting Tasks Index-Toddler Scale (SEPTI-TS)

1. Caregiver mental health: The Depression, Anxiety and Stress Scale - 21 Items (DASS-21) is a self-reported measure to capture common symptoms of depression, anxiety, and overall stress for parents. The DASS-21 has been implemented in Latin America & the Caribbean and exhibits adequate validity and psychometric properties. We will use the Depression and Stress scales

2. Parents’ early stimulation: The Family Care Indicators (FCI) have been used extensively across low- and- middle-income countries and South America, particularly in impact evaluations and large-scale household surveys (such as UNICEF’s Multiple Indicators Cluster Survey). The FCI will allow us to characterize caregivers’ engagement in stimulation and learning activities, such as playing, singing, or telling stories. To complement the FCI, specifically focusing on positive stimulation related to the non-cognitive development of the child, we add a few items from the Warmth and Support domain of the PSDQ (Oliveira et al., 2018).

3. Parents’ discipline behaviors: The CTSPC is a parent report measure of their discipline behaviors, originally developed and validated in the US (Straus et al., 1998) and also validated in Brazilian (Bonfim et al., 2011; Reichenheim & Moraes, 2006). Specifically, we will use a subset of the CTSPC, including some items in the Minor Physical Assault and Psychological Aggression domains, which are related to physical and psychological discipline behaviors relatively common in Brazil.


4. Parenting self-efficacy: The SEPTI-TS was originally developed in the U.S. (Coleman & Karraker, 2003) and has undergone validation for diverse populations, including the Dutch population (van Rijen et al., 2014) and the French-speaking population (Meunier & Roskam, 2009). We decided to use a subset of the Self-efficacy for Parenting Tasks Index-Toddler Scale (SEPTI-TS; Coleman & Karraker, 2003). Specifically, we use items from the Nurturance/Valuing/Empathetic Responsiveness, Play, and Teaching domains.

The project will conduct a randomized controlled trial and a multifactorial experiment, gathering two points of data for the evaluation (baseline and follow-up).
Participants will include parents (or primary caregivers) of children under age 30 months who are participating in the national home visiting program for socioeconomically vulnerable families (Criança Feliz) in the state of Ceará. We will recruit families for participation in this study using a government administrative database for vulnerable families with children under age three (i.e., the Cadastro Único). Staff from Criança Feliz will recruit potential participants.
We planned this trial with the objective of assessing impact in real implementation. As such, we randomized in advance to baseline, when informed consent was collected by trained enumerators. We had initially planned to include 3,200 families and 290 home visitors, but we obtained consent from 1,599 families receiving services from 281 home visitors nested in 200 Centros de Referencia de Asistencia Social (CRAS) from 41 municipalities. As such, our sample comprises 1,599 families and 281 home visitors.
We randomized the 290 home visitors to a treatment group (which will receive the Afini intervention plus normal Criança Feliz services; 60%) or a control group (which will receive normal Criança Feliz services only; 40%). The randomization was conducted in Stata 17 MP.
Subsequently, consistent with a multifactorial design, we re-randomized the home visitors in the treatment group according to the following factors (there are 8 unique conditions, reflecting the different combinations of these factors):
1. delivery through a standalone app vs. WhatsApp messenger;
2. sending alerts once vs. twice per week;
3. customizing content to caregiver preference vs. children’s developmental needs.

The project will conduct a randomized controlled trial and a multifactorial experiment, gathering two points of data for the evaluation (baseline and follow-up).
Participants will include parents (or primary caregivers) of children under age 30 months who are participating in the national home visiting program for socioeconomically vulnerable families (Criança Feliz) in the state of Ceará. We will recruit families for participation in this study using a government administrative database for vulnerable families with children under age three (i.e., the Cadastro Único). Staff from Criança Feliz will recruit potential participants.
We planned this trial with the objective of assessing impact in real implementation. As such, we randomized in advance to baseline, when informed consent was collected by trained enumerators. We had initially planned to include 3,200 families and 290 home visitors, but we obtained consent from 1,599 families receiving services from 281 home visitors nested in 200 Centros de Referencia de Asistencia Social (CRAS) from 41 municipalities. As such, our sample comprises 1,599 families and 281 home visitors.
We randomized the 290 home visitors into two groups within municipality strata: a treatment group (which will receive the Afini intervention plus normal Criança Feliz services; 60%) and a control group (which will receive normal Criança Feliz services only; 40%). The randomization was conducted in Stata 17 MP.

Subsequently, consistent with a multifactorial design, we re-randomized the home visitors in the treatment group according to the following factors (there are 8 unique conditions, reflecting the different combinations of these factors):
1. delivery through a standalone app vs. WhatsApp messenger;
2. sending alerts once vs. twice per week;
3. customizing content to caregiver preference vs. children’s developmental needs.

Randomization done in office by a computer, using Stata 17 MP

Home visitor

Yes

Planned: 290 home visitors. Effective: 281 home visitors

Effective sample: 1,599 caregiver-child dyads. Our sampling frame included 3,200 caregiver-child dyads linked to 290 home visitors. The planned sample assumed that 50% (1600) would provide consent to participate in the study. Our effective sample, after requesting consent is 1,599 caregiver-child dyads

164 home visitors in the treatment group and 117 home visitors in the control group

Planned sample: MDES=0.24 SD, assuming 10% attrition, 50% of families provide consent, 30% takeup, alpha=0.05; power=0.80; ICC=0.04 Effective sample: MDES=0.34 SD, assuming 20% attrition and 30% takeup, alpha=0.05; power=0.80; ICC=0.04

Analysis Plan_0607