Optimal Design of Digital Early Childhood Development Interventions: A Multifactorial Randomized Control Trial

Afini (i.e., the intervention) is an interactive “virtual assistant” that uses a chatbot to recommend simple, fun, age-appropriate activities that parents and other caregivers can complete with young children to support child development. Afini includes 1,500 activities that have been delivered to 70,000 users in Latin America. Afini’s activity recommendations are evidence-based and reflect principles of behavioral economics (e.g., by providing scientific rationale for the activities). Although they are designed to be “culturally neutral,” Afini activities can be adapted to reflect materials, games, and concepts that are most relevant to a given context or program. Activities focus largely on supporting children’s language, cognitive, motor, and social-emotional development.

We will test the benefits of Afini as an add-on to the Criança Feliz national home visiting program (relative to a control group that will keep receiving the Criança Feliz program as usual) and the relative benefits of specific Afini features, focusing on tradeoffs for cost-effectiveness. Specifically, we will examine:
1. delivery through a standalone app (a lower-cost alternative that does not rely on third-party fees) vs. WhatsApp messenger (an existing provider trusted by families, which may improve engagement);
2. sending alerts (i.e., unsolicited activity recommendations) once (less expensive) vs. twice (more intensive) per week;
3. customizing content to caregiver preference (may improve engagement) vs. children’s developmental needs (may lead to greater child-level impact).

Afini seeks to promote young children’s healthy development by supporting these day-to-day interactions that serve as the basis of nurturing care. Our theory of change is that parents (or other caregivers in the home) will engage with Afini and complete recommended activities with children. These processes will lead to improvements in parents’ knowledge, provision of stimulation, and mental health, which will improve the quality of their interactions with children as well as children’s ECD outcomes (i.e., motor, cognitive, language, and social-emotional skills). We hypothesize that Afini will benefit families already receiving home visiting because, unlike Criança Feliz (which does not currently have a defined curriculum), Afini provides concrete, specific, and customized activity recommendations designed specifically for vulnerable families (e.g., minimal text, few required materials). Furthermore, Afini can be used at families’ convenience, improving accessibility.

2023-04-15

2024-01-19

1. Child Development: We will use the Caregiver-Reported Early Development Instruments - Long Form (CREDI).

The CREDI is a parent-reported tool to measure child development using simple yes/no questions that has been used and validated in multiple countries, including Brazil, and demonstrates adequate psychometric properties to measure with precision motor, cognitive, language, and social-emotional development. We may also include additional measures of the child development if we receive additional funding to do so. These measures will focus on the same domains of child development, but will use direct assessment methods (to complement the parent-report approach of the CREDI).

1. Caregiver mental health: The Depression, Anxiety and Stress Scale - 21 Items (DASS-21)
2. Home learning environment: Family Care Indicators
3. Parenting knowledge, attitudes, and beliefs

1. Caregiver mental health: The Depression, Anxiety and Stress Scale - 21 Items (DASS-21) is a self-reported measure to capture common symptoms of depression, anxiety, and overall stress for parents. The DASS-21 has been implemented in Latin America & the Caribbean and exhibit adequate validity and psychometric properties
2. Home learning environment: The Family Care Indicators (FCI) has been used extensively across low- and- middle- income countries and South America, in particular, in impact evaluations and large-scale household surveys (such as UNICEF’s Multiple Indicators Cluster Survey). The FCI will allow us to characterize caregivers’ engagement in stimulation and learning activities, such as playing, singing, or telling stories, and also the home learning environment, including the availability of books, toys, and playthings. We may also include additional measures of home learning environment if we receive additional funding to do so. These measures will focus on similar dimensions of home learning, but will use observational methods (to complement the parent-report approach of the FCI).
3. Parenting knowledge, attitudes, and beliefs: We will use a set of questions that have been previously used in the context of South America in impact evaluations in the region and has been validated and shown to have adequate psychometric properties. The instrument is intended to measure caregiver knowledge, attitudes, and beliefs on basic developmental processes (e.g., timing of developmental milestones) and basic parenting practices.

The project will conduct a randomized controlled trial and a multifactorial experiment, gathering two points of data for the evaluation (baseline and follow-up).
Participants will include parents (or primary caregivers) of children under age 30 months who are participating in the national home visiting program for socioeconomically vulnerable families (Criança Feliz) in the state of Ceará. We will recruit families for participation in this study using a government administrative database for vulnerable families with children under age three (i.e., the Cadastro Único). Staff from Criança Feliz will recruit potential participants.
We planned this trial with the objective of assessing impact in real implementation. As such, we randomized in advance to baseline, when informed consent was collected by trained enumerators. We had initially planned to include 3,200 families and 290 home visitors, but we obtained consent from 1,599 families receiving services from 281 home visitors nested in 200 Centros de Referencia de Asistencia Social (CRAS) from 41 municipalities. As such, our sample comprises 1,599 families and 281 home visitors.
We randomized the 290 home visitors to a treatment group (which will receive the Afini intervention plus normal Criança Feliz services; 60%) or a control group (which will receive normal Criança Feliz services only; 40%). The randomization was conducted in Stata 17 MP.
Subsequently, consistent with a multifactorial design, we re-randomized the home visitors in the treatment group according to the following factors (there are 8 unique conditions, reflecting the different combinations of these factors):
1. delivery through a standalone app vs. WhatsApp messenger;
2. sending alerts once vs. twice per week;
3. customizing content to caregiver preference vs. children’s developmental needs.

The project will conduct a randomized controlled trial and a multifactorial experiment, gathering two points of data for the evaluation (baseline and follow-up).
Participants will include parents (or primary caregivers) of children under age 30 months who are participating in the national home visiting program for socioeconomically vulnerable families (Criança Feliz) in the state of Ceará. We will recruit families for participation in this study using a government administrative database for vulnerable families with children under age three (i.e., the Cadastro Único). Staff from Criança Feliz will recruit potential participants.
We planned this trial with the objective of assessing impact in real implementation. As such, we randomized in advance to baseline, when informed consent was collected by trained enumerators. We had initially planned to include 3,200 families and 290 home visitors, but we obtained consent from 1,599 families receiving services from 281 home visitors nested in 200 Centros de Referencia de Asistencia Social (CRAS) from 41 municipalities. As such, our sample comprises 1,599 families and 281 home visitors.
We randomized the 290 home visitors to a treatment group (which will receive the Afini intervention plus normal Criança Feliz services; 60%) or a control group (which will receive normal Criança Feliz services only; 40%). The randomization was conducted in Stata 17 MP.
Subsequently, consistent with a multifactorial design, we re-randomized the home visitors in the treatment group according to the following factors (there are 8 unique conditions, reflecting the different combinations of these factors):
1. delivery through a standalone app vs. WhatsApp messenger;
2. sending alerts once vs. twice per week;
3. customizing content to caregiver preference vs. children’s developmental needs.

Randomization done in office by a computer, using Stata 17 MP

Home visitor

Yes

Planned: 290 home visitors. Effective: 281 home visitors

Effective sample: 1,599 caregiver-child dyads. Our sampling frame included 3,200 caregiver-child dyads linked to 290 home visitors. The planned sample assumed that 50% (1600) would provide consent to participate in the study. Our effective sample, after requesting consent is 1,599 caregiver-child dyads

164 home visitors in the treatment group and 117 home visitors in the control group

Planned sample: MDES=0.24 SD, assuming 10% attrition, 50% of families provide consent, 30% takeup, alpha=0.05; power=0.80; ICC=0.04 Effective sample: MDES=0.34 SD, assuming 20% attrition and 30% takeup, alpha=0.05; power=0.80; ICC=0.04