Home visiting at scale: impact evaluation of SAF-Cuna Más in Perú

Last registered on December 21, 2015

Pre-Trial

Trial Information

General Information

Title
Home visiting at scale: impact evaluation of SAF-Cuna Más in Perú
RCT ID
AEARCTR-0000988
Initial registration date
December 21, 2015

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
December 21, 2015, 10:45 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Primary Investigator

Affiliation
Inter-American Development Bank

Other Primary Investigator(s)

PI Affiliation
Inter-American Development Bank
PI Affiliation
Inter-American Development Bank

Additional Trial Information

Status
On going
Start date
2013-04-01
End date
2017-06-01
Secondary IDs
Abstract
Psychosocial stimulation delivered to poor, disadvantaged children and their mothers through home visits, has been shown to have large and sustained impacts on child development and later outcomes in life. This existing evidence comes mainly from small efficacy trials and a limited number of efforts to implement this type of intervention at a pilot scale. The current research project aims at understanding what magnitude of impacts can be obtained when an intervention that delivers psychosocial stimulation services through home visits to children 0-3 years of age in poor rural districts of Peru is rolled-out and implemented at large scale as a national program.

The Programa Nacional Cuna Más (Cuna Más) was created in Peru in 2012. One of its two services, Servicio de Acompañamiento a Familias (SAF), was designed based on the successful and well-evaluated Jamaica parenting intervention. In two years, it was scaled-up to reach 60 thousand children.

Our aim is to understand the effects of this intervention on child development, on parenting practices, and on the quality of the home environment (play materials and play activities). We also aim at understanding the extent to which heterogeneity in the magnitude of such effects is associated to the quality of the home visits being delivered. Our research used the gradual roll-out of SAF to implement a randomized-control trial in 180 districts (municipalities) throughout 12 departments in Peru. The results of this study will inform on the magnitude of the impact of a psychosocial stimulation delivered at scale through home visits. They will also be informative of key elements of home visiting quality that are most associated to that impact and that need to be consolidated when taking this type of intervention to scale.
External Link(s)

Registration Citation

Citation
Araujo, Maria Caridad, Marta Rubio-Codina and Norbert Schady. 2015. "Home visiting at scale: impact evaluation of SAF-Cuna Más in Perú." AEA RCT Registry. December 21. https://doi.org/10.1257/rct.988-1.0
Former Citation
Araujo, Maria Caridad, Marta Rubio-Codina and Norbert Schady. 2015. "Home visiting at scale: impact evaluation of SAF-Cuna Más in Perú." AEA RCT Registry. December 21. https://www.socialscienceregistry.org/trials/988/history/6430
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
SAF- Cuna Más is a home visiting service that aims to promote psycho-social stimulation through improved interactions between mothers or principal caregivers and their children with the aim of achieving better child outcomes. The program follows a structured weekly curriculum based on the child’s age and developmental stage. The delivery of this service occurs through weekly one-hour home visits that are carried out by a community member who has been trained by the Program. For every ten home visitors, the Program has one supervisor who is in charge of pre-service and in-service training, of mentoring and of helping with the planning of individual visits. Supervisors are required to have higher levels of education than home visitors.

Since July 2013, the service is being rolled out in poor, rural districts of Peru. Its gradual expansion allowed for the design of a randomised control trial on a sample of approximately 5,869 children (households) to evaluate it.

Data on child development outcomes and detailed socioeconomic household information were collected at baseline, before the program implementation started. A follow-up assessment of children’s development and a household survey was conducted slightly after two years. Children included in the baseline sample were 1-24 months of age. Given that the SAF serves children until 36 months of age, at follow-up, a proportion of the children in the baseline sample will have graduated from the program.

The original evaluation design was aimed at comparing the following groups:
1. A group receiving weekly home visits (T1).
2. A group receiving weekly home visits and bi-weekly group meetings conducted by Program supervisors. The idea was to offer families a space to interact with one another. Home visitors would also learn from seeing their supervisors conducting the sessions and interacting with the families (T2).
3. A control group, receiving no intervention.
Due to an underestimation of the workload of Program supervisors, it was not feasible for the Program to carry out the bi-weekly group meetings on a regular basis. These were cancelled shortly after baseline and therefore groups T1 and T2 ended up receiving virtually the same intervention. Hence, they were consolidated into one treatment group (T).
Intervention Start Date
2013-07-01
Intervention End Date
2017-06-01

Primary Outcomes

Primary Outcomes (end points)
Key Final Outcomes
Children’s cognitive, language, motor, and personal-social development.

Key Intermediate Outcomes
Child rearing practices: time and resources for child stimulation in the home.

Key Process Outcomes
Additionally, we will be measuring the quality of the home visits, their contents and their characteristics.
Primary Outcomes (explanation)
Key Final Outcomes
1. Children’s cognitive, language and motor development
We assessed cognitive, language, motor, and personal-social development at the time of the follow-up survey using the Spanish version of the third edition of the Ages and Stages Questionnaire, ASQ-3, adapted to the Peruvian context.

In addition, a subsample of children from both treatment and control groups were assessed with the third version of the Bayley Scales of Infant and Toddler Development (Bayley-III), suitably adapted for the context. We administered the cognitive, language, and fine motor scales of the Bayley-III.

Key Intermediate Outcomes
1. The presence of toys and learning materials in the house was assessed together with parental involvement with the child, the child’s routines and organisation of the child’s time inside and outside the family house. This was assessed using the Family Care Indicators, developed by UNICEF, and selected items of the Responsivity subscale of the Home Observation for the Measurement of the Environment (HOME).
2. We collected information on the mother’s knowledge of nutrition and stimulation, and her perceptions regarding the importance of these for children’s development through a short selection of items from the Knowledge of Infant Development (KIDI).

Key Process Outcomes: Quality of home visits
1. Aspects of process quality of the home visits (Home Visitor Practices and Engagement) will be measured using the Home Visit Rating Scale, HOVRS A+V2.1.
2. Aspects of the contents covered during the home visits and other structural characteristics, such as length and frequency will be measured using a checklist designed by the research team for this project and will be supplemented with program administrative records.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Randomization was carried out at the level of the municipalitiy (distrito) and using pairwise matching. From a list of all municipalities and villages (centros poblados) in Peru that were eligible for SAF, we eliminated those villages that did not have a minimum number of children 0-24 months old, based on the 2007 Census and the SISFOH household roster used for targeting the conditional cash transfer program. This left a roster of 531 municipalities that were ranked based on their poverty level (based on INEI’s 2009 poverty map). Based on this ranking, municipalities were organized in groups of three and 60 of these trios were randomly selected. Within each group of three municipalities, each one municipality was randomly allocated into T1, T2 and C. Within each municipality, the two villages with the largest number of children 0-24 months of age were selected. During the field work and prior to the survey administration, all households in these villages were registered in a village-level roster. From those with children younger than 24 months of age, 15 households per village were selected at random to be in the baseline survey.

Hence, the final evaluation sample consists of 360 villages, 120 randomly assigned to C and 240 randomly assigned to T (T1+T2).

At baseline, the total sample size was 5,869 children in 5,620 households from these 360 villages in 180 municipalities in 12 provinces of Peru. All children were 1-24 months of age.

Of these children, a subsample was assessed at follow-up using the Bayley-III. Given practical constraints, it was unfeasible to administer the Bayley-III in all municipalities in the sample. Hence, we selected the subsample of municipalities in which to administer the Bayley-III as follows. First, we dropped the 40 villages initially assigned to T, where the Program decided not to intervene at a later stage; as well as 1 village initially assigned to C which ended up receiving the intervention. To preserve the experimental design, we also dropped the matching villages in the trios to which these 41 villages belonged. Second, and given the difficulties and inability of the research team to have the Bayley-III translated and adapted into any indigenous languages, we excluded all villages where more than 30% of the baseline surveys were administered in a language other than Spanish. As before, in order to preserve the experimental design, all villages in the matching trio were dropped. Lastly, we excluded 6 villages (1 in Amazonas, 2 in Cuzco and 3 in Loreto) that were very disperse and hard to reach, together with the matching villages in their corresponding trios.

Moreover, within these municipalities, we identified the subsample of children estimated to be younger than 42 months of age at follow-up, given that the test is designed to assess children up this age.

Hence, the Bayley-III subsample includes 1,492 children who will be younger than 42 months old at follow-up in the 158 villages satisfying the criteria above (109 in T and 49 in C).

In all treatment (T) villages of the Bayley-III subsample, we also filmed a sample of home visits carried out by the home visitors who had been assigned to the children in T. We scored these videos using the HOVRS A+V2.1 and a short checklist we developed to measure the content and other quality features of the home visits, to all home visitors who work with the children in the Bayley-III sample (this is, 109 villages in 71 municipalities). For each home visitor (of a child in the Bayley-III subsample), we collected these measures during a home visit for each child in the Bayley-III subsample and any other child the home visitor regularly serves, on average 3 per home visitor.
Experimental Design Details
Randomization Method
Randomization was done in office by a computer using the statistical software Stata.
Randomization Unit
Randomization (treatment assignment) was carried out at the municipality (distrito) level.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
Treatment will be clustered at the municipality level, hand in hand with the choice of unit of randomization.
180 municipalities (clusters), 120 treatment and 60 control.
Sample size: planned number of observations
At baseline, the total sample size was 5,869 children in 5,620 households from 180 municipalities and 360 villages in 12 provinces of Peru. All children were 1-24 months of age.
Sample size (or number of clusters) by treatment arms
60 municipalities (distritos) and 120 villages in each treatment group (T1, T2) and 60 municipalities and 120 villages in control group.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Assuming a power of 80%, a significance level of 5% for our estimates, perfect compliance, no attrition, and an intra-cluster correlation (ICC) of 0.05, our sample size was designed to allow us to identify minimum detectable effects of 0.15 SD on the ASQ-3. The intra-cluster correlation (ICC) from ASQ-3 at baseline, controlling for household wealth and demographics, was 0.08. Following the same assumptions as above and using this updated ICC, we estimated that the subsample of 1,492 children who will be assessed on the Bayley-III, will allow us to identify a minimum detectable effect of 0.22 SD on the Bayley-III (or 0.229 SE assuming 7% attrition).
IRB

Institutional Review Boards (IRBs)

IRB Name
Universidad Peruana Cayetano Heredia, Vicerrectorado de Investigación, Dirección Universitaria de Investigación, Ciencia y Tecnología (DUICT), Comité Institucional de Ética
IRB Approval Date
2015-07-15
IRB Approval Number
15004

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials