Play to Learn - Mass Media Impact Evaluation - Colombia

The intervention will consist of sending Play to Learn (PtL) Watch, Play Learn video blocks that focus on developing socio-emotional learning and math skills for internally and forcibly displaced four year-old children. Contents and methodology have the potential to address the acute lack of education opportunities among children living in vulnerable contexts like migration or forced displacement.

2022-09-19

2023-01-31

Emotion recognition by 4-year-old children
Emotion identification by 4-year-old children
Self-regulation strategies of 4-year-old children
Early mathematics skills

Emotion recognition by 4-year-old children: Children will be able to identify and name emotions
Emotion identification by 4-year-old children: Children will be able to identify specific emotions
Self-regulation strategies of 4-year-old children: Children will be able to identify others’ and own feelings in the context of common social situations
Early mathematics skills: Children will be able to demonstrate increased levels of a composite of early mathematic skills, including of (i) counting - verbal number production of 1 to 10 Items, verbal counting, what comes next,, (ii) shape discrimination - nonverbal identification and sorting, (iii) Position and location of things in relation to other objects; (iv) measurement terms (height, width)

Whether effects of the intervention on primary outcomes are moderated by home environment (perceived refugee environment index). caregiver characteristics (caregiver’s education level, mental health, , thnic/immigrant origin), or child gender and age

Spatial relation (hiding, left/right) questions that extend beyond the primary spatial relations outcomes (inside/outside, between, over/under)
Counting/cardinality are a spillover of the primary outcomes

The project will be designed as a randomized experiment to measure the impact of Watch, Play Learn video blocks targeting four-year-old children. The RCT intervention will last 19 weeks and will be conducted in Barranquilla and Soledad.

The sample will consist of 1,000 households that will be randomly assigned to a treatment (500 households) and a control group (500 households). Households will be Venezuelan migrants living in Colombia and Colombian nationals living in 16 pre-identified with a high concentration of Venezuelan migrant population.

The recruitment and baseline data collection will be conducted in-person and the neighborhood leaders will be informing all the community about the recruitment activity based on WhatsApp messages previously designed by the research team (see marketing section below).

If the caregiver and the child are eligible and accepted to be part of the study, we will proceed with the baseline questionnaires. The caregiver’s survey will inquire about caregivers’ basic information on household and income; education; socioeconomic status as well as mental health status. Meanwhile, the child’s survey will assess children's SEL and mathematics-skills A baseline cost survey will also be conducted which will try to gauge utilization of resources by caregivers which are relevant to the study context (such as mobile phone, money spent on internet, etc. ) and the time spent and resources employed by them to engage with their children through different academic or non-academic activities.
The unit of observation will be at the household level. Thus, after recruitment and baseline we will randomize households into treatment and control groups to have equal sample sizes. The study is powered to detect a minimum effect of 0.175 sd.

During the 19 week intervention we will monitor metrics related to content access and usage using the deployment platform of video delivery.

We will conduct an endline data collection after the intervention is completed. This data collection will be in-person and the team will contact both caregivers and children that participated in the baseline. Topics of the endline will be similar to those in baseline including additional questions to conduct a cost effectiveness analysis.

The project will start with a pilot testing baseline measurement instruments and users’ experiences with contents. The pilot will be followed by a full RCT.
Recruitment and Baseline
For the recruitment and baseline data collection, we will set up stations in different strategic points of the pre-identified neighborhoods for caregivers to approach and register for the study. Community leaders within the neighborhoods will help advertise the activities. If at the moment of registration the household is eligible to participate in the study, the data collection team will set up appointments to interview both caregivers and children. Only those households who complete both surveys will receive the incentives which are a kit for the child and a 15,000 COP gift card for the caregiver.

The stations will be carefully designed in order to provide an enabling environment for children to concentrate on the evaluation instruments and for enumerators to be able to interact with children and caregivers in a productive manner. We will aim to have open large spaces with individual tables for each child where they feel secure and can focus on the tasks presented by the enumerators. The stations will also have space for the caregivers so they can be present at all times.

The stations will be rotated whenever the registration rates decrease, we expect to rotate registration points every three days, and we expect to have two teams working simultaneously. The neighborhood leaders will be informing all the community about the recruitment activity based on WhatsApp messages previously designed by the research team (see marketing section below).

There will be three stages of interaction during the recruitment and baseline:


Screening. We will collect the initial information of the caregivers who are 18 years or older to assess whether they are eligible to participate in the study; we will inquire about nationality, access to a smartphone and to the internet. The screening short survey will have an approximate duration of ten minutes.
Consent. We will have a two-stage consent. First, before the screening, briefly explaining the consent and then asking the screening questions. Next, if the person is eligible, they will receive the rest of consent information. During the consent, we will ask about the interest to participate in the program; this consent implies both the consent to receive the materials and participate in the program if they are selected, and consent to provide information in the baseline survey. During consent, caregivers will need to provide assent on behalf of the child.
Baseline data collection. If the caregiver and the child are eligible and accepted to be part of the study, we will proceed with the baseline questionnaires. The caregiver’s survey will inquire about caregivers’ basic information on household and income; education, socioeconomic status as well as mental health status. Survey application will last around 30-40 minutes. Meanwhile, the child’s survey will ask about children's SEL and mathematics-skills related behavior, activities, and outcomes. Survey application will last around 45 minutes.

Endline
For the endline data collection, we will again set up stations in different strategic points of the neighborhoods for caregivers and children to approach. The stations will again be carefully designed in order to provide an enabling environment for children to concentrate on the evaluation instruments and for the enumerators to be able to interact with the children and caregivers in a productive manner. Any learnings from the recruitment and baseline data collection will be considered for the design of the endline stations.

There will be three stages of interaction during the endline:


Consent. We will ask participants for their consent to participate in the endline survey. Caregivers will need to provide assent on behalf of the child.
Endline data collection. If the caregiver and the child accept to participate in the endline survey, we will proceed with the questionnaires. Any learnings from the recruitment and baseline data collection will be considered for the questionnaire's design.

Target population
Venezuelan migrants and Colombian nationals living in vulnerable neighborhoods with high concentrations of Venezuelan migrants.
For the pilot, the population will be in one neighborhood of Santa Marta that has a high concentration of Venezuelan migrants
For the RCT, the population will be up to 16 neighborhoods of Barranquilla and Soledad that have a high concentration of Venezuelan migrants. The final number of neighborhoods, which depends on the success of securing the sample size of 1,000 households, will not affect the RCT design as the randomization is conducted at the household level.

Randomization will be done using the randtreat Stata command after the baseline phase.

Treatment will be randomized at the household level.

No

N/A

2,000 observations (1,000 caregivers and 1,000 children)

Treatment group: 500 caregivers and children aged 4
Control group: 500 caregivers and children aged 4.

Total sample size = 1000 households, accounting for 35% attrition at level 1 that is individual household level, MDES=0.175