Evaluating Learning Interactions (ELI)

The ELI intervention will send 3–4 text messages per week to parents in two treatment groups and one control group. One treatment will send parents messages promoting unconstrained learning interactions, such as guides to open-ended conversation and play that parents can do with their children. The second treatment will send parents messages promoting constrained learning interactions, such as direct instruction on literacy skills where the goal is a correct response from children. Control messages will be unrelated to promoting children's skill development. We focus only on literacy skills so that we can concentrate messaging on only one skill to maximize impact and minimize the length of the assessments. Each treatment will be delivered over six months by text messages in Spanish and English.

Treatment 1: Academic learning. Messages in this group will provide parents with a new vocabulary word and its definition and will prompt parents to teach the child that word. An example of such a message is: “[CHILD NAME] learns from you each day! Share today's word with [CHILD NAME]. A SHIP is a very big boat that can carry people or things across the sea. Ask [CHILD NAME]: What is a ship? Is a ship big or small?”

Treatment 2: Curiosity-based learning. Parents in this group will receive messages with the same vocabulary word as the parents in Treatment 1, but the messages will guide parent-child conversation about the word, encouraging abstract and open-ended interaction. An example of this kind of messaging is “[CHILD NAME] learns from you each day! Today, tell [CHILDNAME]: A ship is a very big boat that can carry people or things across the sea. Ask [CHILD NAME]: Have you been on a SHIP? What do you think it would be like to be on a ship? Where would you like to go on a ship?”

Control: Placebo messages. The control group will receive messages about topics unrelated to parent-child learning interactions.

Throughout both treatments we will occasionally send parents “engagement prompts.” An engagement prompt is a request for parents to engage their child in an activity or a conversation and then text back a response indicating that the task was completed.

We will also employ additional strategies to boost and maintain parents’ engagement with the program. In both treatment groups we will use parent identity salience framing for many of the messages because, as noted above, this framing improved parent involvement in a prior text-based parent engagement program that we evaluated. An example of identity salience framing is this: “Be the parent you want to be for [CHILD NAME]!” This is then followed by the message particular to the treatment group of the parent.

In both treatment groups, we will also provide inducements throughout the program to boost attention to the messages. These inducements can include texting back to parents when they respond to an engagement prompt, providing recognition to parents who respond, and so on. These will be the same for both treatment groups.

2023-05-01

2024-01-31

We will assess the change in children’s literacy skills and their change in curiosity about future learning. Reliable administrative data for preschool-aged children’s skill development does not exist, and this study will include some children who are not enrolled in formal preschool; therefore, we will have to assess children’s skills as part of ELI. Our research team has extensive experience conducting child skill assessments.

At baseline and follow-up we will briefly survey parents on demographic characteristics, attitudes toward learning interactions at home with their child, and investments in their child’s school readiness. In the survey at the end, we will ask for user feedback on their experience with the ELI messaging program. We will also look at differences in the rate of parent feedback to periodic engagement prompts.

We will recruit low-income families with preschool-aged children in collaboration with our implementation partners. All recruited parents will be offered half of their incentive payment upon enrollment in ELI and the other half upon completion of the post-intervention parent survey and child assessments. Consent will be actively obtained from participants and contact information and demographic data will be collected upon enrollment. Enrolled parents will be randomized into one of the two treatment groups or the control group. Baseline parent surveys and child assessments of literacy and curiosity will be administered just prior to the start of the intervention. As part of the 6-month intervention, parents will receive 3-4 text messages per week in their native language of English or Spanish. Periodically parents will be asked to text back a response indicating they are engaged with the ELI program. At the end of the intervention, endline parent surveys and child assessments will be administered.

Consented and enrolled parents will be randomized using a computer program i.e. Stata.

Randomization will be at the family level.

No

750 families

750 families

250 families in treatment promoting unconstrained learning interactions; 250 families in treatment promoting constrained learning interactions; 250 families in control group

We set power to 80% and the significance level to p=.05. Under these assumptions the standardized minimum detectable effect size (MDSE) is 0.25 on literacy skills (mean difference of 3.75 with a SD of 15 on PPVT). Assuming the measure of curiosity has 8 chances to choose a novel or familiar item, the MDSE is 0.25 (mean difference of 0.33 with a SD of 1.33 given a normal distribution from 0 to 8). Furthermore, assuming the measures of curiosity contain measurement error with a reliability coefficient R of 0.75, the calculation of the attenuated effect size from Phillips and Jiang (2016) gives a MDSE of around 0.29.

ELI PAP 2023