Evaluation of Parental Engagement SMS Intervention on GiveDirectly beneficiaries in Lilongwe Rural, Malawi

Last registered on August 09, 2022

View Trial History

Pre-Trial

Trial Information

General Information

Title

Evaluation of Parental Engagement SMS Intervention on GiveDirectly beneficiaries in Lilongwe Rural, Malawi

RCT ID

AEARCTR-0009879

Initial registration date

August 03, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

August 09, 2022, 4:31 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country

Malawi

Region

Lilongwe

Primary Investigator

Name

Jack Thunde

Affiliation

IDinsight

Contact Primary Investigator

Other Primary Investigator(s)

PI Name

Mtise Mwanza

PI Affiliation

IDinsight

Contact Investigator

PI Name

Valentina Brailovskaya

PI Affiliation

IDinsight

Contact Investigator

Additional Trial Information

Status

On going

Start date

2022-04-25

End date

2023-03-31

Keywords

Education

Additional Keywords

Nudges, SMS-Intervention

JEL code(s)

I2, N37

Secondary IDs

Prior work

This trial does not extend or rely on any prior RCTs.

Abstract

IDinsight, GiveDirectly and Movva are partnering to rigorously evaluate a Movva SMS-based intervention aimed at increasing parental engagement to improve schooling outcomes for children of GiveDirectly recipients in Lilongwe Rural. The objective of this study is to assess the impact of the SMS intervention on children’s attendance, time spending, learning, motivation to go to school and parental engagement with children.

Several relevant studies suggest that the intervention may be a promising solution to improve schooling outcomes in Malawi. SMS interventions for parents have been shown to create literacy gains for preschool children (York et al. 2019, Doss et al. 2019) and help parents track school progress (Bergman 2019). Increased parental engagement in their children's education through games and positive reinforcement has been shown to improve economic, psychological, and socio-emotional outcomes many years later (Gertler et al. 2021, Walker et al. 2021). Angrist et al. 2021 demonstrate that SMS messages with learning activities improve learning by 0.12 SD in Botswana. The intervention used by Movva has been evaluated in Brazil and Cote D'Ivoire. In Brazil, the authors found that general text messages that increase the salience of children's outcomes bring about increases in attendance, test scores, and promotion rates (Cunha et al., 2017). The largest impacts are found for SMSs sent three times a week without requiring parents to respond. Delivery times or consistency of delivery times do not have differential impacts.

The SMS intervention, developed by Movva, will be implemented in TA Khongoni over a 4 to 6 months period. The SMSs will consist of multiple education subthemes to be covered in sequence over weeks. Each subtheme has a sequence of 4 messages, with 2 messages being sent to parents weekly. The first message sent in the first week of a sequence will contain a fact and the second message will contain an activity for the parent to do with their child. In the second week, the first message will be interactive with a question for the parent to respond to and the final message of the sequence will reinforce the general message of the sequence.

The main objective of the study is to understand how SMSs affect parental engagement, school attendance and child use time.

External Link(s)

Registration Citation

Citation

Brailovskaya, Valentina, Mtise Mwanza and Jack Thunde. 2022. "Evaluation of Parental Engagement SMS Intervention on GiveDirectly beneficiaries in Lilongwe Rural, Malawi." AEA RCT Registry. August 09. https://doi.org/10.1257/rct.9879-1.0

Sponsors & Partners

Experimental Details

Interventions

Intervention(s)

Movva created an SMS intervention that uses nudges to to increase parental attention to their
children. In cases where refocusing attention is the key driver, as is the case in parental involvement in
children’s education, salient interventions such as this are bound to have larger effects than information
provision (such as general importance of education). Informational interventions are constrained by the
frequency at which information is available (often at low frequency in developing countries), while
nudging and reminding parents about engagement with children can be implemented at much higher
frequency.

The theory of change (TOC) for Movva’s SMS-based intervention is as follows. Guardians will receive
two weekly messages to encourage them to engage in their child’s education with the objective that their
child will enjoy a better life. In the short term, these messages will provide awareness of the importance
of engaging with their children, suggestions of activities parents can do with their children and basic
school information. This knowledge will lead caregivers to engage in activities with their children, pay
more attention to their children’s school life, and value their child’s commitment to schooling. In the
medium-term, children will feel supported, leading to improved socio-emotional skills. Caregivers will also
motivate their children to improve their school performance, resulting in more dedication and
commitment by the children to school activities. These effect channels combined will result in improved
school performances, more diligent students and students who complete their secondary education.
Lastly, in the long-term, the quality education received by these children will result in educated young
people who can access better job opportunities and act as responsible citizens in their communities, with
a higher quality of life.

IDinsight, GiveDirectly and Movva are partnering to rigorously evaluate a Movva SMS-based intervention aimed at increasing parental engagement to improve schooling outcomes for children of GiveDirectly recipients in Lilongwe Rural. The objective of this study is to assess the impact of the SMSs on child attendance and enrollment and non-school activities like child labour and parental aspirations for their children.

Intervention Start Date

2022-08-01

Intervention End Date

2022-12-31

Primary Outcomes

Primary Outcomes (end points)

School attendance

Primary Outcomes (explanation)

We considered two ways of collecting information on attendance (1) school records which we believe are accurate, but likely subject to missing data (2) children’s reports which may be less accurate but we have greater assurance that the data will be more complete.
School records:
In our pilot study, we were unable to find most children’s records within schools. Qualitative evidence suggests that it’s driven by 1) lack of parental knowledge of which grade the children belong to 2) mismatch in names given to the research team by parents vs those that are used in schools. At endline, we will devise a number of strategies that will improve our ability to find child records (such as looking for children across grades), however, it’s still possible that children will not be found in school records and attendance data will have many missing observations.
Children reports about attendance yesterday
Collecting information on child attendance may be subject to reporting bias since some of the children will be very young, however, we believe that children’s report on attendance to be the most reliable for impact quantification given constraints. First, unlike the school attendance data, we expect the child report data to be more complete since we have been successful at interviewing children during piloting. Second, parental reports may not be a reliable source of this information since previous (unpublished) research suggests that the intervention changes parental awareness of children’s lives, therefore, they will know more about their lives compared to control parents.
Unfortunately due to project and intervention time constraints, some of our phone survey data collection will fall on exam/holidays during which the school will be closed. If that is the case, we will ask the child to think of the last day school was opened and record whether he/she was present on the last day.
At the analysis stages, we will construct an index of the following 2 variables:
Whether a child reports attending school yesterday (or the last day when the school was opened).
Percentage of days (out of 5) child attended school in the week of interest (according to school records)
We will construct an Anderson index (Anderson 2008) out of these two variables to create a final variable for the analysis. If the child is not enrolled, attendance will be set to zero. If the observations are missing for one of the outcomes, only 1 variable will be used. The determination of whether the intervention is successful or not will be made based on the estimate on this variable. The intervention will be considered successful if the estimate has a pvalue smaller than 0.1
The supplementary tables (where the impact estimates on 2 outcomes will be examined separately) will be used to assess the magnitude of the effect.

Secondary Outcomes

Secondary Outcomes (end points)

Child time use
Child-parent engagement
Learning outcomes
Children motivation
Children re-enrollment

Secondary Outcomes (explanation)

See PAP for construction of the outcome

Experimental Design

IDinsight will estimate the impact of the SMS intervention through a randomised control trial (RCT) across ~2600 GiveDirectly beneficiary households with children of school-going age. The RCT will take place in Lilongwe Rural, TA Khongoni. The estimated intervention impact will be measured after comparing outcomes between two groups: 1) Treatment - those that will receive SMS interventions 2) Control - those that will not receive SMS interventions.

Experimental Design Details

Randomization Method

Randomization was done in an office by a computer using Stata v17.

Randomization Unit

Household

Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters

N/A

Sample size: planned number of observations

2210 households after non-response (2600 before accounting for non-response)

Sample size (or number of clusters) by treatment arms

1300 households control, 1300 households treatment

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

Child attended school yesterday - 0.05 (MDE), 0.43 (SD), 6% (Percentage MDE) Number of days attended (out of 5) - 0.07 (MDE), 0.72 (SD), 2% (Percentage MDE) Child time use (hours) - 0.24 (MDE), 2.28 (SD), 13% (Percentage MDE) Child-parent engagement (SD) - 0.10 (MDE), 0.98 (SD), N/A (Percentage MDE)

Supporting Documents and Materials

IRB