SMS Reminders for Childhood Vaccination: Experimental evidence from Sierra Leone

Context: With the introduction of a malaria vaccine in April 2024 in all districts except Freetown city (Western Area Urban district), the vast majority of children in Sierra Leone are now due for 10 immunization visits: at birth (BCG, oral polio), 6 weeks (Penta 1+), 10 weeks (Penta 2+), 14 weeks (Penta 3+), 6-months (Malaria 1), 7-months (Malaria 2), 8-months (Malaria 3), 9-months (Measles 1, Yellow Fever), 15-months (Measles 2), and 18 months (Malaria 4). As of November 2024, children residing in Western Area Urban (Freetown) are not eligible for the malaria vaccine. These children continue to have a six-visit immunization schedule. Given the recent introduction of the malaria vaccine, there is uncertainty around if and when the malaria vaccine may be offered to children in Freetown. With this in mind, the current pre-registration plan assumes 10 immunization visits for children in Western Area Rural clinics and 6 immunization visits for Western Area Urban clinics.

Intervention: We will randomly assign caregivers to receive SMS reminders for vaccinating their children. These reminders serve to increase the salience of the upcoming vaccination appointment, help mitigate procrastination, and may provide new or forgotten information. Within all 30 clinics, one contact for each child born during the study period will be randomly assigned to receive two short, informative SMS reminders before the child is due for their Penta 1, (Malaria 1/ Vitamin A), Measles 1 and final (Measles 2 / Malaria 4) immunizations. We are collaborating with UNICEF to develop a RapidPro flow to send the messages.

An example message text is as follows: Dear [CAREGIVER NAME]. Your child [CHILD NAME] is due for a marklate visit. Come to the next marklate (vaccination) day at [CLINIC NAME] on [DAY OF WEEK, MONTH DAY, YEAR / TOMORROW].

Caregiver contact information will be collected at birth and up to Penta 3. IPA enumerators will train health workers at each clinic to collect babies’ contact information and birth records. This training will depend on the register that is already in place; in particular:

Western Area Rural: These clinics have the new U2 register to collect malaria doses and contact numbers. The training will therefore be brief and mainly to teach them how to inform caregivers about the intervention.

Western Area Urban: The training will teach them how to record contact numbers in the remarks/comments column as well as how to inform caregivers about the intervention.

Babies’ birth records and contact information will be collected at clinics for four months.

The SMS will be the only additional contact with caregivers in the treatment arm. Clinic staff and community health workers will conduct all their usual follow-ups in both treatment and control clinics. At the end of the study, we will interview a sample of caregivers about the SMS reminders. We will also survey the health workers delivering the intervention to learn about their beliefs and attitudes towards the task of recording contact information.

2024-11-29

2025-04-04

1. Number of vaccines received on time, defined as coming within 2 months of each vaccine’s due date. The total number of timely vaccines is 4, one for each of Penta 1, (Malaria 1/ Vitamin A), Measles 1 and final (Measles 2 / Malaria 4).
2. Number of vaccines by 20 months independent of timeliness.

The number of vaccines received on time captures the overall effects of our intervention on timely vaccination weighting every vaccine equally. The number of vaccines independent of timeliness captures the overall effects of our intervention on vaccination under a certain age, weighting every vaccine equally.

We will collect all vaccine outcomes through a digitization of the clinic registers where health workers record each child’s vaccination dates and personal information. We will also survey a random sample of caregivers across all study clinics and verify the clinic register data against children’s vaccination cards. Our data will be an individual-level dataset for each child containing their entire immunization history including their date of birth, the vaccines they received, and the date at which they received them.

Vaccination outcomes:
Number of measles doses by 20 months.
Completed vaccination on time, defined as coming within 2 months of a vaccine’s due date, for the each of the following vaccines:
Penta 1 due at 1.5 months
Penta 2 due at 2.5 months
Penta 3 due at 3.5 months
Malaria 1 due at 6 months (WAR only)
Malaria 2 due at 7 months (WAR only)
Malaria 3 due at 8 months (WAR only)
Measles 1 due at 9 months
Measles 2 due at 15 months
Malaria 4 due at 18 months (WAR only)
Completed vaccination under 20 months old for each of the following vaccines:
Malaria 1 (WAR only)
Malaria 2 (WAR only)
Malaria 3 (WAR only)
Measles 1
Measles 2
Malaria 4 (WAR only)

Other outcomes and descriptives:
Caregiver self-reported recall of SMS messages (for treated caregivers).
Send and receive rates for SMS messages (for treated caregivers).
Frequency and intensity of vaccination outreach activities at the clinic level.

Vaccination outcomes:
The effects on uptake rates of specific vaccines aims to help us understand the impact of the SMS messages.

Other outcomes:
Indications of receipt of and engagement with the SMS messages can serve as a first-stage for understanding the role of SMS messages in shifting vaccination behaviors.
The frequency and intensity of vaccination outreach activities helps us to understand whether the SMS messages crowd out other clinic engagement to promote vaccinations

30 Government clinics in Western Area Rural and Western Area Urban are assigned to the SMS treatment arms, and 30 are assigned to a pure control. Within all clinics in the SMS messaging arm, one contact for each child born during the study period will be randomly assigned to treatment or control.

Treatment: As described above, treated contacts receive two short, informative SMS reminders before the child is due for their Penta 1, (Malaria 1/ Vitamin A), Measles 1 and final (Measles 2 / Malaria 4) immunizations. Caregiver/infant pairs who enroll at the clinic only at Penta 1 will receive one fewer message.

Control: Control contacts receive no SMS messages, although their contact information is still collected.

The primary comparisons will be between treated and control contacts in the SMS messaging clinics. The pure control arm allows for a comparison of control contacts in the SMS messaging clinics to contacts in the pure control clinics, to assess spillovers.

Not available

Randomization done in the office by a computer.

First, we randomize clinics into pure control and the SMS messaging arm. Randomization is performed at the level of Peripheral Health Units (PHUs, a.k.a health clinics) and stratified by district, clinic type / size. Specifically, we stratify the randomization by whether a clinic is i) a Community Health Center or any other PHU type with a population larger than the 70th percentile of clinics, iii) any other PHU type with a population lower or equal to the 70th percentile of clinics.

Then, for all clinics in the SMS messaging arm, within each clinic we select groups of four contacts in the order that they are recorded in our data (i.e., the order in which they arrive at the clinic, to achieve balance on arrival time). We randomly assign two contacts to treatment and two to control within each group.

Individual and Primary health unit (PHU) -- health clinic

No

60 PHUs (clinics)

We will capture immunization data for 4 monthly cohorts of children in each of our study clinics. Expected sample for individual-level analysis - full immunization rate under one year and timeliness for vaccines: N = 30 (clinics) x 4 (monthly birth cohorts) x 35 (average birth cohort size, conservative estimate) = 4,200 contacts

Clinic-level analysis: 30 pure control clinics, 30 messaging arm clinics

Individual-level analysis: 2,100 control contacts, 2,100 treatment contacts (depending on birth cohort sizes in practice)

4.5pp