Are text message reminders effective in promoting compliance with COVID-19 behavioral change requirements? Evidence from the city of São Paulo

The stimuli aimed to incentivize self-distancing and mask wearing behaviors. The intervention consisted of four text messages, to be delivered every two days. Messages were designed to inform why these behaviors were important, instruct on how to properly do so, and motivate compliance. The first message was informative, the second was a call to action to stay at home, the third was instructional, and the fourth was a call to action to wear a mask in public. Messages 1 (information) and 3 (instruction) were the same across all treatment arms. Messages 2 (motivation to self-isolate) and 4 (motivation to wear masks) varied per treatment arm, with each one exploring a behavioral principle: social norms, reciprocity, civic duty, self-efficacy, and risk perception. The principles were selected after an extensive review of the behavioral science literature and the rationale behind them is explained in detail in the following subsections. By randomizing the behavioral principle across arms, the study seeks to investigate what kind of mechanism is better suited to promote behavior compliance in the COVID19 context.

2020-06-02

2020-06-08

We have five primary outcomes:
* Beliefs index
* Behavior - leave home index
* Behavior - wear a mask index
* Behaviour - keep distance index
* Awareness index

Indexes will be constructed based on our survey questionaire.
* Beliefs index
Additive index ranging from 3 to 10 of:
Would you say that you are ____ of being infected by the coronavirus?
4 points: very scary
3 points: scary
2 points: a little scary
1 point: not scary at all
In your opinion, what is the best way to deal with coronavirus:
3 points: isolate everyone, but those who work in essencia services?
2 points: isolate only eldery and people in the risk group?
1 point: let everyone behave as they wish?
Do you believe people should wear a face mask in public?
3 points: yes, all the time;
2 points: yes, sometimes,
1 point: no

* Behavior - leave home index
Additive index ranging 0 to 8 of (yes = 1, no = 0):
In the last seven days, have you left your house to:
go to work?
go to the grocery store, to the farmacy or to the bank?
go to the hospital or to seek medical assistance?
walk a pet or a child?
walk or to do sports?
take care of someone who needs assistance?
meet someone who doesn't live with you?
any other reason?

* Behavior - wear a mask index
Additive index ranging from 2 to 8 of:
In the last seven days,
when you left home, have you worn a mask covering mouth and nose:
4 points: all the time
3 points: sometimes
2 points: a few times
1 point: never
when you received a package or open your house’s door, have you worn a mask covering mouth and nose:
4 points: all the time
3 points: sometimes
2 points: a few times
1 point: never

* Behaviour - keep distance index
Index ranging from 1 to 4 of:
In the last seven days, when in public, have you kept distance from other people?
4 points: all the time
3 points: sometimes
2 points: a few times
1 point: never

* Awareness index
Additive index ranging from 0 to 3 of:
What distance must you keep from others in public?
1 point - right answer: 2 meters
0 points - wrong answer: any other answer
If I am wearing a mask and the other person too, do we need to keep distance?
1 point - right answer: yes
0 points - wrong answer: no
If I am wearing a mask for 1 hour and it gets umid, do I need to change it?
1 point - right answer: yes
0 points - wrong answer: no

Control variables:
1) Do you know someone who was diagnosticated with coronavirus?
If yes, how close were you to that person?;
2) Do you know someone who died because of coronavirus?
If yes, how close were you to that person?;
3) Gender;
4) Age;
5) Schooling;
6 )Has health insurance;
7) Is São Paulo City resident;
8) Neighborhood of residence;
9) Remember having received an SMS about coronavirus from the City of São Paulo;

The experiment design does not have a confirmatory hypothesis on what family of SMS performs better. Nevertheless, we will perform exploratory analysis to compare all treatment arms with one another. Socioeconomic information will be used as a control in the confirmatory and exploratory analysis.
We also plan to explore heterogeneous effects using our control variables and time (number of days) between survey response and the end of the intervention as an exploratory exercise.

The intervention will be evaluated through a randomized control trial with 76,324 participants, stratified by gender and neighborhood. Participants were randomly selected from the e-government platform (SP156) database that initially contained 755,585 records, and assigned to one of six arms: five treatment arms and one control arm in which participants do not receive any messages. After the text messages were sent, citizens will be contacted on their cellphones to answer a survey designed to measure beliefs, recall, awareness, and self-reported behavior related to the epidemic.
Given the experiment includes five treatment arms that will be compared with the control group, to maximize statistical power, the number of participants assigned to the control was 2.24 (square root of 5) times larger than each treatment group. As a result, the division of the total number of phone number records in our database was the following: (i) control: 233,488, (ii) treatment arm 1: 104,419, (iii) treatment arm 2: 104,420, (iv) treatment arm 3: 104,419, (v) treatment arm 4: 104,420, (iv) treatment arm 5: 104,419.
Because of limited resources, we only sent text messages to a representative sample of each arm. Considering our power calculations, we aimed to have 2,572 post-treatment completed interviews from the control group and 2,538 from each treatment arm. And given the telephone center provided us with an estimate of 20% response rate for the phone calls, we sent text messages to a sample five times bigger than the aimed number of interviews: around 12,860 in the control group, and around 12,690 in each treatment arm. The intervention was delivered to 63,465 records between June 2 and June 8, and 12,859 records were kept in the control group. Due to budget constraints, we were unable to send a placebo SMS to a group of people or to do a pre-treatment survey.
Our strategy to assess the impact of the messages is based on a telephone survey that will be conducted by the SP156 telephone center. The survey instrument (questionnaire) included a small number of items and was designed to be completed in only 6 minutes. The questionnaire included four batteries of questions that measure: (i) beliefs, (ii) self-reported behavior, (iii) comprehension/ recall, and (iv) sociodemographic variables. The first three sections comprise the study outcomes. As such, we will combine items per section to create additive indices, and then evaluate the impact of the messages on each of the first three dimensions (outcomes). The sociodemographic information collected in the last part of the instrument will be used as controls.
We will estimate an intention-to-treat (ITT) effect. We are solely estimating the ITT, and not the Average Treatment Effect (ATE) because we cannot know the compliance rates, that is, whether the person opened the text messages, only if one has received it. We will conduct a one-tailed test, with α = 5%, β = 80%, and Dunnet’s correction for multiple comparisons (family-wise error). With this set, we have two percentage points of minimum detectable effect.

Randomization done in office by a computer.

Individual level randomization

No

NA

15,262 - survey responses 76,324 - inicial sample

----- Control -----
2,572 - survey responses
12,859 - inicial group size
----- Treatment arm 1-----
2,538 - survey responses
12,693 - inicial group size (SMS)
----- Treatment arm 2-----
2,538 - survey responses
12,693 - inicial group size (SMS)
----- Treatment arm 3-----
2,538 - survey responses
12,693 - inicial group size (SMS)
----- Treatment arm 4-----
2,538 - survey responses
12,693 - inicial group size (SMS)
----- Treatment arm 5-----
2,538 - survey responses
12,693 - inicial group size (SMS)

We will conduct a one-tailed test, with α = 5%, β = 80%, and Dunnet’s correction for multiple comparisons (family-wise error). With this set, we have two percentage points of minimum detectable effect.

PAP_covid_sms_sao_paulo