Adaptive Trial for COVID-19 Messaging Campaigns

Participants receive SMS messages to encourage them to adopt key health behaviors: hand washing, social distancing, respiratory hygiene and wearing a mask when outside. The messages vary in the behavior they target, the framing, and the timing when they are sent.

2020-08-17

2020-10-20

Prevalence rate for washing hands regularly with soap and water; social distancing

Using the "exploration sampling" approach, we will identify the most effective combinations of framing and, timing/frequency in increasing the adoption of the various preventative behaviors. The initial allocation of the treatment is equal for all arms. We adjust the shares as we obtain information from continuous phone surveys.

For each targeted behavior we have 5 frames (public vs private X gain vs loss, and a neutral frame). Each participant receives 4 messages, either twice in the morning for two days, or once in the morning / once in the evening for two days. Together we have 10 trial arms for each behavior.

Randomization done in office on computer using Stata

Individual

No

NA

5600 completed phone interviews of households

The share of observations assigned to each treatment will vary depending on the effectiveness of different messaging interventions.

We conduct an adaptive randomized controlled trial to evaluate the impact of a SMS-based information campaign on the adoption of social distancing and handwashing in rural Bihar, India, six months into the COVID-19 pandemic. We test 10 arms that vary in delivery timing and message framing, changing content to highlight gains or losses for either one’s own family or community. We identify the optimal treatment separately for each targeted behavior by adaptively allocating shares across arms over 10 experimental rounds using exploration sampling (Kasy and Sautmann, 2021). Based on phone surveys with nearly 4,000 households and using several elicitation methods, we do not find evidence of impact on knowledge or adoption of preventive health behavior, and our confidence intervals cannot rule out positive effects as large as 5.5 percentage points, or 16 percent. Our results suggest that SMS-based information campaigns may have limited efficacy after the initial phase of a pandemic.

Bahety, Girija; Bauhoff, Sebastian; Patel, Dev; Potter, James (2021) "Texts Don’t Nudge: An Adaptive Trial to Prevent the Spread of COVID-19 in India" CGD Working Paper 585

https://www.cgdev.org/publication/texts-dont-nudge-adaptive-trial-prevent-spread-covid-19-india

We conduct an adaptive randomized controlled trial to evaluate the impact of a SMS-based information campaign on the adoption of social distancing and handwashing in rural Bihar, India, six months into the COVID-19 pandemic. We test 10 arms that vary in delivery timing and message framing, changing content to highlight gains or losses for either one’s own family or community. We identify the optimal treatment separately for each targeted behavior by adaptively allocating shares across arms over 10 experimental rounds using exploration sampling. Based on phone surveys with nearly 4,000 households and using several elicitation methods, we do not find evidence of impact on knowledge or adoption of preventive health behavior, and our confidence intervals cannot rule out positive effects as large as 5.5 percentage points, or 16%. Our results suggest that SMS-based information campaigns may have limited efficacy after the initial phase of a pandemic.

Bahety, Girija; Bauhoff, Sebastian; Patel, Dev; Potter, James (2021) "Texts Don’t Nudge: An Adaptive Trial to Prevent the Spread of COVID-19 in India" Journal of Development Economics (153): 102747

https://doi.org/10.1016/j.jdeveco.2021.102747