Healthcare Nudging in the Development Context: an RCT in the Republic of Armenia

We run a population-level randomized controlled trial (RCT) to understand the impact of targetted invitation letters and reminders on the take-up rate of a cervical cancer screening program implemented by the Government of the Republic of Armenia. The cervical cancer screening program is part of the "Disease Prevention and Control Project" financed by the World Bank (P128442).
Our experiment randomly assigned 20800 eligible females aged 30-60 in Shirak province to receive different invitation letters (neutral, negatively framed, concern for others) and reminders (no reminder, neutral, negatively framed, concern for others). Roughly 60% of the sample was randomly selected to receive a reminder.
The no-frame no-reminder letter constitutes the control group for our experiment.
The invitation letters are received by subjects 3 weeks before the scheduled screening week, whereas the reminders are received 1 week before. Both the letters and the reminders are sent to the postal address of each woman via registered mail through Haypost (the official national postal operator of the Republic of Armenia).

2019-05-06

2019-06-28

The key outcome variable we are interested in is the participation rate in the cervical cancer screening program.
The de-identified data will be accessed through the e-health platform that contains all the necessary information: the medical history of the patient, her address, the hospital she is registered at and the doctor she is registered with. When a patient visits a specific practitioner and undertakes tests, the information is stored in the platform. This will allow us to assess, whether a given patient visited her general practitioner and undertook the screening test or not.
During the experiment the Health Project Implementation Unit distributed participation lists to the practitioners and asked them to fill in the names of the patients who visited the medical institution with the primary aim to take part in the screening program. We will also be given access to these de-identified data.

The aim of our experiment is fewfold:

a) First, we want to assess the effect of specific manipulations of the invitation letter format on the take-up rate of the cervical cancer screening program.
b) Second, we strive to understand the impact of reminders on the take-up rate of the cervical cancer screening program.
c) Third, we want to uncover the most optimal letter-reminder combination.

a. Invitation letter

We tested three versions of the invitation letters:

-Neutral (slightly positive)
-Enhanced loss frame
-Concern for others

The "Neutral" letter informes the recipient about the screening program, the scheduled screening date and provides very brief information about the positive effects of participating in the screening.
Unlike the "Neutral" letter, the "Negatively Framed" letter provides the recipients with extended information about the negative consequences of not participating in the screening program. The rest of the information is exactly the same as that in the neutral letter.
Everything else the same, the "Concern for Others" letter replaces the negative information with a concern for others message, manipulating the social preferences of the recipients. More specifically, the letter asks the recipients to participate in the screening program for the sake of their loved ones, who expect them to live a happy and long life.

b. Reminders

The content of the reminders is exactly the same as that of the invitation letters. Roughly 60% of the sample is randomly selected to receive a reminder.

Overall, the RCT consists of 8 treatments.
1. Negatively framed letter- No reminder (N=2600)
The patients in this treatment received a negatively framed letter and no reminder.
2. Negatively framed letter- Negatively framed reminder (N=2600)
The patients in this treatment received a negatively framed letter and a negatively framed reminder.
3. Concern for Others Letter- No reminder (N=2600)
The patients in this treatment received an other-regarding letter and no reminder.
4. Concern for others Letter- Concern for Others reminder (N=2600)
The patients in this treatment received an other-regarding letter and an other-regarding reminder.
5. Neutral Letter- No reminder (N=2600)
The patients in this treatment received a neutral letter and no reminder.
6. Neutral Letter-Negatively framed reminder (N=2600)
The patients in this treatment received a neutral letter and a negatively framed reminder.
7. Neutral Letter- Neutral reminder (N=2600)
The patients in this treatment received a neutral letter and a neutral reminder.
8. Neutral Letter-Concern for Others Reinder (N=2600)
The patients in this treatment received a neutral letter and an other-regarding reminder.

We will also have access on subjects not involved in the RCT (i.e, not receiving invitation letters) from the population health registers.
Both the letters and the reminders were disbursed via registered mail. The letters were received 3 weeks prior to the screening date, whereas the reminders were received 1 week prior to the screening date. The RCT was implemented in Shirak province between May-July, 2019 in collaboration with the Ministry of Health of the Republic of Armenia, United Nations Development Program in Armenia and Armenia National SDG Innovation Lab (joint initative of the Government of the Republic of Armenia and the United Nations). Shirak province was chosen for fewfold reasons. First, Shirak is the most economically disadvantaged region of the Republic of Armenia. Thus, if cancer is detected at later stages, the majority of the population cannot afford the high treatment costs. Under these circumstances, early detection of potential cancer is of vital importance. Second, the participation rate in the cervical cancer screening program in this region was one of the lowest in the country before the experimental intervention .

Randomization was carried out by the reseachers in office using STATA 15. The seed was set to 123456789.

We carried out Individual level randomization for both the treatment group each subject should be exposed to and for the timing of letter delivery (i.e., week of invitation).
Allocation to treatment group was stratified by general practitioners to make sure that there was full support by the general practitioners. Hence, the analysis will be conditional on GP fixed effects, but there is no need to cluster by GP since the randomization is done individually.

No

The RCT was not clustered.

The representative sample contains 20800 females aged 30-60 randomly selected from the population of eligible females in Shirak province. The 20800 individuals were randomized into 8 treatments with 2600 subjects in each treatment.

Overall population of eligible females in Shirak province: roughly 42000 females aged 30-60.
Randomly selected for the study: 20800 females aged 30-60.
Number of treatment arms: 8

Assuming a standard deviation of the outcome of 0.259 (currently observed in the Armenian population), we estimate a minimum detectable effect with power 0.8 and significance 0.05 for pairwise comparison of the take-up rate across any two treatment groups of the size we have allocated (2600 subject per group) of 2.01 percentage points (we assume two-tailed tests are being carried out). A higher MDE is obtained if we consider that letters may fail to be delivered or the standard deviation of the outcome may be higher.