How to raise awareness during a pandemic? Evidence from Pakistan

Two types of awareness campaign strategies are implemented: 1) direct awareness campaigns for NGO beneficiaries (treatment arm 1) and 2) awareness campaign via the remote mobilization of local task forces on the ground (treatment arm 2). Both interventions are implemented via the phone. In sample 1, out of 1147 villages, 764 were randomized to receive the awareness campaign. The remaining 383 villages serve as control group. The final sample consists of 1526 (sampled in two waves: 1147+379) villages. Out of these, 1016 (764+252) villages were randomized to receive the awareness campaign. The remaining 510 (383+127) villages serve as control group (with no intervention). The awareness campaign is conducted via two modalities: via phone calls only 762 (573+189) villages or phone calls and remote mobilization of local task forces 254 (191+63) villages. In more detail:

Treatment arm 1: In 1016 villages NGO beneficiaries are contacted by NGO representatives over the phone and receive a 15-minute awareness raising session on general topics relating to COVID-19, specifically: i) an introduction to what COVID-19 virus is and how it is transmitted; ii) a description of the main risk groups and symptoms of the COVID-19 virus, iii) information on recommended preventive behavior, and iv) recommendations on how to react in case the respondent suspects that he/she or a household member is infected by the virus. In addition to delivering the above awareness message (referred to as “basic awareness message”), a subset of individuals was further randomized to receive one of four types of additional (5-min) messages. Each of these “additional messages” highlights a different aspect related to the COVID-19 virus: i) the (potential) severity of sickness, ii) the (potential) likelihood of contracting COVID-19, iii) the (potential) likelihood of infecting other family members, and iv) the (potential) cost of contracting the COVID-19 virus.

Treatment arm 2: In 254 villages, in addition to the above, 2) a local health task force (LHTF) composed of social workers or other NGO collaborators are remotely trained to implement simple awareness activities such as distributing posters, spreading awareness messages via loudspeakers (e.g. about hygiene rules and social distancing recommendations).

2020-09-23

2020-12-07

For each of our sub-hypotheses, we construct one index consisting of several related outcome variables. These are used as the main outcome variables to test the effects of the awareness campaign. Note that hypotheses 1 and 2 refer to intermediary outcomes, whereas hypotheses 3, 4 and 5 refer to final outcomes.
Awareness: We expect treated individuals to have higher levels of awareness of the ongoing COVID-19 pandemic compared to the control group. Improved awareness is manifested in increased knowledge about the corona virus, the symptoms related to COVID-19, and possible preventive measures. Additionally, we expect that treated individuals are able to better identify misconceptions or false information about the virus as compared to those assigned to the control group.
Hypothesis 1 The treatment increases respondents’ awareness of the COVID-19 virus. H 1.1 The treatment increases knowledge about the COVID-19 virus.
H 1.2 The treatment reduces misconceptions about the COVID-19 virus.
H 1.3 The treatment reduces stigma about the COVID-19 virus.
Perceptions: We expect the treatment to alter perceptions about: (1) the severity of the disease, (2) the likelihood of contracting COVID-19, (3) the likelihood of infecting other family members, (4) the expected cost of getting infected, and (5) the influence of individual behavior on the likelihood of contracting the virus.
Hypothesis 2 The treatment changes respondents’ perceptions about COVID-19.
H 2.1 The treatment changes perceptions about the severity of the COVID-19 virus [d].
H 2.2 The treatment changes perceptions about the likelihood of getting infected with the COVID-19 virus [p(l + s; h; V)].
H 2.3 The treatment changes perceptions about the likelihood of infecting others with the COVID-19 virus [pi(.)].
H 2.4 The treatment changes perceptions about the costs of getting infected with the COVID- 19 virus [C].
H 2.5 The treatment changes perceptions about own behavior and the probability of getting infected with the COVID-19 virus.
Here, we do not make claims about the direction of changes. For instance, a positive adjustment indicates that the individual will report at endline a higher likelihood of contracting the virus and/ or infecting others, relative to the baseline and to peers in the control group. Additionally, selected beneficiaries may report perceiving the virus to be more severe and/ or perceiving treatment costs to be higher. The direction of change is governed by a number of factors that will be examined further in Section 3.3 on the heterogeneity of effects.
Behavior The treatment is expected to improve adherence to preventive measures. The preventive measures under scrutiny include: Hand hygiene practices, wearing of a mask, reduced labor supply and reduced social interactions. Following the theoretical framework presented in Section 2.2, reduced labor supply and social interactions are more costly for individuals given their direct impact on the household’s utility.
Hypothesis 3 The treatment increases respondents’ adherence to preventive measures. H 3.1 The treatment increases the number of reported prevention measures.
H 3.2 The treatment reduces mobility.
H 3.3 The treatment reduces social interactions.
H 3.4 The treatment reduces labor supply.
Health: We further test the hypothesis that the treatment has an effect on the health status of recipients. We capture (only) self-reported health status.
Hypothesis 4 The treatment improves respondents’ general health status.
The awareness treatment is administered to only one of the household members. However, in addition to the anticipated direct impact of the awareness treatment on the respondent’s behavior and health status, we expect an indirect impact of the treatment on other household members through a spillover of information. We expect this indirect effect to be larger if the respondent is the household head and hence the main decision maker. In the case of the LHTF treatment, which is administered at the village level, all individuals in treatment villages can directly benefit from information provided by the program. Effects on household members other than the main respondent can thus be understood as indirect effects only, or a combination of indirect and direct effects, depending on whether a household member was reached by the LHTF.
Hypothesis 5 The treatment impacts the prevention behavior and health status of other house- hold members and village residents.
H 5.1 The treatment reduces mobility of other household members.
H 5.2 The treatment reduces social interaction of other household members.
H 5.3 The treatment reduces the labor supply of other household members.
H 5.4 The treatment improves the general health status of other household members and village residents.
Finally, we expect that the higher the intensity of the intervention, the higher the impact on the outcome variables. Intensity is defined in terms of exposure to more than one treatment at the same time, with a maximum of three possible treatments, Tba, Tad, and LHTF, at the same time.
Hypothesis 6 The higher the intensity of treatment, the higher the impact on key outcome variables.

see above

see above

see above

We conduct an RCT with ten treatment arms (excluding the control group). The RCT is designed to evaluate two different strategies of conducting remote awareness campaigns for NGO beneficiaries. The first one consists in an awareness campaign over the phone and the second one is awareness campaign over the phone accompanied by awareness via a local health task force (LHTF) team mobilized over the phone but conducting awareness activities on the ground(e.g. awareness via posters, loudspeakers). We followed a two-stage randomization process.
First, we stratify the prioritized universe of revenue villages by the implementing partner. In each stratum, 33% of revenue villages were assigned to control, and 67% to treatment. Among the treated villages, 75% were assigned to receive awareness campaign over the phone, while 25% were assigned to receive the awareness campaign over the phone and – in addition – via the LHTF.
Secondly, treated individuals were randomized to receive five different awareness treatments about COVID-19 over the phone: basic awareness(t0); basic awareness + additional message on the potential severity of sickness (t1); basic awareness + additional message on the potential likelihood of contracting COVID-19 (t2); basic awareness + additional message on the potential likelihood of infecting others (t3); basic awareness + additional message on the potential cost of contracting COVID-19 (t4).
While the baseline data for randomization is fixed, we do collect additional refresher contacts in the same villages and also plan to collect a new sample of additional villages to which the study will be extended.
The same experimental design will be applied.

We conduct an RCT with ten treatment arms (excluding the control group). The RCT is designed to evaluate two different strategies of conducting remote awareness campaigns for NGO beneficiaries. The first one consists in an awareness campaign over the phone and the second one is awareness campaign over the phone accompanied by awareness via a local health task force (LHTF) team mobilized over the phone but conducting awareness activities on the ground(e.g. awareness via posters, loudspeakers). We followed a two-stage randomization process.
First, we stratify the prioritized universe of revenue villages by the implementing partner. In each stratum, 33% of revenue villages were assigned to control, and 67% to treatment. Among the treated villages, 75% were assigned to receive awareness campaign over the phone, while 25% were assigned to receive the awareness campaign over the phone and – in addition – via the LHTF.
Secondly, treated individuals were randomized to receive five different awareness treatments about COVID-19 over the phone: basic awareness(t0); basic awareness + additional message on the potential severity of sickness (t1); basic awareness + additional message on the potential likelihood of contracting COVID-19 (t2); basic awareness + additional message on the potential likelihood of infecting others (t3); basic awareness + additional message on the potential cost of contracting COVID-19 (t4).
While the baseline data for randomization is fixed, we do collect additional refresher contacts in the same villages and also plan to collect a new sample of additional villages to which the study will be extended.
The same experimental design will be applied.

We apply the procedure outlined above for randomization. In each stage, we apply a re-randomization procedure incorporating multivariate balance checks.


In order to allow us to test the null hypothesis that the treatment effect is zero (or any other value) and to calculate confidence intervals we will apply randomization inference.
To reduce the number of tests in our analysis, we pool primary outcomes that are similar in the sense that they are expected to be proxies of the same underlying behavior and test the same hypotheses or sub-hypotheses. Details on this and other strategies to adjust for multiple hypothesis testing are presented in the PAP.

Randomization at revenue village (RVs) and individual level.

Yes

1526 (sampled in two waves: 1147+379) villages

18,789 individuals (12,883 in sample 1 and 5,906 in sample 2). More individuals will be added and randomized into the study following two upcoming data collection waves.

Number of clusters (villages) by treatment arms

No awareness: 510 villages
Awareness via phone only: 762 villages
Awareness via phone + loudspeakers: 254 villages

Cross-randomized messages via phone:
Basic awareness phone calls in 204 villages
Basic awareness phone calls + info on severity of coronavirus 193 villages
Basic awareness phone calls + info on individual's prob of getting infected 203 villages
Basic awareness phone calls + info on HHm's prob of getting infected 222 villages
Basic awareness phone calls + info on cost of health treatment 194 villages

To calculate the base (or pre-treatment) level of all main outcome variables, we used baseline data from the subsample (sample 1) of observations already available at the time of registration. This data was collected from beginning of August until the 3rd week of September and includes 12,883 individual observations. We look at MDEs for three different evaluations: i) any awareness treatment vs. the control group; ii) any awareness treatment + local health task force (LHTF) vs. awareness treatment only; and iii) any awareness treatment + an additional awareness message vs. awareness treatment only. Whereas i) and ii) are implemented at village level, iii) is randomized at individual level. We estimate minimum detectable differences (MDEs) given alpha=0.5 and power=0.8 and consider a conservative scenario in terms of attrition. For treatments which are implemented on village level, we estimate to be left with 6 observations per village at the end of the trial. This equals 37.5% of the observations we are aiming for and thus an attrition rate of 62.5%. For treatments implemented at individual level, we use an attrition rate of 60%. The following MDEs account for the inclusion of control variables. Given the above assumptions, we can detect a relative change from a baseline level of 13.3 percent to an expected level of 15.9-16.6 percent (impact of 19.5-25.8%) for knowledge of the main COVID-19 symptoms, depending on the level of comparison; a relative change from a baseline level of 29.0 percent to an expected level of 32.4-33.5 percent (impact of 11.7-15.4%) for knowledge of the main preventive measures; a relative change from a baseline level of 37.3 percent to an expected level of 41.4-42.7 percent (impact of 10.9-14.5%) for being highly or moderately concerned about getting infected; a relative change from a baseline level of 47.8 percent to an expected level of 52.1-53.5 percent (impact of 9.0-11.8%) for being highly or moderately concerned about infecting others; a change from a baseline level of 12,636.03 Pakistani Rupees (PKR) to an expected level of 15,349,07-16,224.40 PKR (impact of 21.5-28.4%) for the perceived average cost of getting infected; a relative change from a baseline level of 32.4 percent to an expected level of 27.9-29.1 percent (impact of -10.2-14.0%) for having had visits in the previous 7 days; a relative change from a baseline level of 39.4 percent to an expected level of 34.7-35.9 percent (impact of -9.0-11.9%) for having worked outside home in the previous 7 days; a change from a baseline level of 2.62 hours to an expected level of 2.23-2.33 (impact of -11.0-14.9%) for hours worked per day in the previous 7 days; a relative change from a baseline level of 23.3 percent to an expected level of 26.4-27.4 percent (impact of 13.1-17.6%) for practicing all 3 main preventive measures; and a change from a baseline level of 0.1 to an expected level of 0.061-0.078 (impact of -23.2-40.7%) for the amount of people within a household with common corona virus symptoms.