Evaluation of a Mass Media Family Planning Campaign on the Uptake of Modern Contraceptive Methods in Burkina Faso

The intervention we evaluate is a mass media campaign through local radio stations in Burkina Faso.

The media campaign design and production will be implemented by Development Media International (DMI) in an attempt to change contraception-related behavior and improve health outcomes in Burkina Faso. The campaign will be broadcasted in different languages corresponding to the local languages of the targeted clusters.

Based on formative qualitative research in the field, messages are formulated to overcome the cognitive barriers and social norms that prevent the uptake of modern contraceptives. The media campaign, which will last 3 years, will include short (60-seconds) radio spots, phone-in programs, and interviews with stakeholders and key figures.

The main themes include:
-Delaying the age at first pregnancy
-Increasing the time interval between pregnancies
-Reducing the number of total children per woman
-Benefits of family planning in enhancing financial conditions, health, and opportunities for the family
-Information on safe modern contraceptive methods including types, sources, advantages, and disadvantages of different contraceptive methods

The media campaign is coupled with radio distribution to increase listenership among women without radio in targeted and control clusters. Recipients that maintain functioning radios until the endline will be eligible to enter a lottery for prizes.

2016-06-01

2018-12-31

Primary outcomes:
1. Contraceptive prevalence rate (including modern methods and effective traditional methods such as withdrawal. To understand mechanisms we will also analyze impact separately by modern and effective traditional methods)

A. Usage of contraception and family planning
2. Percentage of women intending to use contraception in the future
3. Percentage of women seeking family planning advice
4. Percentage of women discussing family planning with their partners and others
5. Percentage of women who are fecund and sexually active who do not want to become pregnant but are not currently using contraception

B. Impact of use of contraception (these are important impact questions but we will have limited power to test for them so do not expect to find significant impacts).
6. Average time lapse between pregnancies
7. Number of unwanted pregnancies
8. Number of births per 1,000 women of reproductive age (if accurate population level data on total births in communities become available)
9. Number of family planning products delivered throughout the study period (from administrative data, if available)

C. Knowledge, attitudes, and perception of about contraception and family planning
10. Mean effects on survey questions relating to knowledge of contraceptive methods
11. Mean effects on survey questions relating to attitudes towards contraception
12. Mean effects on survey questions relating to knowledge of family planning
13. Mean effects on survey questions relating to attitudes towards family planning
14. Mean effects on survey questions relating to women’s perceptions of fertility and birth spacing
15. Mean effects on survey questions relating to partners’ perceptions of fertility and birth spacing (as reported by women)

D. Potential knock on effects (third level of priority)
16. Mean effects on survey questions relating to perceptions on gender norms
17. Mean effects on survey questions relating to behavior reflecting women empowerment
18. Mean effects on survey questions relating to women's subjective health and well-being
19. Mean effects on survey questions relating to domestic violence and sexual harassment

E. Distribution of contraception at ministry-operated clinics (administrative data)
20. Number of injectables distributed each month by each clinic (top coded at the 99th percentile)
21. Number of pills distributed each month by each clinic (top coded at the 99th percentile)
22. Number of implants distributed each month by each clinic (top coded at the 99th percentile)
23. Number of male condoms distributed each month by each clinic (top coded at the 99th percentile)
24. Number of contraceptive days distributed each month by each clinic (top coded at the 99th percentile)
25. Number of leaving births (top coded at the 99th percentile)

10. Mean effects on survey questions relating to knowledge of contraceptive methods
• Variables include: knowledge of the existence, price, source, advantages, and disadvantages of different methods, including rejection of misconceptions such as contraception causing sterility or sickness

11. Mean effects on survey questions relating to attitudes towards contraception
• Percentage of women who think that it is embarrassing to buy a contraceptive method
• Percentage of women who think that using contraceptive methods is a sign of not trusting their partner

12. Mean effects on survey questions relating to knowledge of family planning
• Percentage of women who know benefits of spacing births
• Percentage of women who know benefits of delaying the age of marriage for young girls

13. Mean effects on survey questions relating to attitudes towards family planning
• Percentage of women who think it is acceptable to talk about family planning in public (radio, schools, posters, etc.)
• Percentage of women who think that a woman should be able to control the number of children she has during her lifetime

14. Mean effects on survey questions relating to women’s perceptions of fertility and birth spacing
• Women’s perception on the ideal age at first birth (in standard deviation units from the control group)
• Women’s perception on the ideal time lapse between first and second birth (in standard deviation units from the control group)
• Women’s perception on the ideal number of children in total (in standard deviation units from the control group)

15. Mean effects on survey questions relating to partners’ perceptions of fertility and birth spacing (as reported by women)
• Partners’ perception on the ideal time lapse between first and second birth (in standard deviation units from the control group)
• Partners’ perception on the ideal number of children in total (in standard deviation units from the control group)

16. Mean effects on survey questions relating to perceptions on gender norms
• Percentage of women who think that it is better to be a man than a woman
• Percentage of women who think that boys should have better access to resources in education
• Percentage of women who think that men must be more educated than their wives
• Percentage of women who think that men should have better access to consumption of meat and imported products

17. Mean effects on survey questions relating to behavior reflecting women empowerment
• Percentage of women working or participating in a productive activity
• Percentage of women participating in decision-making when it comes to different household expenditures

18. Mean effects on survey questions relating to women's subjective health and well-being
• Percentage of women satisfied with their lives
• Percentage of women considering themselves healthy compared to other women in the village
• Percentage of women considering themselves happy compared to other women in the village

19. Mean effects on survey questions relating to domestic violence and sexual harassment
• Percentage of women whose husbands / husbands’ families get jealous when they walk to other men
• Percentage of women whose husbands / husbands’ families don’t allow them to see their female friends
• Percentage of women whose husbands / husbands’ families insist on knowing where they are in the village at any time of the day
• Percentage of women whose husbands / husbands’ families ever threatened to harm them or their families
• Percentage of women whose husbands / husbands’ families ever destroyed their personal objects
• Percentage of women whose husbands / husbands’ families ever physically hurt them

20 - 23. Using clinic administrative data
If the administrative data contains a large number of zeros or outliers, we will use three strategies to account for this distribution. At this point, we do not have full clarity on what the zeros indicate and we consider three possible explanations. For some months, the zeros could represent no activity or contraceptive distribution. Alternatively, the clinic may have not have updated their records, so the zero actually represents missing data. Additionally, in recording distribution, clinics may allocate all the contraceptives distributed in one quarter to one month in particular., which generates zeros and large outliers. Our three strategies are:

1) Top-code outcomes at the 99th percentile: For each contraceptive method, we replace any clinic-month observation that exceeds the 99th percentile of the number of contraceptives distributed, with the exact value of the 99th percentile of all clinic-month observations for that method.
2) Inverse hyperbolic sine transformation: This transformation is defined by log(yi + (yi^2 + 1)^1/2). This transformation is approximately equal to log(2yi) or log(2)+log(yi) and can be interpreted as a logarithmic dependent variable.
3) Average monthly distribution data over three- and six-month periods.

24. We estimate total contraceptive days provided through clinic distribution of contraceptives using the four primary methods of contraceptives in our administrative data. This outcome represents the number of days a single women would have contraceptive coverage based on clinic distribution of all contraceptives. We multiply the number of days each method prevents pregnancy if used effectively by the number of each method distributed in each month, and we aggregate this across four methods. We assume that a condom provides one day, pack of pills provides 30 days, an injectable provides 91 days, and an implant provides either 1095 or 826 days (we have two brands in our administrative data).

The intervention will take place in 16 local radio station coverage areas available for the study, 8 of which will be randomly assigned to treatment. This number is limited by the number of radio stations that can be sampled without overlap in the broadcasting reach to allow for separation of treatment and control groups. These radio stations will also broadcast messages in local languages corresponding to the rural zones targeted. The possibility of leakages between treatment and control groups is therefore unlikely, as people living in rural areas speak different languages.

Although the messages diffused will address both men and women to influence cognitive barriers to usage of contraception among couples, our study will only survey women at the age of reproduction (15-49 years old).

Our total sample size includes 7515 women across more than 260 villages across in 16 radio station clusters. Women selection will be stratified on education and access to radio, while village selection will be stratified by average distance to a clinic. Our sample also includes survey data 461 clinics and administrative data from 839 clinics within 50 km of a study radio station.

In summer 2017, DMI distributed radios to half of all women (in both treatment and control areas) who reported not owning a radio during our survey as selected through random assignment. This treatment was layered on in response to the 2016 baseline survey data that revealed that listenership in study communities was lower than expected based on previous media surveys in rural Burkina Faso conducted by Development Media International in 2011 and 2012.

-We conducted a baseline survey to capture baseline levels of mCPR and other population characteristics. We used these to stratify our sample prior to randomization.
-We monitor supply of contraceptive methods throughout the intervention in an attempt to disentangle demand and supply factors in changing the level of contraceptive uptake. We draw on administrative data from the Ministry of Health on distribution of contraceptives at clinics.

Stratified randomization of villages and women selection is done in office by a computer.

The mass media treatment is administered through radio stations, and therefore on clusters formed by the regions of coverage of these radio stations.
The layered radio distribution treatment is randomized at the household level in both treatment and control villages.

Yes

The number of radio station clusters is limited to 16 to make sure that the broadcast regions do not overlap.

For the radio distribution treatment, the sample includes 2,787 households with a women receiving a radio.

We will survey 252 villages and 461 clinics across the 16 clusters. Out total sample size is 7515 women. Our administrative sample covers 839 clinics and data is shared monthly. For the radio distribution treatment the number of observations is 3,138 women (in 2,787 households).

Among the 16 regions, 8 will be assigned to treatment and the remaining 8 to control.

For the radio distribution treatment, 1397 households will be assiged to the treatment group and 1390 to the control group.

Based on DHS data in the rural areas where our study takes place: (1) Baseline level of mCPR: 12%. (2) ICC at the radio station level clusters: 0.013/ (3) ICC at the villages level: 0.06. (4) Detectable treatment effect corresponding to 80% power: 6 percentage points increase in mCPR. Using simulations on Stata, we test the robustness of power to different levels of baseline mCPR, since this level is reported differently by different sources. We also test the robustness of power to different levels of ICC. As expected, power is most sensitive to the radio station level ICC, since it's the highest level of randomization, and the level on which treatment is assigned and administered. Empirical data suggests however that this ICC level is very low in our sample. Actual power will be higher than in this exercise because (1) we will stratify, (2) we will have a panel structure and (3) will control for the baseline level of mCPR and other explanatory variables. The ICC level should be lower after controlling for explanatory variables as some of the differences between regions is due to differences in observables such as education). There is uncertainty over ICC and the baseline level of mCPR and there is a risk that power is lower than we think it is based on these data and simulations. If that is the case we will only be able to pick up a larger MDE. For the radio distribution treatment, the randomization is done at the household level and the sample only include women without access to a radio at baseline. We calculated that our sample size will provide sufficient statistical power to detect an impact of 5 percentage points (from 22% to 27%) among women without access to a radio at baseline.