A randomized control trial study of the Mali branch of the multi-country programme: “Responding to COVID-19 through Social Protection Systems in the Sahel”

In response to the COVID-19 pandemic and subsequent economic downturn, the Government of Mali, with support from BMZ/KfW, is implementing a cash transfer programme targeted to members of WSG in the southern regions of Koulikoro, Segou and Sikasso.

Treatment 1: An unconditional cash transfer plus complementary services
This intervention combines unconditional cash transfers with complementary services. WSG members receive five cash transfers over the course of six months (between March 2023 and August 2023), each with a value of 45,000 XOF (~68 USD). In this intervention arm, cash transfers are accompanied by complementary activities including:
• Child-sensitive training on key family practices, which cover areas of breastfeeding, nutrition, health (including on COVID-19), girl schooling.
• Gender-sensitive economic empowerment activities, covering (1) sensitization to invest part of the transfer in productive activities, (2) platform for selling and marketing group products, (3) training on basic business accounting, (4) set up women’s savings & credit groups, where appropriate (to ease access to credit), (5) group guarantees when applying for microfinance credit.
• Gender-transformative leadership and decision-making training on women’s rights, responsibilities and advocacy, covering (1) women’s rights and leadership, (2) women’s standing within household and community via financial contributions from transfers, (3) environmentally friendly practices for waste reduction / disposal, (4) sustainable agriculture & livestock practices.
Complementary services will be implemented partly by (i) Direction Nationale du Développement Social through the “Mama Yeleen” strategy which is based on interpersonal communication and women leadership; (ii) implementing NGO partners and (iii) Jigisemeyiri, the national social safety net programme.

Treatment 2: Services only
This treatment arm will only receive the complementary services but not the unconditional cash transfer. They will start receiving the cash transfers after the endline survey (November 2023).

2023-03-05

2023-08-10

• Household consumption, poverty and food security
• Child-level outcomes (birth registration, infant and young child feeding practices, nutritional status, health, early childhood development, child protection);
• Gender-related outcomes (women’s empowerment, women’s participation in agriculture and business activities, intra-household time use, gender norms and intimate partner violence (IPV));
• WSG-level indicators (membership, participation rates, amounts received, savings and credit amounts).

Household consumption, poverty and food security
• Construction of household consumption will follow the methods described in the Mali Poverty Profile 2018/2019 and aggregates will be calculated in adult equivalents (AE) per month.
• Monetary poverty will be calculated using the consumption aggregates and compared to the official poverty lines in Mali. We will use the poverty lines for the Rural Sudan zone since most households in the sample live in that zone. Poverty lines will be inflated to the price level at the time of the surveys using the official CPI figures. The CPI figures are only available for selected regions, one of them being Sikasso, and thus the CPI numbers from this region will be used to index the poverty line to account for inflation. The food poverty line and non-food poverty line will be inflated separately with their corresponding CPI.
• Food security will be measured using the Food Consumption Score (FCS), developed by the World Food Programme (WFP). It combines the diversity and frequency of consumption of different food groups during the week preceding the interview. The FCS has a possible range of 0-112 where higher values indicate better food consumption. The WFP has established several cut-offs to determine an acceptable FCS (FCS higher than 41), borderline FCS (between 28 and 41) and poor FCS (less than 28).
• Another measure of food insecurity is the coping strategy index (CSI), also developed by WFP, which measures the number of coping strategies households applied in the seven days prior to the survey. The index is measured on a scale of 0-56, with higher values indicating higher levels of food insecurity. A household with a CSI value of 10 or higher is considered to adopt intense coping strategies, indicating poor food security.
• Household-level indicators will be disaggregated by the sex of the household head (female versus male heads).

Child-level outcomes
• Birth registration: an indicator measuring whether the birth of the child was registered with the appropriate authorities.
• Health: prevalence of diarrhea in the last 2 weeks (self-reported); prevalence of fever in the last 2 weeks (self-reported).
• Infant and young child feeding practices: exclusive breastfeeding for children under 6 months; minimum meal frequency for children 6-23 months; minimum diet diversity for children 6-23 months; minimum acceptable diet for children 6-23 months. All following the 2021 edition of the ‘Indicators for assessing infant and young child feeding practices’ by WHO & UNICEF.
• Nutritional status: prevalence of stunting, wasting and underweight, as well as the average levels of height-for-age z-score, weight-for-height z-score and weight-for-age z-score, following WHO 2006 growth standards methodology.
• Early childhood development: following MICS methodology, indicators will be constructed to measure the level of interaction with children 0-59 months (at baseline) old during the previous three days based on six activities: reading books, telling stories, singing songs, taking out of the house, playing and naming, counting or drawing. Indicators will measure whether a child had 4+ activities with an adult member of the household, 1+ activities with the mother, 1+ activities with the father and the number of activities with the mother, father and any adult member of the household.
• Child discipline: using the child discipline module introduced in the MICS, an indicator will be constructed to measure the percentage of children aged 1 to 4 years who experienced any form of violent discipline (physical punishment and/or psychological aggression) within the past month. Also an indicator will be constructed that measures the percentage of caregivers that belief that a child needs to be physically punished in order to bring up, raise, or educate a child properly. This module is only introduced in the endline survey.
• All child-level indicators will be disaggregated by sex and age groups as appropriate.

Gender-related outcomes:
• Women’s empowerment: women’s empowerment will be measured using 13 survey items covering four key dimensions: inputs in productive decisions, control over the use of income, leadership in the membership group, and gendered attitudes and beliefs, following studies by Malapit et al (2017) and Miedema et al. (2018). Principal component analysis will be conducted on items characterizing each dimension and the first component will be considered. The index associated with each of the key dimensions will be defined as a binary variable equalling 1 if an individual loading factor is greater than the average and 0 otherwise. We will also estimate impacts on each of the components separately.
• Women’s participation in agriculture and business activities: the survey elicits the main sources of income for women and an indicator will be constructed for each of the following eight sources: agriculture; vegetable gardening; food/animal commerce; non-food commerce; breeding/fishing; remittances/pensions/family support; crafts, contractors/small trades; other income sources. The survey will also collect information on the access to agricultural land (measured by a binary indicator, plus total area of land) and the operation of a non-agricultural income generating activity (IGA), with indicators measuring the engagement in an IGA (yes/no), type of IGA (based on 24 business codes), the level of decision-making power for the IGA (woman herself, woman together with others, or only others), and the average profit or loss during the previous month.
• Knowledge of essential family practices: Using a survey module with 14 knowledge questions, an indicator will be constructed measuring the share of correct answers.
• Intra-household time use: using a time use module in the survey, indicators will be constructed to measure the participation (yes/no) and number of hours in the following household tasks: collecting water; collecting firewood; caring for children; cooking or cleaning; caring for sick members; caring for elderly members. And in terms of economic activities: fishing; operating/assisting a non-farm family business; operating their own non-farm business; raising livestock; farm work; collecting nuts or other tree fruits; casual or parttime work outside the household; formal work outside the household.
• Gender norms: Using a survey module with 11 gender norms statements, an indicator will be constructed measuring the share of answers corresponding to progressive gender norms. We will also estimate impacts on two indicators based on specific categories of gender norms statements (time use, harmful practices), as well as impacts on each of the statements separately.
• Intimate partner violence (IPV): following methodology developed by WHO, a survey module measuring exposure to IPV is included and the following indicators will be constructed: Experiencing controlling behaviors (ever & last 12 months); experiencing emotional violence (ever & last 12 months); experiencing physical violence (ever & last 12 months); experiencing sexual violence (ever & last 12 months); experiencing emotional/physical/sexual IPV (ever & last 12 months); experiencing physical/sexual IPV (ever & last 12 months); sought help/told someone about IPV (among those that experienced IPV).

WSG level outcomes:
• Membership: the number of members in the group
• Participation rates: Whether all members participate regularly in the WSG meetings (yes/no)
• Savings: Savings amount per WSG's member per month (FCFA), and total amount of savings since creation of the group (FCFA)
• Access to formal credit: whether WSG has access to formal credit (yes/no), and the amount of credit taken by the WSG in the past 6 months (FCFA)
• Credit granted: whether the group has started to grant credit to members (yes/no), and the amount of credit granted in total since creation of the group (FCFA)

• Mental health: Using a survey module eliciting the level of worry during the previous seven days about 12 topics, the following indicators will be constructed: Worried about violence; Worried about social relationships (family, relatives and love); Worried about physical health; Worried about financial situation (bankruptcy, finances, employment); Worried about alcohol, drugs and substances use; Worried about food and/or drinking water.
• Girl’s schooling: literacy, enrolment, current attendance and current grade for girls 3-17 years old. Also disaggregated by school-level (primary vs secondary).
• Child labour: Following ILO international conventions and recommendations, child labour will be defined as work that is harmful to children’s health or interferes with their schooling. The household survey includes data on time spent in economic activities and household chores for household members aged 5 years or older. Data for children aged 5 to 17 years will be used to construct measures of long hours of work (economic activities or household chores).
Following UNICEF Multiple Indicator Cluster Survey (MICS) (UNICEF, 2015, Mali Final MICS Report), the following age-specific thresholds for weekly hours will be used to identify long hours in economic activities:
o Age 5-11: 1 hour or more
o Age 12-14: 14 hours or more
o Age 15-17: 43 hours of more
The following thresholds for weekly hours will be used to identify long hours in household chores:
o Age 5-14: 28 hours or more
o Age 15-17: 43 hours or more

The quantitative study is a cluster randomized control trial (cRCT), which exploits a gradual horizontal expansion of the cash plus programme to an additional 245 WSGs (starting from the 310 already being served). The 245 WSG will be randomly assigned into two treatment arms (cash plus complementary services and complementary services only, 82 WSGs each) and one control arm (81 WSG). The complementary services only arm would start receiving cash after 9 months (November 2023, after the end of the follow-up data collection), when also the control group will start receiving the full cash plus programme. More specifically, the study arms are as follows:
• T1: Cash transfers + complementary services to WSG members, starting March 2023. A total of five cash transfers payments of 45,000 XOF are planned in 2023 (March, April, June, July, August).
• T2: Complementary services only to WSG members, starting May 2023 (this study arm will then start receiving cash transfers in November 2023)
• Control: No intervention to WSG members until November 2023 (cash transfers + complementary services will start in November 2023)
Randomization of WSG into the three study arms will be carried out at village level, with all WSGs in the same village being assigned the same treatment. Prior to randomization, villages will be stratified according to their size in terms of the number of WSGs, considering that some villages have high concentration of WSGs. WSGs have, on average, 25 members.

Quantitative data collection instruments
At baseline, the impact evaluation included five types of instruments:
• Quantitative household survey: administered to the head of household or another adult sufficiently informed about household matters. Key outcomes measured include modules on household socio-demographic characteristics, subjective well-being, schooling, health status, time use, housing, household assets, expenditures, economic activities, land / livestock, revenues, savings and credit, consumption, COVID-19 awareness, practices and perceived impacts, social assistance, shocks and coping strategies. In order to track households between waves, in addition to the exact location, the contact details of the interviewed households as well as of their neighbours were collected. This will help to track households and main respondents at follow-up.
• Quantitative women’s questionnaire: to be administered to women WSG members to whom the cash transfers are targeted – the same woman will then be tracked to be interviewed in follow-up surveys. Key outcomes to be measured include subjective well-being, nutritional knowledge, child health, child nutrition, child development, anthropometrics, income generating activities, land, crop use and sales, agricultural production, revenues, savings group practice and perceptions, social support, mental health, bargaining power in household decision making, gender preferences and attitudes, gender norms and exposure to intimate partner violence.
• Quantitative WSG questionnaire: to be administered to WSG president to determine group composition, participation rates, savings amounts mobilized, access to formal financial services, including credit, credit volumes disbursed by the group, consequences of COVID-19 on group operations and current challenges.
• Quantitative community questionnaire: to be administered to community leaders or other well-informed members of the community in every village to explore socio-demographics of the village, access to markets and social services, prices, gender social norms, shocks (including COVID-19), agriculture, sustainable environmental practices and existing humanitarian/ development programs.

At follow-up, three modules will be added to the quantitative women questionnaires: child discipline, vaccinations, and programme’s operational performance. Moreover, the following questionnaire will be added:
• Quantitative questionnaire for Mama Yeleen: The questionnaire will gather information on the specific training received by Mama Yeleen, and the training activities delivered by them.

Data analysis
The data collected will be analysed both at baseline and endline. Baseline data will be used to examine balance between the three groups (two treatment and one control group) with respect to the outcome variables of interest and among relevant household and individual characteristics using simple Wald tests of equality. This is an essential requirement to verifying that the randomization worked and a valid counterfactual is established. Knowledge of the similarities and differences between the groups will inform selection of control variables for the impact estimation model.
We intend to use Analysis of Covariance (ANCOVA) to estimate impacts using baseline (June/July 2022) and endline (November/December 2023, about 16 months after baseline) surveys. We will test for robustness of results using difference-in-difference methods. For outcomes only collected at endline, cross-sectional impacts will be calculated, including baseline covariates. Standard errors will be clustered at the level of randomization, i.e. the village level.

Heterogeneity analysis
Key contextual factors hypothesized to moderate programme impact include gender norms, marital status/household structure (monogamy vs polygamy). These indicators will be interacted with the treatment indicator to examine heterogeneous effects along these measures.

Multiple hypothesis testing
For the primary outcomes, the analysis will apply a correction for multiple hypothesis testing (MHT) to account for the large number of hypothesis tests. When estimation many different impacts, there is a risk of falsely rejecting a null hypothesis (i.e. finding a significant impact) just by chance. The analysis will apply the procedure to control for this False Discovery Rate (FDR) proposed by Anderson (2008) based on Benjamini, Krieger and Yekutieli (2006) and will calculate sharpened q-values for selected indicators. We will also construct domain indexes by aggregating several indicators of the same domain into one index to reduce the number of tests that we run and to get a sense of whether the programme had a general effect on certain domains. These domains will be determined at the household, woman and child level and their indicators will include:
• Household wellbeing index: household consumption per AE, household FSC and household CSI
• Women’s wellbeing index: women’s empowerment score, mental health and IPV
• Child wellbeing index (among children under 5 at baseline): health status in the last 2 weeks, nutritional status and early childhood development

The sampling frame for the study is the list of the 245 WSGs and their members selected for programme expansion. The registry contains basic information on geographic location, WSG name, and the number of members. As mentioned earlier these groups will be randomized into three treatment arms, with 82 WSGs in T1 and T2 and 81 WSGs in the control group.
At first stage, 40 WSGs will be randomly sampled from the three study arms, maintaining the stratification by village size used during random assignment. At second stage, considering that the lists of WSG members will be available at the time of data collection, 15 women WSG members, age 18-49, for the individual and household-level samples will be selected using simple random sampling.
Sampling will be prepared ahead of time, given the availability of WSG lists. If the woman cannot be found or refuses to participate, enumerators will be provided with a list of replacement respondents. Replacement women will be drawn at the same time as the main sample to provide an additional 5 replacements in each WSG.
Randomization into study arms will first be done at the village level, to the extent possible, 1 WSG from each village will be randomly selected from each of the villages into the study. Aiming for interviews with 15 women per WSG required selecting approximately 40 WSGs in each treatment arm. Accordingly, baseline data will be collected from a panel of 40 WSGs randomly selected for interviews from each study arm, and interviews will be conducted with 14-15 women per WSG (about 580 per study arm) and their corresponding households. At follow up, the same sample of WSGs, women and their households will be interviewed.

Village

Yes

71 villages

1740 (580 per treatment arm)

T1: 23 villages, 40 WSG, 580 women/households
T2: 23 villages, 40 WSG, 580 women/households
T3: 25 villages, 40 WSG, 580 women/households

The minimum detectable effect size (MDE) of a treatment effect of the cash-plus intervention (T1) compared to the control group is a 12 percentage point reduction in the percentage of households living in monetary poverty (from a base of 31%), when using a confidence level (alpha) of 95%, power (beta) of 80%, attrition rate of 10% and an intra-cluster coefficient (rho) of 0.0772 (calculated from baseline data). Similarly, the MDE of a treatment effect of the cash-plus intervention is a 0.295 SD increase in the level of household consumption, using a confidence level (alpha) of 95%, power (beta) of 80%, attrition rate of 10% and an intra-cluster coefficient (rho) of 0.0912 (calculated from baseline data). As for the services only intervention (T2), we calculate a MDE of an increase of 3.6 points on an index measuring the share of correct answers on 12 key family practices (from a baseline value of 51.25 points out of 100) using a confidence level (alpha) of 95%, power (beta) of 80%, attrition rate of 10% and an intra-cluster coefficient (rho) of 0.0793 (calculated from baseline data).