Local Governance and Crisis Response in Mali

To answer our research questions, we implement two experiments. The first is related to anticipatory action programming and implements an experimental game with local leaders in Mali, varying key features of how anticipatory action would be implemented to understand how these differences might change support for anticipatory action vs. humanitarian aid among local leaders. The second experiment focuses on aid transparency and discretion. This is designed as a discrete choice exercise, asking respondents to choose between aid profiles that vary along different parameters in order to understand how local leaders value transparency (or lack thereof) over aid distribution and local discretion over aid distribution.

To answer these questions, we implement two experiments. The first is related to anticipatory action programming. All respondents are informed of several critical pieces of information using a uniform script and ``storyboard'' designed to be accessible to respondents with low literacy levels who have likely never heard of AA programming before. This storyboard is provided as an attachment in our pre-analysis plan. We designed this cartoon-based ``storyboard'' format to maximize comprehension and ensure uniformity of comprehension, as we expect that some respondents may have low literacy levels and as we expect that the concept of AA will be new to some respondents. We piloted the storyboard to ensure that local actors understood the concepts. We employ two experimental manipulations in order to answer our primary research questions:

(1) We experimentally vary the accuracy of early warning signals. In one condition, respondents are told that early warning systems accurately predict a flood with 50% probability. To ensure that the probability is salient to respondents, they are shown a jar / cup with 5 black marbles and 5 white marbles and told that the number of black marbles represents the number of times out of 10 that a flood would be accurately predicted. In the second condition, respondents are told that early warning systems accurately predict a flood with 20% probability. They are shown a jar / cup with 8 black marbles and 2 white marbles and told that the number of black marbles represents the the number of times out of 10 that a flood would be accurately predicted. In both conditions, respondents are reminded that even if the early warning signal is wrong sometimes, they can still prevent damage over the course of many years by taking preventative actions if they think that their village may get a lot of floods over time.
(2) In the ``donors'' condition, respondents are told that donors would distribute supplies when an early warning signal comes to help prevent flood damage. In the ``local discretion'' condition, respondents are told that supplies for AA would be stored in grain banks, and the commune aid distribution commission would be responsible for moving supplies from grain banks to communities when an early warning signal comes to help prevent flood damage. In both cases, respondents are shown images on the cartoon storyboard of the truck of an international donor distribution shovels and sandbags or a grain bank, respectively, to fix ideas.

After going through this storyboard, we ask respondents to allocate 5 million CFA between AA and humanitarian aid. We then ask respondents to close their eyes and draw a real marble out of the jar / cup which contains the number of black vs. white marbles which they have been experimentally assigned. We then explain that a flood occurred and, based on which color marble they draw, (1) whether the early warning signal would have been accurate and (2) what outcomes their investment in AA vs. humanitarian would result in.

The second, independent experiment is related to discretion and transparency over aid. This experiment is designed as a discrete choice exercise, asking respondents to select between two ``aid profiles'' three different times. To start, all respondents are told: ``Imagine your commune experienced a flood. Two different humanitarian donors could respond in theory, but they have different ways of designing their programming. In both cases, donors are able to respond, but you can see that the amount of aid will not be nearly enough to cover the needs of your commune from the effects of the flood. Given this, and the different donor strategies, which donor do you think would work best for your commune?'' After this prompt, respondents are presented with two aid profiles to choose between, three different times. The aid profiles randomly differ along three key parameters:

(1) Aid discretion: We randomly vary the level of discretion that local leaders have over aid distribution logistics. In the ``low discretion'' condition, the profile indicates that the donor has a formula for identifying needy households in your commune and can deliver aid directly to households. In the ``high discretion'' condition, the profile indicates that the donor would need support from the aid distribution commission in your commune to choose which households to target and to find local people or companies who could handle transportation and logistics for delivering aid.
(2) Aid transparency: We randomly vary the level of transparency provided over aid distribution. In the ``low transparency'' condition, the profile indicates that the donor starts all programming with an information campaign over the radio informing citizens in your commune about how to prevent flood damage. In the ``high transparency'' condition, the profile indicates that the donor starts all programming with an information campaign over the radio informing citizens in your commune about how to prevent flood damage and about how households were selected for receiving aid as well as how transportation and logistics around aid are handled.
(3) Aid type: Each pair of profiles is randomly-assigned to receive one of three aid types: (a) food aid; (b) cash transfers; or (c) agricultural inputs (e.g., seeds). The profile pair which the respondent is comparing always holds constant the type of aid. So whereas the respondent makes a choice between two profiles that vary along one of the above two characteristics, respondents are never making a choice over aid type. We did this because we thought aid type may matter so much that it could reduce our ability to detect effects of the dimensions of more substantive interest. We vary aid type across (not within) profile pairs to make the exercise more engaging. This also allows us to measure preference over aid type using an additional rating question that comes after the final profile comparison.

In theory, there are six unique profile pairings where at least one of the two dimensions (transparency and discretion) varies across the two profiles in the pair (it would be uninteresting if each profile were exactly the same). We focus on the four possible pairings where only one dimension is varied at a time. We thus ignore the two possible pairings in which both dimensions vary simultaneously, e.g., donor 1 has low discretion and high transparency and donor 2 has high discretion and low transparency. We do this to maximize power because our hypotheses all reflect conditional statements in which exactly one dimension varies.

With four possible profile pairings and three pairings per respondent, we constrain randomization so that each respondent sees 1) each profile pairing at most one time (and does not see one of the pairings), 2) each aid type exactly once, and 3) the pairing of aid type with profile comparison is balanced across the sample. Further, we randomly vary the order in which respondents see the type of aid and associated profile comparisons.

After each profile pair, we ask respondents which profile they would select. After the final profile pair, we ask respondents to rate each profile in the pair.

2024-03-15

2024-04-30

For the anticipatory action game: Allocation to humanitarian aid vs. anticipatory action programming; support for anticipatory action programming (4-pt scale) (AA7); satisfaction with initial allocation decision (AA4); desire to switch allocation (binary, AA5); new allocation decision to humanitarian aid vs. anticipatory action (AA6)

For the aid profiles experiment: Chooses high transparency profile (binary); Chooses high discretion profile (binary); Aid profile rating (1-10)

The study is being conducted in approximately 125 communes in rural Mali across 7 regions. In each commune, we aim to recruit the mayor and four other leaders at the commune level: the state-appointed civil servant - the Secretary General, an opposition politician, and 2 civil society leaders. Within each commune, we will also sample village-level leaders in four villages: the village chief, the women's leader, the youth leader, and another resource person or key interlocutor for the village on whom the chief relies. With approximately 16 village-level leaders and approximately 5 commune-level leaders per commune, we aim to recruit about 2500--2700 respondents in total from across the 125 communes. Each respondent will receive both of the experiments described here.

The study is being conducted in approximately 125 communes in rural Mali across 7 regions. In each commune, we aim to recruit the mayor and four other leaders at the commune level: the state-appointed civil servant - the Secretary General, an opposition politician, and 2 civil society leaders. Within each commune, we will also sample village-level leaders in four villages: the village chief, the women's leader, the youth leader, and another resource person or key interlocutor for the village on whom the chief relies. To identify the four villages, we will sample the commune seat, one village that is has a weaker or oppositional relationship with the commune government, one village that is close to the mayor, and one village that is identified by the mayor as being most-affected by crises in the past year. In cases where the mayor was unable to provide this information to aid in sample selection, we select villages at random within the commune until reaching four villages in total within the commune. With approximately 16 village-level leaders and approximately 5 commune-level leaders per commune, we aim to recruit about 2500--2700 respondents in total from across the 125 communes. Each respondent will receive both of the experiments described here.

For the Anticipatory Action (AA) game, we randomly assign two separate features of the setup. We use simple randomization separately for each. In the first experiment, respondents have a 50% probability of being assigned to one of the following conditions:(1) High accuracy (or an 80% chance of the early warning sign being correct) or (2) Low accuracy (or a 50% chance of the early warning sign being correct). In the second treatment, respondents have a 50% probability of being assigned to one of the following conditions: (1) High discretion (the commune's own aid committee distributes aid) or (2) Low discretion (the donor distributes aid).

For the discrete choice experiment, we randomly assign aid type with 1/3 probability at the level of the donor profile pairing so that each profile that is being compared is holding aid type constant. For the attributes of discretion and transparency, we do not use simple randomization. We are only interested in a subset of the possible combinations of these two features. In particular, we are interested in the below four contrasts:
(1)The effect of transparency conditional on discretion
(2) The effect of transparency conditional on discretion
(3) The effect of discretion conditional on no transparency
(4) The effect of discretion conditional on transparency

As a result, we have four distinct pairings of donor profiles where only one of the two attributes -- transparency and discretion -- is different across the two profiles, holding the other one constant. Because we also want to randomize aid type at the level of the donor pairing, this leaves us with 4x3 = 12 possible donor pairings.

Each respondent receives 3 of these 12 possible pairings. However, we want each respondent to be constrained to receive only one of each aid type and, at most, one of each of the four contrasts of interest with respect to discretion and transparency. This leaves us with four unique sets of profile pairs.

Each respondent is thus assigned one of the four unique sets of three profiles with equal probability. To avoid order effects, the order of the three profiles in each set is randomly assigned. And to avoid order effects within each profile pair, the treatment of interest (whether transparency or discretion is High) is randomly assigned to occur in Donor profile 1 and Donor profile 2 with each probability.

This randomization protocol is equivalent to blocking. The assignment of each of the four contrasts is blocked on the assignment of aid type.

individual level randomization

No

2500 leaders

2500 leaders

For the AA game: 1,250 in low accuracy, 1,250 in high accuracy, 1,250 in low discretion, 1,250 in high discretion
For the transparency / discretion exercise: 1,875 high vs. low transparency when discretion is high; 1,875 high vs. low transparency when discretion is low; (3) 1,875 high vs. low discretion when transparency is low; and (4) 1,875 high vs. low discretion when transparency is high.

AA game: In our sample of 2,500, we have 0.99 power to detect an effect size of 0.2 standard deviations with a p value of 0.01667 (to account for our three primary hypotheses). We have a power of 0.87 to detect an effect size of 0.1 standard deviations. Transparency / discretion vignettes: Because each participant gets three of four contrasts and thus for 2500 respondents, N=1875 for this test. For the expected minimum detectable effect size, we consider the effects of attributes in conjoint experiments as a reasonable guide. In a meta-analysis of highly cited forced-choice conjoint experiments, the median average causal mediation effect size was 0.05. Using DeclareDesign, we find that we have a power of 0.99 to detect an effect size of this magnitude, or to detect a difference between an estimate of Y=0.55 and 0.5. We have about 0.73 power to detect a difference between an estimate of Y=0.53 and 0.5. If we implement conservative multiple comparison corrections for our four hypothesis tests in this theory family by dividing our p value of interest by 4 (p<= 0.0125), then we have a power of 0.97 to detect a difference of 0.05 and a power of 0.54 to detect a difference of 0.03. We expect that we will have more power to detect Hypotheses 4, 6 and 7 than Hypothesis 5 because the latter is about the difference between two estimates (preference for discretion when transparency is high vs. low; we expect both will themselves be different than 0.5) rather than the estimate being different from 0.5.