Evaluating alternative targeting of social assistance in fragile settings

The interventions in this study occur on two levels: the community leaders and the community members. We describe each of these as follows.
Intervention 1: Ranking of households and allocation of hypothetical and actual cash transfers by community leaders
We ask community leaders to rank households in their respective EAs from the most to the least needy based on their needs assessment for social assistance. We provide community leaders with a lump-sum of cash transfers that they will allocate among their ranked households based on community level random assignment into different budget categories and discretion levels built into the experimental design (see Experimental Design section). We exogenously vary the nature of the transfer (hypothetical versus real), the amount of money available for transfers, and the level of discretion granted to community leaders. This would allow us to study, for example, whether discretion in the selection of beneficiaries and allocation of funds makes community-based targeting more Rawlsian - maximizing the welfare and wellbeing of the worse-off households (Rawls, 1999; 2001) or Utilitarian and hence-maximize the sum of the individuals’ utilities within the community regardless of differences in individual wellbeing (Arrow, 1973; Yaari, 1981; Mill, 1993).

Intervention 2: Peer- and self-ranking of households
Similar to the community-based ranking, we ask all survey households in each EA (approximately 20) to rank themselves and other fellow households in our sample in the same EA from the most to the least needy. Respondents provide these ranks with a view of targeting for social assistance. We build on the 2019 survey, which contained rich information on pre-conflict status of households including consumption, assets and other important modules that allow us to estimate alternative targeting measures including PMT.
These two interventions allow us to compare the rankings from community leaders and peers against the constructed consumption-, poverty-, asset-and a parametrized PMT-based targeting using pre- and post-conflict data.

2023-11-10

2024-01-31

Primary outcomes
1. Households’ access to CBT transfers (whether the household receives a transfer or not)
2. Amount of transfer received by households.
3. Amount of transfer devoted to running costs or administrative costs by community leaders
4. Distribution of transfers or inequality in the distribution of transfers

Primary outcomes explanation
1. Households’ access to CBT transfers (whether the household receives a transfer or not). The first outcome we consider is a binary outcome capturing whether a household receives transfer from community leaders from the pool of funds community leaders are given to disburse to households. The number of households or the share of households receiving transfer from the community leaders is one of the key measures we employ to capture how community-based targeting functions under different scenarios and settings. Financial constraints and discretion may encourage community leaders to distribute social assistance differently. Exposure to conflict may also affect how community leaders target beneficiaries.
2. The amount of transfer households receive from CBT transfers. This slightly relates to the first outcome and hence measures the amount of transfer households receive from community leaders. This considers the intensive margin of participation in social assistance. The amount of transfer each household receives is related to the number of households selected to receive cash transfers. Community leaders may maximize the total number of households covered by a social assistance program or the amount of transfer each household receives. Thus, the first and second outcomes will be analyzed jointly.
3. The share of resources used to cover administrative costs of community leaders. Community leaders were offered the option of keeping up to 10 percent of the budget allocated for the community as remuneration for their time. First, each community leader will be asked to decide independently how much he/she would like to keep as a “running cost” for the group’s operations. The choice set contains the following options: 0 percent, 2 percent, 4 percent, 6 percent, 8 percent, and 10 percent. Then the full group of community leaders will decide on a negotiated amount to keep for themselves. This allows us to study how alternative designs of CBT affect negotiation behavior and whether negotiated decisions generate better pro-social outcome for the community compared to individual decisions.
4. Distribution of transfers or inequality in the distribution of transfers, measured by Gini coefficient. This outcome uses Gini coefficient based on the amounts of transfers received among members of the community. This measure allows us to identify whether some features and designs associated with CBT make community leaders inequality averse or not.

Secondary outcomes
1. The distribution of transfers across beneficiaries
2. The number of beneficiaries in each village
3. Average transfer received by households

In this section, we outline the experimental design of the intervention at the community leaders’ level. As discussed above, community leaders of the 180 communities are the key actors in this intervention. The intervention follows community level clustered randomization in which the 180 communities are randomly assigned into one of four groups/ treatment arms. The treatment assignment is based on (i) whether communities receive actual transfer or hypothetical (control), (ii) the nature of discretion given to community leaders when allocating transfers (full discretion versus rule-based) and (iii) the size of the transfer pool available to community leaders to distributed among households within the community (constrained budget involving 10,000 Birr versus relaxed budget of 20,000 Birr). The treatment arms generated by combinations of these treatments are outlined below:

(1) Control: Rule-based targeting using hypothetical transfer of 20,000 Birr (C): This group serves as a control cluster where community leaders are not given any actual funds but are instructed to act as if they have a hypothetical budget of 20,000 Birr to distribute among households in their community. Community leaders are first asked to rank households based on their need for social assistance. They are then asked to allocate this notional budget among the 20 households included in our sample. During this ranking process, leaders are required to strictly adhere to pre-defined rules provided by the research team. These rules are carefully selected to mimic the targeting criteria used in actual social assistance programs in Ethiopia. More specifically, community leaders are asked to prioritize those households who: (i) had difficulty satisfying their food needs; (ii) own no or little asset (e.g., livestock, land); (iii) have limited income-generating activities or capacity; (iv) have lost productive assets due to shocks (e.g., conflict, drought); and (v) have lost family members recently.
(2) Rule-based incentivized targeting with relaxed budget (T1): Another group of communities are randomly assigned to a cluster that receives real transfer funds with a budget of 20,000 Birr (about 360 USD). In this cluster (i) Community leaders are required to rank households based on five pre-determined targeting criteria and allocate the transfers. These criteria are similar to those in the control group and mimic the targeting criteria used by the national safety net program in Ethiopia, the PSNP (e.g., Gilligan et al., 2009; Hoddinott et al., 2012; Abay et al., 2022).
(3) Rule-based incentivized targeting with constrained budget (T3): This group of communities follows similar rules as those in control group, but they receive a constrained budget of 10,000 Birr (about 180 USD). Community leaders are required to rank households based on the five criteria outlined above and allocate the 10, 000 Birr to the community members in our sample. These criteria are designed to mimic the targeting criteria used by the national safety net program in Ethiopia, the PSNP. This treatment arm serves to test the implication of budget constraint.
(4) Incentivized-discretionary targeting (T4): The fourth group of communities are provided a budget of 20,000 Birr to distribute as social assistance to households identified as in need. Here, community leaders rank households based on their own criteria they collectively agree upon. The establishment of these ranking criteria is entirely left to the discretion of the community leaders. It is up to the leaders to determine who among the ranked households gets how much of the 20,000 Birr transfer assigned to the community.

Not available

The randomization is done at the village (EA) level using the baseline list of villages. The initial selection of villages into our sample considers their accessibility for a survey. The selected and accessible villages are then randomly assigned into four groups. A reserve list has been prepared in case some of the villages become inaccessible due to conflict.

Village or community level.

Yes

180 clusters

180 communities and about 3,000 households

T1 (Control): 53 villages
T2 (Rule-based, 20, 000 ETB): 41 villages
T3 (Rule-based, 10, 000 ETB): 42 villages
T4 (Discretionary, 20, 000 ETB): 44 villages

As this project is designed to test multiple hypotheses, the power calculation considers these hypotheses and respective outcomes. We compute the number of clusters needed for the primary outcomes described above, assuming that there are a known and fixed number of households in each cluster (village). In the baseline sample there were an average of 20 households in each village, and we anticipate being able to trace about 85 percent of them, 17 households per village. Our power calculations aim to achieve the standard and widely adopted 80 percent power at a significance level of 5 percent. We note that the power calculations are computed and reported only for the primary outcomes. Given that we have several hypotheses and primary outcomes, we computed the number of clusters and associated sample size needed for each primary outcome separately, and then selected the maximum sample needed to detect impacts across these outcomes. To compare the performance of alternative variants of community-based targeting, we compute statistical power and sample size needed to detect a reasonable impact on households’ access to CBT transfers (whether the household receives a transfer), the amount of transfer received by households as well as the share of budget devoted as “running cost”. The rule-based community-based targeting we adopt in this study mimics the targeting approach followed by Ethiopia’s flagship national social safety program, the PSNP. This allows us to exploit important information on the distribution of PSNP participation in our baseline data. Our design introduces three important variations that generate three hypotheses related to these changes to the usual rule-based CBT: (i) what happens when real stakes and payments are introduced; (ii) how do community leaders prioritize allocation when they face resource constraints, and (iii) does granting community leaders more discretion on the criteria to be used for ranking and distributing the transfers matter. To test these hypotheses, we employ alternative indicators that capture the decision-making processes and hence distribution of the cash transfers offered by community leaders. About 30 percent of households in our sample are PSNP beneficiaries at baseline, and we assume that in the absence of any additional intervention community leaders could choose 30 percent of households to receive for transfer distribution. Furthermore, we hypothesize that reducing the available resources by 100 percent to 10,000 Birr (about 180 USD) can reduce the number of beneficiary households by 17 percentage points, and also reduce the transfer amount that goes to beneficiary households by 35 percent. To detect these impacts, we need 41 clusters in each treatment arm. Community leaders are expected to behave differently in hypothetical versus real transfer scenarios, both in terms of the share of the total budget they take out for “running costs” as well as the way they allocate resources to potential beneficiaries. For instance, community leaders may behave more pro-socially and hence devote smaller resources for themselves in hypothetical scenarios. We assume that in the hypothetical setting, about half of the community leaders might request the maximum amount, while this is likely to increase by 23 percentage points when there are real incentives and transfers. Detecting this impact requires about 50 communities in the control group and roughly three times that in the treatment group. Based on these calculations, we allocate about 30 percent of the 180 communities into the hypothetical arm with no actual transfer and divide the remaining communities into three equal groups. We stratify the random assignment of communities across regions.