Unconditional cash transfers in Kiryandongo refugee settlement, Uganda

GiveDirectly is an international nonprofit that implements unconditional cash transfer programs directed at communities living in extreme poverty. GiveDirectly implements a range of transfer designs across its projects, including one-off amounts that roughly equal the annual income of poor households. To ensure the integrity of the process, GiveDirectly employs a set of independent checks to verify that recipients meet the eligibility criteria and were not asked to pay bribes during the selection process.

GiveDirectly aims to understand the role of unconditional cash transfers as one possible route to improving the self-reliance of refugees in protected displacement contexts. The program intends to provide around USD$1000 in unconditional cash transfers to all of the ~10,000 refugee households in Kiryandongo Refugee Settlement and about 5,000 host community households in Kiryandongo district, Uganda.

UNHCR estimates indicate that there are around 1.4 million refugees in Uganda. Many of the forcibly displaced people are expected to remain in Uganda over the long-term given the prolonged character of the conflicts in their countries of origin. The expected long-term stay and progressive refugee policies in Uganda designate the country as an appropriate setting to analyze the lasting impacts of cash transfers.

2020-02-10

2020-04-30

The primary outcome is the economic well-being of our households. This will be measured using three primary indicators:
1) Household consumption, 2) Household assets (including savings and loans), and 3) Agricultural and non-agricultural business revenue

1) Household consumption: This indicator measures the total monthly value of household consumption and spending including food consumption, consumption of temptation goods, and spending on non-food items. Specifically, the indicator will be constructed as follows:
a) Sum of food consumed by the household at home and away from home in the last 7 days, multiplied by 4.3 to obtain a monthly value, b) sum of temptation goods (alcohol, tobacco, lottery and gambling) consumed by the household at home and away from home in the last 7 days, multiplied by 4.3 to obtain a monthly value, c) sum of frequent non-food spending in the last 7 days (e.g., fuel and transport, etc.), multiplied by 4.3 to obtain a monthly value, d) sum of frequent non-food spending in the last 4 weeks (e.g., water, rent, etc.), divided by 4 and multiplied by 4.3 to obtain a monthly value, and e) sum of infrequent non-food spending in the last 12 months (e.g., clothing, education expenses and taxes, etc.), divided by 12 to obtain a monthly value.
All total values are added, divided by the World Bank’s 2018 Ugandan Shillings (UGX) and United States Dollar (USD) Purchasing Power Parity (PPP) conversation rate, and winsorized at the top 1%.

2) Household assets (including savings and loans): The assets indicator estimates the total current market value of common moveable and unmoveable household assets (including livestock), savings and the net balance of currently outstanding loans given and taken. All total values are added, divided by the World Bank’s 2018 Ugandan Shillings (UGX) and United States Dollar (USD) Purchasing Power Parity (PPP) conversation rate, and winsorized at the top 1%.

3) Agricultural and non-agricultural business revenue: This indicator measures the total monthly revenue from all formal and informal enterprises fully or partially owned by any household member including agricultural (based on crops sold) and non-agricultural business revenue. Monthly agricultural revenue is calculated by collecting total agricultural revenue from the six months prior to the survey and dividing by six. The total revenue from all agricultural and non-agricultural businesses are divided by the World Bank’s 2018 Ugandan Shillings (UGX) and United States Dollar (USD) Purchasing Power Parity (PPP) conversation rate, and winsorized at the top 1%.

We will investigate the following supplementary outcomes to understand other aspects of welfare: 1.1) Psychological well-being, 1.2) Food security, 1.3) Female empowerment, 1.4) Business ownership, 1.5) Employment status of survey respondent, 1.6) Migration, and 1.7) Household composition. We will also explore two community level outcomes as part of the study, including: 2.1) Social cohesion, and 2.2) Safety and security.

1.1) Psychological well-being: This individual-level outcome variable is based on a composite index following the procedure proposed by Anderson (2008) and adapted from Haushofer and Shapiro (2016). It comprises the standardized weighted average of the following components: a) The total score of ten elements of the Center for Epidemiologic Studies Depression Scale (CES-D scale), which surveys the frequency of different types of feelings among respondents, b) the total score of the four elements of the Cohen’s stress scale, c) the numerical responses to the World Value Survey's sections that focus on happiness and life satisfaction, and d) the total score from a thirteen-element-custom-worries questionnaire. All components are coded in a way that higher values represent higher levels of well-being.

1.2) Food security: The food security index is a standardized weighted index which measures the number of days in the last seven days that adults and children: skipped or cut meals; went entire days without food; and went to bed hungry. It also includes the number of meals eaten yesterday that contained protein. Responses are appropriately signed so that higher values represent better food security. The index is taken from Egger et al. (2019) to allow comparability across GiveDirectly projects.

1.3) Female empowerment: This individual-level composite index comprises the standardized weighted average of a time-use measure that captures the proportion of hours spent on household and care work by female respondents on the day before the survey (negatively coded), the proportion of school-aged girls enrolled in school, and the average number of days school-aged girls attended school in the last five days when school was open. School-aged girls are between 6 and 18 years old. The index is created using the inverse-covariance weighting technique of Anderson (2008).

1.4) Business ownership: This indicator is based on a binary variable that takes the value 1 if the respondent's household has one or more businesses as of endline date.

1.5) Employment status of survey respondent: This individual-level outcome measures whether the survey respondent is in formal or informal employment or self-employment. This is measured for households with a working-age (15-64) respondent.

1.6) Migration: These outcomes measure the actual migration pattern of the household. We measure the number of household members who have migrated out of the settlement since baseline, and number of household members who have migrated into the settlement since baseline.

1.7) Household composition: This outcome measures the size of households and the dependency ratio as scattered sub-outcomes: a) Household size: Total number of household members, and b) Dependency ratio: ratio of dependents – people younger than 15 or older than 64 – to the working aged population (ages 15-64).

2.1) Social cohesion: Community level data is collected about relationships within the refugee settlement and also among refugees and host communities. This composite index is based on a custom social cohesion questionnaire covering a subjective rating of the relationships for different groups of interest (refugee-host relations, inter-refugee relations, and intra-family relations) and the self-reported frequency of disputes between refugee and host communities. It is coded in a way that higher values represent higher levels of social cohesion.

2.2) Safety and security: This composite index is based on a custom safety and security questionnaire covering self-reported incidences of crimes experienced by the respondent’s household, a worries scale about crime and safety in the community, and perception of crime and conflict levels in the community. It is coded in a way that positive values represent higher levels of safety and security.

The study has three major parts: 1) A randomized control trial will be used to measure the causal impact of the cash transfers for refugees, 2) A longitudinal qualitative study to examine how and why change does (or doesn’t) take place in refugee and host community households, and 3) An observational study will examine cash transfer recipients from the host community.

The mixed methods approach will examine the impact of the cash transfers by carefully identifying what changed as well as why and how the change happened. Results of the program will inform future decision-making regarding cash transfers by donors and implementing organizations. This pre-analysis plan primarily relates to the randomized control trial. Details regarding the analysis approach for the longitudinal qualitative study can be found in the attached, more detailed analysis plan.

The randomized control trial will use a phased-in design. Under GiveDirectly’s program, eligible households in the refugee settlement have been divided into 24 cohorts through a public lottery (see ‘Assignment method’ below). Cash transfers will be distributed by cohort, with cohort 1 receiving its cash transfers in the first, cohort 2 receiving its cash transfer second etc. Cohorts are expected to be phased-in on a monthly basis, though in practice operational factors will cause some deviations from the planned schedule. For the randomized control trial’s phased-in design, the treatment group will consist of households in cohorts 1, 2, and perhaps 3. The control group is households in cohorts 17-20.
The timing of this study overlaps with the global COVID-19 pandemic of 2019-2020. The first transfers to our treatment group took place in late February 2020. On March 30, Uganda’s president ordered a nationwide lockdown, which restricted almost all movement and commerce throughout the country. This had an immediate effect on livelihoods in Kiryandongo, as many members in the settlement make money by traveling for casual labor or through running small enterprises. Therefore, the study will provide evidence as to the efficacy of cash transfers in the context of the COVID crisis.

The sampling frame will be built on data provided by UNHCR which encompasses information on all (~10,000) registered refugee households living in Kiryandongo. UNHCR and GiveDirectly have identified around 1005 households as PSN (persons with specific needs), who will receive the cash transfer first but will not be a part of the study. The remaining 8886 non-PSN refugee households were placed in a public lottery for random assignment into one of 24 cohorts. The lottery was conducted during the World Food Programme’s (WFP) monthly food and cash distributions in August 2019. The lottery was run publicly to ensure transparency and buy-in of the community. This approach was recommended by community leaders.

During the lottery, each household was given the opportunity to draw a cohort number. A household representative drew a single number from a bucket containing 24 ping-pong balls labeled 1 to 24. The number that each refugee picked corresponds to their assigned cohort number. After each draw, the ping-pong ball was returned into the bucket for the next participant. This lottery with replacement approach was chosen to ensure that each respondent had the same chance of picking a number between 1 and 24.

574 households that did not attend food and cash distribution were randomized in a separate public event, during which community leaders picked the cohort number on behalf of those remaining households.

One of the limitations of this large public lottery randomization process came from the practical challenge of ensuring no tampering from interested groups. The early cohorts 1 and 2 together comprise 9.56% of the non-PSN settlement population, compared to the expected value of 8.33% (2/24) and are thus over-represented. The later cohorts of 23 and 24, on the other hand, together represent 7.5% of the lotterized households and are thus under-represented by 0.83% compared to the expected value of 8.33%. These differences from the expected value are statistically significant at p<0.01, indicating some tampering with the lottery process. Our control cohorts (17 to 20) together represent 16% of the non-PSN settlement population, compared to the expected value of 16.67%. The difference is not statistically significant (p=0.0886). Despite these disappointing findings from the lottery, we believe we can still achieve credible results given the limited magnitude of tampering and the ability to control for some of the differences.

The treatment and control groups have been constructed by a random allocation of refugee households through the public lottery. Randomization will happen at the household level. For the purposes of this study, a household is defined as a group of individuals who have registered as a single household under the UNHCR/OPM system.

No

Design not clustered

The treatment group consists of refugee households in cohorts 1 and 2 who will have received their cash transfer before the endline survey (note that we may add some households from cohort 3 to the treatment group if non-compliance or attrition cause statistical power to dip to unacceptable levels). The control group consists of a random sample of households from cohorts 17-20 who will have not yet received their cash transfer at the endline survey. There are no additional criteria beyond being a GiveDirectly cash transfer recipient, the assignment to treatment timing through GiveDirectly’s public randomization, and the ability and willingness to provide informed consent to participate in the study. Prior to the lottery process, we planned to sample approximately 1500 households (750 treatment, 750 control). This would have resulted in a minimal detectable effect size (MDES) of 0.145 SDs with 80% power. This effect size is less than the 0.18 SD effect of business revenue observed in a previous GiveDirectly evaluation (Haushofer and Shapiro 2016). IDinsight has powered the study on business outcomes, as the smallest effect size is expected for business outcomes compared to other outcomes of interest. During baseline data collection, IDinsight learned that some physical households will receive more than one cash transfer. This is because cash transfers will be made based on a list of households as registered with UNHCR. However, the group of people that live together and the “household” registered with UNHCR are not always the same. This would be problematic for the study as some of our control households live with family members who will receive a cash transfer before our endline survey, essentially meaning these control households will be treated. Such “spillover” households in the control group, as well as equivalent households from the treatment group, were therefore dropped from the study. As a result of both the issue of multiple transfers per household and challenges locating some of the sampled households, our sample size dropped to 1264 at baseline.

642 households in the treatment group, and 622 households in the control group.

The sample size of 1264 will yield an MDES of 0.16 Standard Deviations (SD).

IDinsight Pre-analysis Plan for RCT of GiveDirectly's UCT program in Kiryandongo Refugee Settlement, Uganda_version 1