Humanitarian Aid and Natural Disasters: An Impact Evaluation in Pakistan

Last registered on November 23, 2016

Pre-Trial

Trial Information

General Information

Title
Humanitarian Aid and Natural Disasters: An Impact Evaluation in Pakistan
RCT ID
AEARCTR-0001782
Initial registration date
November 23, 2016

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
November 23, 2016, 1:00 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
World Bank

Other Primary Investigator(s)

PI Affiliation
University of Mannheim, C4ED
PI Affiliation
University of Mannheim

Additional Trial Information

Status
On going
Start date
2015-11-26
End date
2019-03-01
Secondary IDs
Abstract
Natural disasters, like earthquakes and floods, can cause great losses in human and physical capital. The impacts can be particularly troublesome in developing countries that often lack preparedness to such events. Catastrophes hit the poor and vulnerable especially hard and have led to recurrent humanitarian disasters in the past years. They heighten already existing vulnerabilities in the communities, such as lack of proper shelter, livelihoods, and sanitation, which contribute to disease spread and malnutrition. Economic research on how to mitigate the consequences of natural disasters through recovery and preparedness interventions represents a real research gap. Among the most natural disaster-affected countries in the world is Pakistan. The underlying study aims to make a contribution to the literature by setting-up a randomized control trial (RCT) in areas which faces the highest needs of recovery and humanitarian aid after having been heavily and recurrently affected by disasters in the past. Pakistan's recurrent disasters leave behind critical gaps and heighten vulnerabilities within affected communities. Natural disasters are often followed by chronic malnutrition. In this setting, humanitarian aid interventions targeting areas that face a high likelihood of being exposed to the next natural disaster or emergency are key to prevent degradation of the often already fragile situation and to make communities more resilient when future disasters occur. The intention of the interventions is to build local capacities, meet life-saving needs, support community-level recovery and enhance resilience for the future.
External Link(s)

Registration Citation

Citation
Avdeenko, Alexandra, Markus Frölich and Juanita Vasquez-Escallon. 2016. "Humanitarian Aid and Natural Disasters: An Impact Evaluation in Pakistan." AEA RCT Registry. November 23. https://doi.org/10.1257/rct.1782-1.0
Former Citation
Avdeenko, Alexandra, Markus Frölich and Juanita Vasquez-Escallon. 2016. "Humanitarian Aid and Natural Disasters: An Impact Evaluation in Pakistan." AEA RCT Registry. November 23. https://www.socialscienceregistry.org/trials/1782/history/11977
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
The program of interest is implemented by ACTED, an international NGO. ACTED has been present in Pakistan since 1993 and has become a leading relief and development aid provider in the country. ACTED’s main goals are to deliver integrated, multi-sectorial and good-value relief and immediate recovery to households in the aftermath of an emergency and to strengthen communities’ capacity to manage and reduce risks and increase their resilience for the future. Besides disaster preparedness and emergency response, ACTED is mainly engaged in delivering programs focusing on the reha- bilitation and construction of infrastructure like shelter, sustainable energy development, livelihood support, agricultural development, education and vocational training as well as water, sanitation and hygiene [ACTED, 2016]. ACTED applies an integrated approach involving the comprehensive implementation of interventions such as shelter and Non-Food Items (NFI), Water, Sanitation and Hygiene (WASH) as well as Food Security and Livelihoods Support (FSL). These activities are also part of the program we are evaluating.

The program `Responding to Natural Disasters in Pakistan 2015-2019' under United Kingdom Department
for International Development (DFID) funding is embedded in the context
and needs highlighted in the previous section. The multi-year humanitarian
program focuses on natural disaster preparedness, response and recovery.
It is implemented as part of a consortium of NGOs under the official
name ``Consortium for Natural Disaster Preparedness, Response and Recovery
in Pakistan''.

Basic Humanitarian Package
In what follows we describe the components of what we call the basic humanitarian package, com- posed of several different activities the ACTED team conducted.
Shelter and Non Food Item Assistance. Affected vulnerable households are supported to in- tegrate disaster risk reduction (DRR) practices into shelter construction in a way that strengthens resilience to local hazards. Strong emphasis is placed on beneficiary-driven design and construction, supported by technical trainings and mentoring in the field. Some beneficiary families receive condi- tional cash support or materials to enable construction. In flood-prone areas, DRR techniques such as raising the plinth, reinforcing the base of the wall with a mud ‘toe’, using a mud-lime combination to plaster walls, corner bracing and construction of lighter roofs are encouraged. Households that have suffered from past disasters (2010-2011 floods mostly) and which are still in need of shelter re- habilitation/construction are targeted. These households receive shelter repair kits as well as shelter construction training by technical staff. The most vulnerable households who cannot participate in own shelter construction receive grants to pay for skilled labor to build/rehabilitate their Shelter. In addition, ACTED implements “Cash for Work” activities, which are expected to generate income for recovery while reviving livelihoods. Cash for work services will mostly take place as rehabilitation or construction of irrigation channels, drainage lines, stoves, and nadi (river) filters. The main intention is not to provide cash, but mostly to improve living conditions and livelihoods infrastructure. In particular, work activities shall provide village-wide benefit.
Water, Sanitation and Hygiene. The WASH component of the integrated recovery package is modeled after the Pakistan Approach to Total Sanitation. This Community Led Total Sanitation approach supports community mobilization to construct their own household latrines. The idea is that through creation of demand within communities (Village Sanitation Committees) and support for supply interventions, communities become open defecation free. This approach includes subsidy support for the most vulnerable in the form of demonstration latrines and the distribution of a ‘sanitation kit’ that provides the key materials needed to build a latrine for those also deemed vulnerable. Support for rehabilitation of water supply schemes in these communities is also part of the program and is based on the exact localities’ needs. The PATS intervention also includes support for behavioral change related to improved hygiene practices and options for household treatment of water (distribution of water filters to part of the beneficiaries).
Food Security and Livelihood Support. ACTED FSL activities involve training on agriculture and water management and livestock management and vaccination trainings. Agriculture and water management trainings aim at building the capacity of farmers by organizing them into water user groups. A quick demonstration of improved farm, crop and water management techniques is taught to the beneficiaries. Additionally, livestock trainings aim to provide essential knowledge and skills to communities that own livestock so that they are able to cope with any calamity and minimize livestock losses through proper mitigation and preparedness. The trainings specifically cover the types of livestock emergencies, risks associated with hazards in livestock spectrum, preparedness plan, and distribution of inputs in disaster hit areas of Sindh as well as disease outbreak handling. Additionally, seeds and other agriculture inputs are distributed to a limited amount of households. Others also benefit from kitchen gardening training.

Additional Packages
In the second stage of the intervention we introduce two types of additional trainings conducted by ACTED. The additional packages have as a goal to transmit key messages of WASH, FSL, and shelter (a) in-person and (b) via telephones.
A. Additional Training - Face-to-face: ACTED performed additional training in the treat- ment villages. Basically this consisted in having two trainers going to the villages and perform- ing an about two hour class training about 25 people once. In this class the trainers taught the residents on one of the three topics, covering how to have better personal and hygiene habits and the importance of using latrines in the WASH training; good agriculture and water man- agement techniques, and livestock management techniques in the FSL training; and building techniques in the Shelter training. In this sense, this additional training sessions had as a goal to convey compact knowledge of these important messages.
B. Additional Training - Phone Calls: The second type of additional interventions exploits the fact that we have mobile numbers for those assessed at baseline (or their neighbors/ entrusted people) and can thus call them to provide telephonic additional training. The callers had a script with the same key messages as in the personal Additional training but written in an interactive, easy to understand way that could be delivered by phone in approximately 20 minutes. The respondent had the opportunity to ask questions. We make in total 1600 calls, with up to eight attempts per call. One concern prior to implementation was the bad telephone connection. Thus, whether the phone calls could be indeed be delivered was closely monitored.
The topics of the trainings were WASH, shelter or FSL, and all individuals in the same village received the calls on the same topic. Not all people from the cluster-village received a call. In fact, we varied whether all individuals from the baseline were called (100%) or only 50%. The latter variation was introduced in order to test possible spill-overs of information between villagers where 50% of individuals were called on the remaining 50%, i.e. whether information is shared between villagers.
Intervention Start Date
2016-03-01
Intervention End Date
2017-03-01

Primary Outcomes

Primary Outcomes (end points)
The two key outcomes of interest are (1) an increase in preparedness for disasters (as measured by intermediate outcomes which capture the application of knowledge delivered in ACTED's training) and (2) the (subsequent) reduction in moderate acute malnutrition. Moreover, in the long-run we expect a general reduction in the prevalence of diseases, an increase in assets (in particular livestock), an increase in income, an improvement in nutritional intake, more positive coping strategies, and an overall more positive subjective assessment of the situation in the village.
Primary Outcomes (explanation)
Intermediate Outcomes:
Y01 FSL:
– Share of households that use soil fertility techniques (mulching, adding animal manure,
leaving the garden fallow) (HH)
– Share of households that use water management saving (laser leveler, drip irrigation)
(HH)
– Share of households that prefer bio-control agents to chemicals to control pests (HH)
– Share of households that take actions to control livestock disease transmission (separation of livestock and own shelter, vaccinating and deworming livestock)
– Share of households that are aware of the needs of livestock (extra care with newborns) (HH)
Y02 WASH:
– Share of households that drink safe water (have access to a functional safe source, save
water in proper and clean containers, treat drinking water) (HH)
– Share of households that have good personal hygiene habits (shower regularly, wash hand
with soap in critical occassions such as after defecating or before eating) (HH)
– Share of households that have access to latrines (HH)
– Share of households that practice safe waste disposal (HH)
Y03 Shelter:
– Share of shelters made of concrete and bricks instead of bamboos or other less resilient
material (HH)
– Share of shelters that incorporate safety improving construction techniques (Built on
natural elevations, strong foundations, walls and roof) (HH)
– Share of households that often and extensively repair their shelters (HH)
– Share of households that live in a non-damaged house (HH)11
Y04 Preparedness:
– We constructed a overall preparedness indicator based on whether household apply FSL,
WASH and shelter key messages (HH)

Final Outcomes:
Y1 Diseases:
– Prevalence of diarrhea (HH)
– Prevalence of other diseases which transmission is related with poor hygiene conditions (malaria, dengue, skin infection, scabies, typhoid fever, polio, Hepatitis A, Cholera, Shigellosis, Pneumonia, Leptospirosis) (HH)
– Prevalence of diarrhea in children under 6 (HH)
– Prevalence of other diseases whereby transmission is related with poor hygiene conditions in children under 6 (malaria, dengue, skin infection, scabies, typhoid fever, polio, hepatitis A, cholera, shigellosis, pneumonia, leptospirosis) (HH)
Y2 Assets:
– Amount of productive assets (crop, land irrigation channels, livestock) (HH)
– Average amount of land cultivated (irrigation and rain fed) (HH)
– Share of households with destroyed or damaged shelters in the cluster-village (FGD)
Y3 Income:
– Average household income in the cluster village (HH)
– Average household outstanding debt in the cluster village (HH) – Average household savings in the cluster village (HH)
Y4 Nutritional status/intake:
– Moderate acute malnutrition (MAM) and severely acute malnutrition (SAM) levels of
children and pregnant and lactating woman. See X6 for definitions.
– Percentage of households with poor or borderline profiles of Food Consumption Score
(HH), see X4.1 for definitions.
Y5 Coping strategies:
– Proportion of households in the highest Coping Strategy Index Score category, see X8 for comparison/description (HH)
– All coping strategies as dummies, see X8 (HH)
Y6 Subjective assessment
– Average life satisfaction in the cluster-village (HH)
– Average feeling of preparedness for a natural disaster or extreme weather event (HH)
– Average belief that NGO’s do a good job (HH)
– Share of households [at least one respondent] in village-cluster that are in need of any assistance in the aftermath of a natural disaster/emergency (HH), see X2 for comparison/ description.
– Number of needs (by topics, FSL, WASH, shelter) in the aftermath of a natural disaster/emergency.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The baseline data were collected in November and December 2015. ACTED carried out a base-needs assessment in 400 at-risk sub-villages of Badin and Kashmore in November and December 2015, and interviewed 4000 households in the process. Additionally 400 focus group discussions (FGDs) were conducted whereby on average 10 participants were asked about the situation in their village; the responses were quantified and recorded. The FGDs were mixed groups of villagers who jointly dis- cussed the questions. We defined as unit of analysis village-clusters, which are geographically distant units of so-called sub-villages (about 1,3 sub-villages compromise one cluster). In our analysis, we included 301 village-clusters. After eliminating non-eligible clusters, 287 village-clusters remained. In each of the clusters about 15 people were interviewed and at least one FGD was carried out. 148 village-clusters were randomly allocated to a basic humanitarian recovery package (step 1). The other 139 were allocated into the control group. The village-clusters allocated into treatment received a humanitarian recovery package delivered by ACTED, which included WASH, shelter, FSL, and cash-transfers interventions. The remaining 139 control villages did not. Moreover, within the 148 village-clusters that already received the humanitarian recovery package, the following treatment arms were additionally randomly allocated: 30 village-clusters receive an additional WASH face-to-face training, 30 village-clusters an additional face-to-face shelter and 30 village-clusters additional face-to-face FSL training. Moreover, we assigned phone calls (by topics) in 208 of 287 village and randomized the number of individuals called per village-cluster.
Experimental Design Details
see attached PAP.
Randomization Method
The randomization was implemented with the computer package Gauss 12.1.0.
The randomization of the village-cluster into treatment and control used a re-randomization procedure that was developed to attain several aims. The main aim of the randomization protocol was
- to attain balance between treatment and control on selected variables;
- possibility of randomization inference;
- to ensure that approximately 100 sub-villages are treated in each of the two districts (Kashmore and Badin);
- to ensure the same number of similar treatment arms in both districts;
- to ensure that nearby sub-villages have the same treatment status (i.e. that is why randomized assignment referred to village-clusters).
Randomization Unit
We randomize the treatment arms in the pool of 287 village-clusters.
As a smaller component of interest we also add individual-level randomization whereby we randomly selected 1601 who receive the training via phone.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
287 village-clusters.
Sample size: planned number of observations
287 village-clusters. Data is collected at the household level.
Sample size (or number of clusters) by treatment arms
287 village-clusters.
148 village-clusters were randomly allocated to a basic humanitarian recovery package (step 1). The other 139 were allocated into the control group.
In step 2 of the randomization we assigned different types of refreshment training to different villages: WASH, shelter, FSL or no additional training. The additional training was only allocated to those villages that were previously assigned the basic package.
Thus, we allocate within the 148 village-clusters that already received the humanitarian recovery package, the following treatment arms: 30 village-clusters receive an additional WASH face-to-face training, 30 village-clusters an additional face-to-face shelter and 30 village-clusters additional face- to-face FSL training.
Additionally, we randomly selected 1601 individuals who were called to receive the same training via phone. These individuals stem from 52 Wash, 52 shelter, and 52 FSL villages. Thereby we implemented phone calls also in 60 control village-clusters. In 50% of the village-clusters we phone-called we called 50% of the individuals, in the other half we called all individuals from our baseline.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Power calculations have been conducted prior to the roll-out of the program, in July 2015. We present estimates based on secondary data. We look only at areas where ACTED planned to operate at that time, namely Punjab, Sindh and Balochistan. If not otherwise indicated, we restricted ourselves to children of age zero to five and women in reproductive years, non-pregnant (age 15-49). To identify the baseline levels we use the following sources of information: 1. We use secondary data from the National Nutrition Survey Pakistan 2011 conducted by the government of Pakistan. 2. We used secondary data from the DHS Pakistan report (2012-2013). 3. Additionally, we use the DHS Pakistan (2012-2013) to recalculate the population share of moderately malnutritioned individuals. We use the same data-set to recalculate the intra-class correlation. 4. Sindh (Pakistan) Nutrition Survey (2011). We assumed ACTED would operate in 100 villages, treating on average 20 families per village. Further assumptions are ρ=0.01 (DHS), power of 0.8, and α=0.05. In the PAP we present different indicators to capture moderate acute malnutrition, one of which indicates that the MDE for children would be 0.037 (MUAC).
IRB

Institutional Review Boards (IRBs)

IRB Name
University of Mannheim
IRB Approval Date
2016-11-22
IRB Approval Number
2016-11-22
IRB Name
University of Mannheim
IRB Approval Date
2016-09-14
IRB Approval Number
2016-09-14
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials