Experimental Design Details
Randomization
In the 2017/8 school year, Hajati was operational in 205 schools (DSS). This number was reduced prior to the start of the 2018/9 schoolyear in two steps:
1. Only schools in the four governorates with the largest number of Hajati DSSs were considered: Amman, Irbid, Mafraq, Zarqa (180 DSSs)
2. The three least vulnerable of these DSSs (based on average vulnerability score of children attending the school) were removed (177 DSSs).
Hajati components were randomly allocated to these 177 DSSs. As part of this randomization process, DSSs that are within close geographical proximity were considered as one cluster. This procedure effectively combines the 177 DSSs into 160 DSS clusters. This step was taken to avoid randomizing schools that are close to each other into different treatment arms. Randomization was carried out by the study team by means of a random number generator. Within each of the four governorates, DSS clusters were randomly selected into one of four mutually exclusive groups: a group receiving both cash transfers and the newly introduced encouragement messages (T1), a group receiving only encouragement messages (T2), a group receiving only cash transfers (T3), and a group not receiving either intervention (T4).
The final allocation of schools by treatment arm is: 39 DSS clusters in T1; 41 in T2; 40 in T3; and 40 in T4. The fact that the number of schools is not exactly equal to 40 in T1 and T2 is an artefact of the stratified randomization; the number of clusters per governorate is not always a multiple of four. We relied on (and adapted) McKenzie and Bruhn’s code as provided in their World Bank Impact Evaluation blog.
Sampling Frame
In the 2017/18 schoolyear, Hajati covered 55,922 school-aged children (6-15) living in households that could be classified as: (i) living in non-camp settings, (ii) vulnerable in accordance with the baseline targeting criteria, and (iii) having at least one child aged 6-15 in one of the 205 double shifted schools. Under the planned budget envelope, even in the restricted set of 39 T1 and 40 T3 DSS clusters not all of these households could continue to receive cash benefits. A decision was therefore made to continue to allocate Hajati cash only to the 25 most vulnerable of these households in each of the T1 and T3 DSS clusters. This procedure was expected to result in about 2,000 households receiving cash benefits and, given that households are expected to have a little over 3 eligible children on average, nearly 7,000 beneficiary children.
One issue we have to account for is that children from the same household may attend different schools. This leads to two complications. First, within the same household, one child may be assigned to a school eligible to receive ‘more’ Hajati benefits than the other. To deal with this issue, the following decision was made. If households have at least one child in a T1 DSS cluster (cash and encouragement), all children will be eligible to receive T1 benefits. If households have no child in T1, but have at least one child in T3 (cash only), all children will be eligible to receive T3 benefits. If households have no child in T1 or T3, but have at least one child in T2 (encouragement only), all children will be eligible to receive T2 benefits (regardless of school assignment). Second, if children attend multiple schools, and we count the household for both schools, our sample size drops below 25 per DSS cluster. To deal with this issue, a decision was made to apply a weighting procedure. If the children in the household attend only one DSS, its weight will be equal to one. If the children attend two DSSs, its weight will be equal to 0.5 (1/2) and so forth. The weighted sum of all households in a DSS cluster needs to be at least 25 (and below 26).
Data
Baseline data will consist of the targeting data collected as part of the initiation of the Hajati program. Endline interviews will consist of a household questionnaire and a child-level questionnaire to be administered to one randomly selected child aged 10 to 16 (at endline) per household. At the time of writing, endline data collection was scheduled to start in the second half of February, 2019.
The household questionnaire builds and expands on the baseline/targeting questionnaire administered at the beginning of the schoolyear 2017/8; it includes modules on health, education, children’s experience in school, living conditions, WASH, access to facilities, expenditures, food consumption, assets, aid, payment abilities and operational performance.
The child questionnaire draws heavily on standardized modules that measure children’s mental wellbeing and have been pre-validated in Arabic. It will be administered to children age 10 to 16 years old at endline. Modules include among others: mental health, self-esteem, social support, trust and optimism, aspirations, fertility, education, violence at home, time use, expenditure and food security.
In addition, we are currently exploring the possibility of implementing a short school questionnaire. The school questionnaire would comprise basic questions to teachers on children’s school attendance and performance in school. In addition, enumerators would be asked to carry out a spot-check, to see if enrolled children are in school on the day of their visit.