Randomization Method
The sample used for AO’s evaluation was selected using a three step process that involved: defining a universe of communities eligible to participate in the program within the Sololá region, dividing this universe into six cluster-level strata, conducting strata-level lotteries that randomly assigned communities to treatment or control, and, finally, selecting a random sample of respondents, within each treatment and control community, to participate in the baseline. These processes are documented in detail elsewhere, nonetheless we summarize them below:
1. Identify eligible communities: The universe of communities eligible to participate in AO was selected using data from the 2011 ENCOVI. The Council selected the Sololá Region for the evaluation, as this area was included in its qualitative work on the links between social norms and girl’s educational attainment. It identified which Sololá communities were eligible to participate in the program by identifying all K’iche’-speaking municipalities which had not previously participated in the program. Once the later were identified, the Council refined its eligibility criteria to include only communities with 500-1500 inhabitants and approximately 30-40 girls, aged 8-19. These criteria were employed, as the Council’s experience suggests this size population is required for the successful implementation of AO. The total number of communities identified by this process is 40, all of which are included in the evaluation.
2. Create strata of eligible communities: Prior to assigning communities to treatment and control, the sample of eligible communities was divided into strata formed by region (highland, lowland) and school type variables (public secondary school, private secondary school, no secondary school) derived from geographic data and the Council’s community census. The final number of strata identified at this stage is 10, the number of communities per strata ranges from 2-7.
3. Randomly assign eligible communities to treatment and control: To assign communities to the evaluation’s treatment and control groups, we used a single-round, random assignment algorithm that placed half of the communities in each strata in the treatment group and half in the control group. To ensure parity in the final number of treatment and control communities, in the first strata with an odd number of communities, we assigned the odd observation to treatment, in the second strata, we assigned it to control and so on.
4. Select a random sample of respondents within treatment and control communities: The Council did not have direct control over the process used to select the community-level sample of approximately 30 girls, aged 12-17, that were invited to participate in the baseline and, possibly, future data collection rounds. Nonetheless, the Council did instruct UNIMER, the external firm responsible for the baseline, to complete this task by randomly selecting the number of girls required from each community from a list of girls located in its community-level census. This step was implemented using Stata 14 on a Population Council computer.