Impact evaluation of PROSPERA’s change on higher education scholarship recipients.

The pilot will be conducted during the 2015-2016 and 2016-2017 school years (August 2015-July 2017) over the PROSPERA beneficiaries that by August 2015 are enrolled in first, second or third grade of the randomly selected high schools within the states identified for this study.

On February 2017 both treatment beneficiaries and their mothers will be informed of the change of the scholarship recipient. Beneficiaries will also receive their personalized debit cards through which they will be able to access their monetary transfer.

Several sources of data will be used for the impact evaluation analysis. The collection of baseline data has been substituted by the merge of different administrative datasets (beneficiaries census, standardized educational test –ENLACE-, and the socioeconomic PROSPERA household survey -ENCASEH-). This compiled information will let us perform a descriptive analysis of the target population, prove the validity of the experimental design and confirm the comparability of the treatment and control groups.

The follow-up survey, planned for October 2016, is expected to collect information of variables such as drop-out levels, consumption, decision-making, and risk behaviors after an entire school year period of treatment. This household level survey will collect information for a sample of both beneficiaries and their mothers. This sample will be constituted based on a random selection of those beneficiaries that will effectively be part of the PROSPERA beneficiaries’ census in November 2015.

High schools and beneficiary students will remain in the alternate scholarship scheme (change in the scholarship recipient) for at least two school cycles and the impact evaluation results will inform the government over the decision to change the scholarship recipient at the national level starting August 2017. If the government decides not to scale up the intervention, the alternate scheme will endure for treatment beneficiaries’ cohort participating in the pilot until they graduate from high school (July 2019). In such a case, the government will resume the traditional scholarship scheme to the new cohorts that enroll in high schools beginning 2018-2019 school year.

2016-01-11

2017-07-31

dropout rates, labor markets participation, consumption at the household level, beneficiaries' consumption, decision making, risk behaviors.

We are proposing an experimental impact evaluation by randomly assigning the treatment status (change in scholarship recipient from mother to youth) to the eligible beneficiaries within the selected schools. This design will allow the comparison of the results of interest in the treatment group with the results of those beneficiaries who will remain in the traditional model (mother receives the scholarship).

randomization done in office by a computer

The randomization unit was the school. The sample framework consisted on all eligible high schools, defined as schools in localities with one Bansefi branch office or ATM available within a distance of 5 km or less (4290), through which beneficiaries can access their debit account where scholarships are deposited. Based on this operative criterion, high schools were selected on a two-stage sampling process. In the first stage, 8 states were randomly selected from 28 eligible with probability proportional to the number of eligible beneficiaries within the state. In the second stage, a total of 1250 schools were randomly selected within these 8 states to either control or treatment group.

Once both states and schools were selected, treatment assignment was randomized in a systematic cyclic manner among high schools in order to ensure balance between treatment and control groups within localities. Half of schools (625) were assigned to the treatment group and the other half (625) to the control group. For this purpose, schools were order by localities and treatment assignment was randomized within each locality.

The states of Guerrero, Michoacán and Tamaulipas were explicitly excluded from the pilot for security issues. Mexico City was also excluded due to operative difficulties and because other pilot interventions are being implemented there.

For the baseline analysis with administrative data the whole population of beneficiaries in the sample high schools will be used (around 100,000 registries). For the follow-up survey we are proposing the selection of 4 beneficiaries per high school level as a third stage in the sample selection (15,000).

Yes

1250 high schools

The beneficiaries' population withing the sample high schools is expected to be 100,000 students

Sample size:
625 treatment high schools and 7800 high school students
625 control high schools and 7800 high school students

From a cost-benefit analysis it was determined that the Minimum Detectable Effect required for maintaining positive returns over the change in the high school scholarship recipient ranged from 1.8 to 2.8; thus, we are aiming for a MDE of 2-3 percentage points. For MDE calculations we are assuming (1) a yearly discount rate of 12%, (2) as income base the 2012 national minimum wage according to the statistics national institute, (3) high school level educational yearly returns of 5 and 8%, and (4) a 57 years worklife (assuming that the youth starts labor activities at 18 and retires at 65). The power calculations are based on the dropout rate indicator, as this is our main outcome. The prevalence for dropout rates cursing 2nd and 3rd level of high school in urban areas –where the majority of the pilot eligible schools are concentrated- is 35.10% and 30.82%, respectively. Power estimations suggest a school sample of 610 for female strata and 625 for male strata in 3rd grade of high school (this is the most demanding grade in terms of sample size) in order to get a MDE of 0.03. Based on costs, operation capacity and execution schedule, we selected a sample of 625 treatment and 625 control schools. With this double stratification, disaggregated analysis over the course level and gender can be conducted. The parameters for these calculations are the following: power level (1-β)=80%, confidence level (1-α)=95%, intracluster correlation (ICC=0.133), dropout rate (32.51).