Impact evaluation of PROSPERA’s change on higher education scholarship recipients.
Last registered on February 20, 2016

Pre-Trial

Trial Information
General Information
Title
Impact evaluation of PROSPERA’s change on higher education scholarship recipients.
RCT ID
AEARCTR-0000944
Initial registration date
February 20, 2016
Last updated
February 20, 2016 11:13 AM EST
Location(s)
Region
Primary Investigator
Affiliation
Interamerican Development Bank
Other Primary Investigator(s)
PI Affiliation
Stanford University
PI Affiliation
Interamerican Development Bank
PI Affiliation
University of Maryland
PI Affiliation
Interamerican Development Bank
Additional Trial Information
Status
In development
Start date
2015-08-17
End date
2017-07-31
Secondary IDs
Abstract
The Office of the National Coordination of PROSPERA, the conditional cash transfer program in Mexico, is considering implementing some adjustments to the design and operation of the program in order to improve the services provided to the beneficiaries. One of the proposed interventions is the change in the high school scholarship recipient, from the mother to the beneficiary. High school beneficiaries have become particularly relevant to the program as dropout rates at this level are especially high (27.9%) . As such, one of the program priorities is to test new scholar incentives redesigns in high school.

This intervention will be implemented as a pilot during the 2015-2016 and 2016-2017 school years. The government is interested in evaluating the impact of this change in terms of dropout rates, employment, consumption and risk behaviors before scaling it up at the national level. For this purpose, we are proposing an experimental design, assigning the change in scholarship recipient to the beneficiaries enrolled in the randomly selected treatment high schools. The sample size for this experiment is 1250 high schools (625 treatment and 625 control) within 8 states (Baja California, Chiapas, Durango, Hidalgo, Mexico, Puebla, Veracruz, and Zacatecas).
External Link(s)
Registration Citation
Citation
Araujo, Maria et al. 2016. "Impact evaluation of PROSPERA’s change on higher education scholarship recipients.." AEA RCT Registry. February 20. https://doi.org/10.1257/rct.944-1.0.
Former Citation
Araujo, Maria et al. 2016. "Impact evaluation of PROSPERA’s change on higher education scholarship recipients.." AEA RCT Registry. February 20. http://www.socialscienceregistry.org/trials/944/history/6905.
Experimental Details
Interventions
Intervention(s)
The pilot will be conducted during the 2015-2016 and 2016-2017 school years (August 2015-July 2017) over the PROSPERA beneficiaries that by August 2015 are enrolled in first, second or third grade of the randomly selected high schools within the states identified for this study.

On February 2017 both treatment beneficiaries and their mothers will be informed of the change of the scholarship recipient. Beneficiaries will also receive their personalized debit cards through which they will be able to access their monetary transfer.

Several sources of data will be used for the impact evaluation analysis. The collection of baseline data has been substituted by the merge of different administrative datasets (beneficiaries census, standardized educational test –ENLACE-, and the socioeconomic PROSPERA household survey -ENCASEH-). This compiled information will let us perform a descriptive analysis of the target population, prove the validity of the experimental design and confirm the comparability of the treatment and control groups.

The follow-up survey, planned for October 2016, is expected to collect information of variables such as drop-out levels, consumption, decision-making, and risk behaviors after an entire school year period of treatment. This household level survey will collect information for a sample of both beneficiaries and their mothers. This sample will be constituted based on a random selection of those beneficiaries that will effectively be part of the PROSPERA beneficiaries’ census in November 2015.

High schools and beneficiary students will remain in the alternate scholarship scheme (change in the scholarship recipient) for at least two school cycles and the impact evaluation results will inform the government over the decision to change the scholarship recipient at the national level starting August 2017. If the government decides not to scale up the intervention, the alternate scheme will endure for treatment beneficiaries’ cohort participating in the pilot until they graduate from high school (July 2019). In such a case, the government will resume the traditional scholarship scheme to the new cohorts that enroll in high schools beginning 2018-2019 school year.
Intervention Start Date
2016-01-11
Intervention End Date
2017-07-31
Primary Outcomes
Primary Outcomes (end points)
dropout rates, labor markets participation, consumption at the household level, beneficiaries' consumption, decision making, risk behaviors.
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
We are proposing an experimental impact evaluation by randomly assigning the treatment status (change in scholarship recipient from mother to youth) to the eligible beneficiaries within the selected schools. This design will allow the comparison of the results of interest in the treatment group with the results of those beneficiaries who will remain in the traditional model (mother receives the scholarship).
Experimental Design Details
Randomization Method
randomization done in office by a computer
Randomization Unit
The randomization unit was the school. The sample framework consisted on all eligible high schools, defined as schools in localities with one Bansefi branch office or ATM available within a distance of 5 km or less (4290), through which beneficiaries can access their debit account where scholarships are deposited. Based on this operative criterion, high schools were selected on a two-stage sampling process. In the first stage, 8 states were randomly selected from 28 eligible with probability proportional to the number of eligible beneficiaries within the state. In the second stage, a total of 1250 schools were randomly selected within these 8 states to either control or treatment group.

Once both states and schools were selected, treatment assignment was randomized in a systematic cyclic manner among high schools in order to ensure balance between treatment and control groups within localities. Half of schools (625) were assigned to the treatment group and the other half (625) to the control group. For this purpose, schools were order by localities and treatment assignment was randomized within each locality.

The states of Guerrero, Michoacán and Tamaulipas were explicitly excluded from the pilot for security issues. Mexico City was also excluded due to operative difficulties and because other pilot interventions are being implemented there.

For the baseline analysis with administrative data the whole population of beneficiaries in the sample high schools will be used (around 100,000 registries). For the follow-up survey we are proposing the selection of 4 beneficiaries per high school level as a third stage in the sample selection (15,000).

Was the treatment clustered?
Yes
Experiment Characteristics
Sample size: planned number of clusters
1250 high schools
Sample size: planned number of observations
The beneficiaries' population withing the sample high schools is expected to be 100,000 students
Sample size (or number of clusters) by treatment arms
Sample size:
625 treatment high schools and 7800 high school students
625 control high schools and 7800 high school students
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
From a cost-benefit analysis it was determined that the Minimum Detectable Effect required for maintaining positive returns over the change in the high school scholarship recipient ranged from 1.8 to 2.8; thus, we are aiming for a MDE of 2-3 percentage points. For MDE calculations we are assuming (1) a yearly discount rate of 12%, (2) as income base the 2012 national minimum wage according to the statistics national institute, (3) high school level educational yearly returns of 5 and 8%, and (4) a 57 years worklife (assuming that the youth starts labor activities at 18 and retires at 65). The power calculations are based on the dropout rate indicator, as this is our main outcome. The prevalence for dropout rates cursing 2nd and 3rd level of high school in urban areas –where the majority of the pilot eligible schools are concentrated- is 35.10% and 30.82%, respectively. Power estimations suggest a school sample of 610 for female strata and 625 for male strata in 3rd grade of high school (this is the most demanding grade in terms of sample size) in order to get a MDE of 0.03. Based on costs, operation capacity and execution schedule, we selected a sample of 625 treatment and 625 control schools. With this double stratification, disaggregated analysis over the course level and gender can be conducted. The parameters for these calculations are the following: power level (1-β)=80%, confidence level (1-α)=95%, intracluster correlation (ICC=0.133), dropout rate (32.51).
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
Hospital Infantil de Mexico
IRB Approval Date
2015-10-09
IRB Approval Number
DG/1000/870/2015
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
No
Is data collection complete?
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports and Papers
Preliminary Reports
Relevant Papers