Experimental Design
The project will be evaluated using the RCT methodology. The evaluation will take advantage of the expansion of Pratham’s activities within the state of Assam, in India. The sample population will be randomly assigned to treatment groups (villages that will receive a particular program), and a comparison group (villages that will not receive any new program).
For this project, the analysis will be based on two main rounds of data collection. Baseline data will be collected from a sample of students and their respective households before the program is initiated (more details below). Endline data will be collected roughly 18 months from the baseline. The data will allows us to measure the causal effect of the intervention by comparing outcomes (e.g. learning achievements) in the treatment and control groups after the project has been running for some time.
More specifically, for the purpose of this project we will identify 200 primary public schools within the state of Assam that are eligible for the program and where Pratham had never been operating before. We will then randomly assign each one of them to one of four groups of equal size:
• Group 1 (Control): these villages/school will not receive any new program, and schooling will continue as usual;
• Group 2 (Learning Camp): in these villages/school Pratham will organize Learning Camps;
• Group 3 (Library Activity Groups): in these villages/school Pratham will organize Library Activity Groups;
• Group 4 (Learning Camp + Library Activity Groups): in these villages/school Pratham will organize both Learning Camp and Library Activity Groups;
The primary outcomes of interest will be learning outcomes (test scores) in figure recognition, mathematics, and language of students enrolled in these primary public schools. Before the beginning of the programs, we will collect detailed baseline data from all 200 schools. We will rely on three key survey tools: children survey, primary caretaker survey, and school survey.
The interventions will start in the treatment villages (groups 2, 3, and 4) after baseline data collection. The Library Activity Groups in groups 3 and 4 will begin immediately after data collection and will keep going for the full project time. The Learning Camps are instead typically administered in three rounds of 10 days each, distributed over a teaching term (about 5 months). Each school in the Learning Camps groups (i.e. groups 2 and 4) will receive the three sessions, although they will be administered in two different phases. A first set of schools will host the camps within 5 months from baseline, while the other set will receive it after that, i.e. roughly five months later. We will randomly select the schools that will receive the program first, stratifying by group.
Endline data collection will take place roughly 18 months after baseline. Given that the Learning Camps only run during one school term, the endline survey will take place at least 5 months after the end of the program in the schools. Endline data collection will involve the same children, families, and school surveyed at baseline and will mirror the baseline data collection in its structures.
2020 update:
The data collection, covering 5,300 children and 4,200 caretakers, was completed in January 2020, right before the COVID-19 outbreak. We then plan to conduct a new follow-up survey towards the end of 2020 to study three novel questions. First, we want to study how children’s learning outcomes and psychological wellbeing have been affected by the pandemic. Second, we want to understand whether the study interventions mitigated some of the negative consequences. Third, we want to understand how families and communities have coped with respect to their children’s education when schools were closed.
Our follow-up study has two primary components: the first is a phone-survey conducted with the caregivers of the children, and the second is an in-person survey and assessment of children. Since we plan to track the same sample - we will first survey Principals/Head teachers of the study schools to update the contact details of the caregivers and to determine the status of school re-opening, teaching-learning activities conducted during the lockdown period, and support provided/received since the COVID-19 outbreak. After collecting/updating all contact details, we will then administer the phone-survey with caregivers, to collect data on educational/learning support received by the household and child during the lockdown, caregiver support to the child, parental perceptions on schools reopening, aspirations for their children, etc. Finally, we will administer the survey and assessment in-person to children when it is possible, since the learning test and the psychological wellbeing module cannot be reliably conducted over the phone.