Re-imagining Education: Do Science Experiments Deliver Quality Education?

The intervention is designed for students in lower-middle grades (grades 6th, 7th and 8th) in public schools where pedagogy follows a traditional approach-where the teacher instructs and students follow. In contrast, this intervention is based on a child-centric, guided discovery/inquiry-based pedagogical style where students are guided to discover their own learning experiences - through several project-based tasks and science experiments. In particular, instructors start by presenting a simple science problem, and through inquisition and hands-on experience with practical experimentation, students are guided to discover the solution to the presented problem. In any session, the student engagement exceeds the uni-direction instruction time, where for at least 75% of the time, students lead the discussions.

Through this intervention, we aim to evaluate whether an alternative learning method (described above; more details follow) impacts students' content-based learning, confidence, curiosity, and creativity. All of these are important inputs for developing critical thinking. While we expect some small effect on academic content-based learning (such as scores on science tests we conduct), we hypothesize that the intervention can have positive effects on students' self-confidence, curiosity, commitment and creativity. Such skills learned through science pedagogy can impact students' engagement with other subjects, and we aim to evaluate this intervention's spillover effect on their engagement in other subjects. For the direct and spillover effect, we consider various mechanisms such as parents' aspirations for their children, socio-economic factors, and biological factors driving our results. We collect this information in the baseline and the endline (to be conducted in 2023).

2022-08-18

2023-03-14

Our primary outcomes include: Self-confidence; creativity, curiosity; Commitment

All outcomes are constructed using the survey questions.

Content-based learning; Aspiration & Barriers; Time spent on Education; Self-Reported Attendance

All outcomes are constructed using the survey questions.

There are two types of interventions in the study, which are similar with respect to the pedagogical approach but differ in terms of resources employed. The intervention provides a hands-on experience to students where they are fully involved in the experiments conducted. The intervention is carried out between August'22 to March'23.

Treatment 1 -- Mobile Science Lab (MSL):
Under this intervention, a mobile van with science experiment kits visits treatment schools with three foundation-hired and trained instructors. The instructor gives a 4-hour session per month to each classroom (grades 6 to 8).

Treatment 2 -- Lab on Bike (LoB): Under this intervention, a motorbike with science experiment kits visits treatment schools with one foundation-hired and trained instructor. The instructor gives a 2-hour session per month to each classroom (grades 6 to 8).

Control: Schools assigned to the control group do not receive the intervention and continue with traditional textbook-based teaching.

Randomization is done in the office by a computer. For randomization, we asked for a big list of schools from Agastya after meeting all their requirements, from which we could randomize schools into treatment and control groups. Agastya provided us with 151 schools in the 4 districts (Ghazipur, Gonda, Lucknow, and Varanasi) of Uttar Pradesh. The list of schools was created based on donors' preferences for geographic location and Agastya's operational requirements - like schools closer to their base location and optimal utilization of resources. Essentially, it meant: (i) ensuring that each geographic block should have treatment schools (ii) catering to a maximum number of students while keeping a reasonable class size: 30-40 students per instructor in each session. In treatment 1, the minimum strength in each school had to be 120-160, while the total strength in a given district should not exceed 2000-2200. In treatment 2, the combined minimum strength in each school had to be 60-80, while the total strength in a given district should not exceed 1000-1200. Agastya pegged the strength based on previous years' enrollment numbers. Out of 151 schools, we dropped 14 schools - first removing 8 outliers (5 very big schools, 1 with low strength, 2 schools with English as the medium of instruction), and finally, 6 randomly selected schools to keep the total treated students within the operational limit of Agastya. We were left with a list of 137 schools before randomization.

The randomization was done using the Unified District Information System for Education Plus (UDISE and UDISE+) data. UDISE is one of the largest Management Information Systems initiated by the Department of School Education and Literacy, Ministry of Education, Government of India, covering more than 14.89 lacs of schools, 95 lacs of teachers, and 26.5 crores of children. UDISE+ is an updated version of UDISE that was in use from 2005-2017.

Using the unique ID of schools (11-digit UDISE code) we merged the school list with the 2017 and 2020 UDISE+ data on school/teacher characteristics such as the caste composition of the school, enrollment in schools by class, the facilities (such as number of classrooms, the condition of classrooms, available textbooks and uniform, concrete structure of schools, playground, etc.) available at the school, teachers characteristics (such as training, gender, number of teachers) at the school and randomized the sample into the two treatment groups, and a control group.

The unit of randomization is school within each block.


The treatment arms were district-specific: Ghazipur and Gonda formed Mobile Science Lab (treatment 1); whereas Lucknow and Varanasi formed Lab on Bike (treatment 2). Within each district, geographic blocks formed the strata within which we randomized the schools across treatment and control groups.

Number of blocks (or strata) in each district are as follows: Ghazipur (3- Sadar, Manihari and Zakhania) Gonda (2- Jhanjhari and Mujhena); Lucknow (2- Chinhat and Mohanlal Ganj); Varanasi (5- Chiraigaon, Harhua 1, Harhua 2, Sewapuri and Kashi Vidyapeeth).

Yes

There are 132 schools/clusters.

12000-15000 students (in the treated and control schools)

We have 27 treatment 1, 41 treatment 2, 64 control schools

The distribution of schools across 4 districts and treatment is as follows: Ghazipur 30 (13 treatment, 17 control), Gonda 28 (14 treatment and 14 control), Varanasi 53 (28 treatment and 25 control), and Lucknow 26 (15 treatment and 11 control).