Teach For Science: An Impact Evaluation of Science Education Initiative’s Fellows Program
Last registered on May 14, 2018

Pre-Trial

Trial Information
General Information
Title
Teach For Science: An Impact Evaluation of Science Education Initiative’s Fellows Program
RCT ID
AEARCTR-0002386
Initial registration date
May 10, 2018
Last updated
May 14, 2018 10:50 AM EDT
Location(s)

This section is unavailable to the public. Use the button below to request access to this information.

Request Information
Primary Investigator
Affiliation
University of Virginia
Other Primary Investigator(s)
PI Affiliation
New York University
Additional Trial Information
Status
On going
Start date
2017-06-01
End date
2018-09-30
Secondary IDs
Abstract
We propose to conduct a randomized evaluation of Science Education Initiative’s fellowship program in Pune, in the Indian state of Maharashtra, in 46 public and charter schools. Science Education Initiative seeks to improve STEM learning in public schools by recruiting highly motivated college students who are placed in schools to work with regular teachers as teaching assistants. The fellows work alongside teachers using scripted lesson plans to deliver potentially better instruction to students. The proposed RCT will examine the effectiveness of the SEI fellow program on student STEM learning outcomes, as well as gender achievement gaps in STEM. Detailed information on management, classroom practices, teacher surveys, and student surveys will provide insights into any potential mechanisms.

External Link(s)
Registration Citation
Citation
Ganimian, Alejandro and Isaac Mbiti. 2018. "Teach For Science: An Impact Evaluation of Science Education Initiative’s Fellows Program." AEA RCT Registry. May 14. https://www.socialscienceregistry.org/trials/2386/history/29457
Experimental Details
Interventions
Intervention(s)
The intervention is a program run by Science Education Initiative (SEI), a non-profit organization based in Pune, in the Indian state of Maharashtra.

The “SEI Fellowship Program” (SEI-FP) attempts to deliver high-quality science and math instruction in schools serving low-income students, by:
i. Developing high-quality content and pedagogical tools aligned with global standards;
ii. Selecting highly-motivated college graduates in Science, Technology, Engineering, and Math (STEM) to serve as “shadow” science and math teachers in schools serving low-income students;
iii. Providing intensive training to fellows on pedagogy, content, and classroom management; and
iv. Monitoring fellow performance and provides continuous feedback to improve classroom instruction.

In 2016, SEI placed 36 fellows across 52 classrooms, and 35 fellows in the 2015 academic year. In the current 2017-18 academic year, SEI has placed 68 fellows in 110 classrooms across 63 PMC schools, reaching approximately 4,000 students
Intervention Start Date
2017-06-01
Intervention End Date
2018-05-05
Primary Outcomes
Primary Outcomes (end points)
Our study will focus on the following outcomes: student achievement (test scores) and classroom instruction. Specifically, we collect the following data:

i. Student assessments in math, science and language
ii. An assessment of teacher subject matter expertise in science and math, based on both domestic (e.g. Pratham teaching capability test, Teacher’s Understanding of Subject Knowledge or TUSK) and international models (e.g. Context Knowledge for Teaching or CKT);
iii. A survey of teachers, based on several international models (including the Teaching and Learning International Study or TALIS, the Program for International Student Assessment or PISA, the Program for International Reading Literacy Study or PIRLS and the Trends in International Mathematics and Science Study or TIMSS);
iv. A survey of students (based on PISA and the MET project’s “Tripod” survey).

Our focus will be not only to document the impact of the program on student achievement, but also to document changes in school management and classroom instruction practices.

We will also conduct one-to-two rounds of unannounced visits to schools and conduct classroom observations to document the program’s impact on: (a) teachers’ presence in school and the classroom; (b) the share of classroom time devoted to instruction, classroom management, and other tasks drawing on the Stallings Observation Protocol, which has been previously used in India and other developing countries) (Stallings et al., 2014; Sankar & Linden, 2014; World Bank, 2016); and (c) different classroom teaching practices (drawing on the Classroom Assessment Scoring System or CLASS).
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
We are undertaking a randomized evaluation of the SEI-FP in 46 public and Vidya Niketan (ViNi) schools under the Pune School Board (PSB). These schools were chosen in conjunction with SEI after considering a number of factors such as the language of instruction, the presence of other NGOs or teaching programs, and the remoteness of the school. Among our set of study schools, one SEI fellow per school was randomly assigned to either grade 5 or grade 6. This approach resulted in 23 grade 5 classrooms and 23 grade 6 classrooms in the treatment group and an equal number of control groups in both grades.
Experimental Design Details
Not available
Randomization Method
Randomization was done in office by a computer. This randomization was conditional on geographic and school timing constraints to minimize the chance that fellows were placed far from where they live or placed in schools that don’t match their availability.
Randomization Unit
The randomization was done at the classroom level in each of the 46 schools.
Was the treatment clustered?
Yes
Experiment Characteristics
Sample size: planned number of clusters
92 classrooms
Sample size: planned number of observations
There will be approximately 2,898 observations, comprising of: (i) an estimated 2,760 students (at an average of 30 students per classroom in 92 classrooms); (ii) 46 SEI fellows placed in the treatment classrooms; (iii) 92 public school teachers teaching in the treatment and control classrooms.
Sample size (or number of clusters) by treatment arms
A treatment group of 46 classrooms (23 grade 5 classrooms and 23 grade 6 classrooms) will receive an SEI teaching fellow. The treatment classes will have about 1380 students in total (about 30 per class)

A control group of 46 classrooms (23 grade 5 classrooms and 23 grade 6 classrooms) will not receive an SEI teaching fellow. The control classes will have about 1380 students in total (about 30 per class)
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
Institute for Financial Management and Research
IRB Approval Date
2017-05-25
IRB Approval Number
N/A
IRB Name
Institutional Review Board for the Social and Behavioral Sciences, University of Virginia
IRB Approval Date
2017-09-07
IRB Approval Number
2017039200