Back to History Current Version

Do Students Benefit from Blended Instruction? Experimental Evidence from India

Last registered on December 07, 2019

Pre-Trial

Trial Information

General Information

Title
Do Students Benefit from Blended Instruction? Experimental Evidence from India
RCT ID
AEARCTR-0003665
Initial registration date
December 12, 2018

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
December 13, 2018, 12:43 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
December 07, 2019, 9:44 AM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
University of California, Irvine

Other Primary Investigator(s)

Additional Trial Information

Status
On going
Start date
2018-12-14
End date
2021-02-28
Secondary IDs
Abstract
This experimental study investigates the causal effect of a teacher capacity building program that promotes blended instruction, on student learning. It will be implemented in government schools in Haryana, India, in collaboration with a large, local NGO ("Avanti Fellows"). The program's objective is to positively affect the instruction of mathematics and science, in grades nine and ten. The study hypothesizes that student learning improves if teachers are given resources and training, to enrich their instruction with video-based learning materials. Secondly, the study hypothesizes that the intervention's cost-effectiveness outperforms that of an alternative model of teacher capacity building, which does not rely on infrastructure upgrades and uses printed workbooks only.
External Link(s)

Registration Citation

Citation
de Barros, Andreas. 2019. "Do Students Benefit from Blended Instruction? Experimental Evidence from India." AEA RCT Registry. December 07. https://doi.org/10.1257/rct.3665-2.1
Former Citation
de Barros, Andreas. 2019. "Do Students Benefit from Blended Instruction? Experimental Evidence from India." AEA RCT Registry. December 07. https://www.socialscienceregistry.org/trials/3665/history/58382
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
240 Government Senior Secondary Schools (GSSS) across 8 districts of Haryana will be equally randomized across two treatment arms and the control group, with 80 schools assigned to each group.

The "ICT group" will receive the full intervention. This includes setting up smart classrooms, provision of digital content to supplement teaching instruction, printed workbooks for practice of students, and capacity building of mathematics and science teachers responsible for teaching class ninth and tenth curriculum.

The "Workbook group"'s program components are equivalent to those administered in the previous group; however, the group does not receive those particular program components related to ICT (i.e., ICT-related infrastructure upgrades or digital content).

The "Control group" continues with "business-as-usual". The schools assigned to the control group will neither receive facilities nor materials. Their teachers will also not undergo the program's teacher training activities.
Intervention Start Date
2018-12-17
Intervention End Date
2021-01-31

Primary Outcomes

Primary Outcomes (end points)
The study's main program outcome of interest is student learning, in math and science. Students in grades nine and ten will be assessed at baseline and through an end-of-year test. Assessments will be administered as paper-based tests (one per grade and subject), under the same, strict governmental oversight as other central exams (with additional monitoring from the research team). Test items are tightly mapped to the official "CBSE/NCERT" school curriculum, but also include items from up to two years below grade-level. Items have been administered in similar contexts previously, in large-scale assessments. From these previous administrations, item response theory (IRT)-based item characteristics are used to maximize each assessments' test information.
Primary Outcomes (explanation)
Estimates of student ability will be calculated using a standard, three-parameter logistic (3PL) IRT model, with a single guessing parameter. In case a 3PL model does not converge, a 2PL model will be used instead. In doing so, anchor items across grades and test-occasions will allow for the (concurrent) linking of estimates onto one common, continuous ability scale per subject.

Secondary Outcomes

Secondary Outcomes (end points)
1) Student performance on subskills
2) Teaching behaviors and instructional quality
3) Intervention monitoring
Secondary Outcomes (explanation)
1) Student performance on subskills
For mathematics, items are categorized as measures of either one of the following four abilities: Algebra; geometry; number system; and data/statistics. For science, items are categorized as measures of either biology, chemistry, or physics. In summary, this subskill analysis will establish the extent to which the program affected a student's probability of mastering each of these subskills.

In the case of subskills, because of the lower number of items per sub-skill, student performance will not be scored with a continuous measure of ability. Instead, I will distinguish between three categories of mastery, for each of the two subjects: students who have mastered grade-level appropriate material; students who have only mastered material from one and two years below their grade-level; and other students. I will determine students' level of mastery empirically, through a Cognitive Diagnostic Model (CDM).

2) Teaching behaviors and instructional quality
The program's effects on teaching behaviors and on instructional quality will be assessed through two instruments: Classroom observations and student reports. First, the program includes bi-weekly school visits and monthly classroom observations. All visits and observations will be conducted in intervention and control schools. For this purpose, the study developed an instrument to measure the program's effect on the quality of instruction a student receives. Secondly, during school visits, a subset of students will be surveyed on common classroom behaviors. The study explicitly defines either of these data sources as a measure of mechanisms, i.e. not as a main outcome.

Romero et al. employ a Lasso procedure to identify potential mediators (from a large number of variables). I will follow the authors' approach (ibid.) to identify mediating instructional behaviors; yet, the Lasso procedure will be forced to include teachers' use of ICT materials as a predictor. In turn, concerning instructional quality, I pre-specify six dimensions that will be investigated as potential mediators (monitoring of student learning; feedback; maximization of learning time; density of the mathematics / science; clarity of content and lack of errors; richness of the mathematics / science).

3) Intervention monitoring
Sign-in sheets will be used to track teachers' exposure to capacity-building activities. Avanti Fellows will moreover provide data from its software backend, to track teachers' use of videos and digital learning materials. Monitoring data will also be collected in the above-mentioned bi-weekly and monthly visits, through a structured school questionnaire.

Experimental Design

Experimental Design
Schools are assigned with an equal split across the three groups (for a total of 80 schools per group). To achieve similar control and treatment groups and to improve statistical power, randomization is stratified. Within districts, schools are sorted into randomization strata of three, grouping schools with similar performance on the Haryana Board Exams together.
Experimental Design Details
To rank schools, I use the average school-level Board Exam score for 2018, overall (covering English, Hindi, Mathematics, Sanskrit, Science, and Social Studies). More precisely, I calculate a school's weighted average, using information on the number of student in each of six performance ranges, on the test. If the number of schools for any district ranking is not divisible by three, I randomly assign the remaining schools. In this case, I assign these remaining schools "globally", i.e. across districts. As more than three schools need to be randomized across districts, I once more group schools with similar Board Exam scores together.

Finally, I repeat the above randomization strategy ten times, selecting the randomization with the smallest t-statistic, from comparisons across the treatment and control groups, for a set of select covariates (using Lasso, I select a set of covariates from India's District Information System for Education (DISE), which are predictive of school-level average pass rates for standardized exams, in grades 7 and 8. The selected covariates are: Number of students in grade 7 and 8; percentage female (students); percentage minority (students); percentage OBC (students); number of teachers; percentage female (teachers); percentage graduates (teachers); years in service (school); co-ed (school); school requires minor repairs; wall missing or damaged; ratio computers/students. and the average school-level Board Exam score for 2018.
Randomization Method
Randomization done in office by a computer
Randomization Unit
School
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
240 schools
Sample size: planned number of observations
29,280 students in the first cohort, and an additional 12,240 students in the following cohort
Sample size (or number of clusters) by treatment arms
80 schools in each of the two treatment groups; 80 schools in the control group
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Intent-to-treat effect of 0.158 standard deviations (SD) on student test scores
IRB

Institutional Review Boards (IRBs)

IRB Name
IFMR Human Subjects Committee
IRB Approval Date
2018-06-25
IRB Approval Number
IRB00007107; FWA00014616; IORG0005894
IRB Name
Harvard University Committee on the Use of Human Subjects
IRB Approval Date
2018-08-15
IRB Approval Number
IRB18-0996
Analysis Plan

Analysis Plan Documents

Revised Pre-analysis Plan 2019-11-26 (detailed table shells and mock figures)

MD5: cfc35690169f42c5eb9192e5f6b69336

SHA1: 54cd16408d4ad95c11a7c16b1a1c604175ff1a24

Uploaded At: December 07, 2019

Pre-analysis Plan 2018-12-12

MD5: f54c8657c026278c77b1b43e405e049d

SHA1: d807bd42c4810614e0909e364ea1204533b23287

Uploaded At: December 12, 2018

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials