Evaluating the Design and Impact of the Secondary School Teacher Training Initiative under the Government of Nepal’s School Sector Development Program

Last registered on February 18, 2020

Pre-Trial

Trial Information

General Information

Title
Evaluating the Design and Impact of the Secondary School Teacher Training Initiative under the Government of Nepal’s School Sector Development Program
RCT ID
AEARCTR-0004338
Initial registration date
June 24, 2019

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
July 09, 2019, 12:51 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
February 18, 2020, 4:44 AM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
University of Minnesota

Other Primary Investigator(s)

PI Affiliation
Tufts University
PI Affiliation
Independent researcher

Additional Trial Information

Status
Completed
Start date
2017-08-17
End date
2019-12-31
Secondary IDs
PW3.10.NP.IE
Abstract
This RCT is designed to evaluate the impacts on the subject knowledge and pedagogical practices of teachers of 9th and 10th grade math and science, and on student learning, of Nepal's School Sector Development Program (SSDP) teacher training (TT). The main teacher training (TT) intervention requires all 9th and 10th grade teachers in government schools to attend government-run in-service teacher training modules that are intended to raise their subject knowledge and to motivate and equip them to use practical, demonstration-based teaching methods rather than more traditional teaching methods. The training includes a 10-day session at an Education Training Center (ETC), and then completion by participating teachers of the equivalent of five days of “self-study project work,” which includes independent lesson plan development and other classroom research and practice activities. The TT intervention in study schools differs slightly from the broader intervention to be rolled out throughout Nepal over the next several years in several respects. First, rather than waiting for teachers and schools to request trainings, the ETCs specifically invited teachers in treatment schools, and were asked not to invite teachers in control schools or other schools within the same small geographic areas (associated with the local Village Development Committees) as the control schools. Second, while in the broader roll-out priority will be given to inviting teachers with permanent positions who had not received training under the previous education plan (the School Sector Reform Program or SSRP), ETCs were requested to invite all teachers of grade 9 and 10 math and science, regardless of contract type or previous training experience. Third, while in principle the full SSDP training will include two modules, each including 10 days of training at an ETC and five days of self-study project work, in practice only the first module has so far been rolled out, and only this first module is included in this evaluation.
The supplementary Video Assignment (VA) treatment requires each trained teacher to submit (before he or she can receive full credit for the training) a video of himself or herself (during an entire class session) implementing one of the new lesson plans that he or she is expected to create as part of the self-study project. The aim of adding the VA is to increase teachers’ motivation for investing serious effort in the self-study project activities, which may be important for translating what teachers learn at the ETC into new and improved classroom practices. In what follows, “TT treatment” will refer to the provision of the basic teacher training intervention without the video assignment, while “TTVA treatment” will refer to the provision of the teacher training intervention with the video assignment.
The study involves 204 schools, of which 102 are randomly assigned to the treatment and 102 serves as controls. Of the 102 assigned to the treatment, 51 are randomly assigned to the standard TT program, while the other 51 are randomly assigned to the TT supplemented by the Video Assignment (VA) treatment.
External Link(s)

Registration Citation

Citation
Glewwe, Paul, Julie Schaffner and Uttam Sharma. 2020. "Evaluating the Design and Impact of the Secondary School Teacher Training Initiative under the Government of Nepal’s School Sector Development Program." AEA RCT Registry. February 18. https://doi.org/10.1257/rct.4338-2.0
Former Citation
Glewwe, Paul, Julie Schaffner and Uttam Sharma. 2020. "Evaluating the Design and Impact of the Secondary School Teacher Training Initiative under the Government of Nepal’s School Sector Development Program." AEA RCT Registry. February 18. https://www.socialscienceregistry.org/trials/4338/history/62935
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
The main teacher training (TT) intervention requires all 9th and 10th grade teachers in government schools to attend government-run in-service teacher training modules that are intended to raise their subject knowledge and to motivate and equip them to use practical, demonstration-based teaching methods rather than more traditional teaching methods. The training includes a 10-day session at an Education Training Center (ETC), and then completion by participating teachers of the equivalent of five days of “self-study project work,” which includes independent lesson plan development and other classroom research and practice activities. The TT intervention in study schools differs slightly from the broader intervention to be rolled out throughout Nepal over the next several years in several respects. First, rather than waiting for teachers and schools to request trainings, the ETCs specifically invited teachers in treatment schools, and were asked not to invite teachers in control schools or other schools within the same small geographic areas (associated with the local Village Development Committees) as the control schools. Second, while in the broader roll-out priority will be given to inviting teachers with permanent positions who had not received training under the previous education plan (the School Sector Reform Program or SSRP), ETCs were requested to invite all teachers of grade 9 and 10 math and science, regardless of contract type or previous training experience. Third, while in principle the full SSDP training will include two modules, each including 10 days of training at an ETC and five days of self-study project work, in practice only the first module has so far been rolled out, and only this first module is included in this evaluation.
The supplementary Video Assignment (VA) treatment requires each trained teacher to submit (before he or she can receive full credit for the training) a video of himself or herself (during an entire class session) implementing one of the new lesson plans that he or she is expected to create as part of the self-study project. The aim of adding the VA is to increase teachers’ motivation for investing serious effort in the self-study project activities, which may be important for translating what teachers learn at the ETC into new and improved classroom practices. In what follows, “TT treatment” will refer to the provision of the basic teacher training intervention without the video assignment, while “TTVA treatment” will refer to the provision of the teacher training intervention with the video assignment.
Intervention Start Date
2017-12-19
Intervention End Date
2018-07-16

Primary Outcomes

Primary Outcomes (end points)
Math and science test scores for students in grades 9 and 10. Also, a subset of items on these tests will be generated for those items for which the content was emphasized in the training.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
• School-level averages of teachers’ scores on subject knowledge assessments (implicit in the teacher evaluations of the student assessment items), estimated using IRT methods (with raw scores as robustness check), separately for math and science.
• Teacher attendance measures:
o Teacher attendance on the first day of the school visit, and over the previous several days, as recorded in school logbooks
o Teacher attendance on the first day of visit, as recorded in the first question on the teacher questionnaire
o Median of students’ report regarding how frequently the teacher is absent (5-point scale)
• Student attendance:
o Student attendance rate on the most recent full day of classes before today, as reported by head teacher, separately for grade 9 and grade 10 students
• From Head Teacher reports on individual teachers:
o The teacher’s command over math or science subject matter (5-point scale)
o The teacher’s interest is in learning ways to teach more effectively (4-point scale)
o Whether the teacher has ever created from local resources any teaching materials – such as models of 3-dimensional shapes, atoms or the solar system – for use in a grade 9 or 10 math or science course
o Frequency of teacher’s use of teaching materials or other visual aids (other than the chalk board) to help explain concepts (4-point scale)
o Whether teacher has ever collected information from local residents outside the school (such as prices in local markets or interest rates offered by local bankers), or required his students to collect such information
o Frequency of teacher requiring students to work together in small groups (4-point scale)
o Frequency of teacher requiring students to work on longer-term project, for which they must gather information and make practical application (4-point scale
• From teacher questionnaires:
o Self report of minutes spent preparing per class during the most recent full week of classes (calculated from reports of total number of classes and total minutes spent preparing, whether during other class periods or outside of class time).
o Self report of whether teacher uses a written lesson plan as a primary guide while conducting a class
o Self report of how often teacher has students work together in small groups (5-point scale)
o Self report of how often teacher uses examples or homework problems involving local information (7-point scale)
o Opinion regarding importance of using examples involving local information (3-point scale)
o Self report of how often teacher has required students to collect local information, whether by interviewing family or community members, observing family or community activities, or taking measurements (for example, of weather conditions)? (7-point scale)
• From student questionnaire, median responses, separately for math and science teachers
o How often teacher gives homework (5-point scale)
o How often teacher checks student’s homework (5-point scale)
o How often teacher returns student’s homework with corrections (5-point scale)
o How often teacher uses class time for asking questions of any students or holding discussions or interactions about math/science concepts with any students (5-point scale)
o How often teacher uses class time to ask YOU (the student) a question or engaging YOU in a discussion of interaction about math/science concepts (5-point scale)
o How often teacher requires the student to work together in small groups (5-point scale)
o How often teacher uses demonstrations involving physical objects made from local materials or other visual aids to help students understand math/science concepts (5-point scale)
o How often teacher uses demonstrations involving diagrams, pictures or information from the internet (5-point scale)
o How often teacher gives quizzes to students (5-point scale)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The study collects data from 204 schools. The primary randomization divides the 204 study schools into two groups of equal size: 1. 102 “Phase I” schools, which were to receive the SSDP teacher training in late 2017; and 2. 102 “Phase II” schools, which were to receive the SSDP teacher training only after May of 2019, and which serve as the control group during the period of study. To minimize the potential for spillover effects of training on Phase II study schools, other schools in the same small geographic areas that contain the Phase II schools will also receive the relevant training only after May of 2019. The secondary randomization divides the 102 Phase I schools into two groups: 1. 51 TTVA schools, in which each teacher must submit a video of himself or herself implementing in his or her classroom a new lesson plan developed as part of the SSDP training to receive full credit for the training; and 2. 51 TT schools, in which no video is required to receive full credit for the training.

To achieve a sample representative of most of Nepal, while containing costs, the research team chose to stratify the sample first by district. To reduce data collection costs further, the team (in consultation with its partners in the Nepalese government) eliminated from consideration 10 of the most remote or otherwise difficult districts. From the remaining 65 districts (which contain 94.3% of Nepal’s schools), the team randomly selected a representative set of 16 districts, and then sampled schools only within those districts.

At the request of the team’s government partners, and to increase the study’s power to identify impact on the teachers for whom the TT training was most likely to have impact, the team sorted schools within districts into two strata: “priority” and “non-priority” schools, and over-sampled the former. Schools were identified as priority schools if there was no evidence (in hard copy records made available by the National Center for Educational Development) of any permanent teacher, or any teacher for which contract type was unknown, having completed all three modules of the SSRP training. The specifics of this rule were dictated largely by the idiosyncrasies of the only existing records identifying which schools and teachers had received SSRP training. The team chose to select two-thirds of the sample within a district from the priority stratum, while the other one-third would come from the non-priority stratum.
Experimental Design Details
Randomization Method
Schools were randomized, after stratification, using random numbers drawn by the research team.
Randomization Unit
School
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
204 schools
Sample size: planned number of observations
8,000 students, and about 400 teachers and 204 head teachers.
Sample size (or number of clusters) by treatment arms
102 control schools, 51 schools with TT treatment, and 50 schools with VA treatment.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
For a power level of 0.8, at the 95% significance level we should be able to detect an increase in test scores of 7 percentage points, which is about 0.3 standard deviations of the distribution of test scores.
IRB

Institutional Review Boards (IRBs)

IRB Name
University of Minnesota Institutional Review Board
IRB Approval Date
2017-07-28
IRB Approval Number
STUDY00000915
IRB Name
Tufts University Social, Behavioral and Educational Research Institutional Review Board
IRB Approval Date
2017-08-03
IRB Approval Number
1707008
Analysis Plan

Analysis Plan Documents

Evaluating the Design and Impact of the Secondary School Teacher Training Initiative under the Government of Nepal’s School Sector Development Program

MD5: 547ea7629de2056f523b45760ffd22b7

SHA1: f96cca2a772846807d0011495238675152283bc8

Uploaded At: June 21, 2019

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
September 30, 2018, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
March 31, 2019, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
203 schools
Was attrition correlated with treatment status?
Yes
Final Sample Size: Total Number of Observations
12,632 students for full endline sample, 9,895 for panel sample (present at baseline and at endline)
Final Sample Size (or Number of Clusters) by Treatment Arms
51 schools with teacher training (TT) only, 51 schools with teacher training and video assignment (TTVA), and 101 control schools
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials