Strengthening Teacher Accountability to Reach All Students (STARS)

Last registered on March 08, 2019

Pre-Trial

Trial Information

General Information

Title
Strengthening Teacher Accountability to Reach All Students (STARS)
RCT ID
AEARCTR-0003977
Initial registration date
March 07, 2019

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
March 08, 2019, 3:58 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
University of Delaware

Other Primary Investigator(s)

PI Affiliation
University of Massachusetts Boston
PI Affiliation
University of Delaware

Additional Trial Information

Status
On going
Start date
2018-05-16
End date
2019-07-05
Secondary IDs
Abstract

Public programs in developing countries, even when proven successful at smaller scales, tend to suffer from implementation challenges when scaled up and may not deliver the intended benefits. For instance, low accountability of public officials like teachers leads to high absenteeism; thus, teacher training programs that may be effective when delivered with intense monitoring for a small number of schools, cannot be implemented at a larger scale if teachers are frequently out-of-school. This project evaluates a large, government-led scale-up of an effective pedagogical technique, targeted instruction, to increase learning among primary school students in Ghana. Additionally, this project investigates how existing government mid-level managers can be leveraged to facilitate the take-up and delivery of the program and enhance effectiveness of government scale-up efforts.

We use a randomized controlled trial to evaluate the impact of the targeted instruction (TI) program at scale, and the incremental impact of combining it with the management training program. The targeted instruction intervention provides training and materials to treatment schools for teaching students in grades 4 through 6 by their learning levels instead of their grade level. The management training program offers additional training to school managers on managerial best practices. Our trial answers the following questions: how effective and cost effective is a large-scale targeted instruction program? Can a government-led management training improve managerial practices and enhance the take-up and impact of this program? The study will be conducted across 210 schools in 20 districts of Ghana.
External Link(s)

Registration Citation

Citation
Beg, Sabrin, Anne Fitzpatrick and Adrienne Lucas. 2019. "Strengthening Teacher Accountability to Reach All Students (STARS) ." AEA RCT Registry. March 08. https://doi.org/10.1257/rct.3977-1.0
Former Citation
Beg, Sabrin, Anne Fitzpatrick and Adrienne Lucas. 2019. "Strengthening Teacher Accountability to Reach All Students (STARS) ." AEA RCT Registry. March 08. https://www.socialscienceregistry.org/trials/3977/history/42844
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
Targeting instruction pedagogy provides teachers with the tools and resources to teach at the learning level of the student, instead of the student’s grade. In this project, it is implemented in grades 4 through 6 during the 2018-2019 academic year.


To encourage fidelity of implementation of targeted instruction, an additional treatment arm provides additional training to school managers in addition to the targeted instruction program---in this treatment circuit supervisors and head teachers participated in a management training. This training focused on school management and encouraging and supporting a reflective teaching practice. Participants were also given a copy of a management training manual with sections on how to support teachers on implementing targeted instruction.
Intervention Start Date
2018-05-16
Intervention End Date
2019-07-05

Primary Outcomes

Primary Outcomes (end points)
The primary outcome variables for students are:
1. Attendance
2. Standardized test score on the mathematics exam at follow-up
3. Standardized test score on the English exam at follow-up
4. Standardized combined mathematics and English test score at follow-up


The primary outcome variables for classrooms are:
1. A teacher present in the classroom
2. A teacher engaged with students
3. Whether teaching and learning materials are visible and/or being used
4. Whether at least one student had an opportunity to express an idea
5. Whether students were divided by level or grade
Additionally, we will construct an index based on the first four measures as a measure of overall teaching effectiveness.


The outcome variables for teachers are:
1. Present at school



The primary outcome variables for head teachers and the school level are:
1. Head teacher attendance
2. Teacher reported observation frequency
3. Teacher reported observation usefulness
4. Management Practices Index (see below)



The primary outcome variables at the CS-level are:
1. Teacher reported number of visits
2. Teacher reported presence of feedback
3. Teacher reported usefulness of feedback
4. HT reported number of visits
5. HT reported number of meetings
6. Self-reported number of meetings with HTs in their circuit
7. Self-reported number of visits to study schools
8. Self-reported number of classroom observations conducted by the CS
Further we will combine all 8 in an index.
Primary Outcomes (explanation)
Subject-specific test scores will be constructed using item response theory (IRT) and standardized based on the baseline mean and standard deviation. We will also test the effect of the intervention on the combined test score. Other outcomes will be obtained from relevant questions in the surveys.

Variables used to construct Head Teacher Management Practices Index:

Using scores on tests and/or assessments to inform promotion

Whether school follows a written curriculum

Whether teachers use lesson plans for teaching

Frequency of classroom observations

Frequency of giving suggestions for how teachers can improve teaching

Usefulness of feedback to teachers

Frequency of how often HT takes initiative to discuss matters in classroom

Whether rewards are given to teachers for good performance

Number of all-staff meetings and trainings

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
This randomized controlled trial is in the 20 districts in Ghana that UNICEF supports. This study will operate in 140 circuits (groups of schools) within these districts. We study 210 in these 140 circuits, that are randomly divided into three groups:

Treatment 1 (Control)—P4-P6 teachers and head teachers continue as usual, receiving no additional training or resources. (70 schools)
Treatment 2—P4-P6 teachers, head teachers, and circuit supervisors all receive targeted instruction training. Schools receive targeted instruction teaching and learning materials. Head teachers and circuit supervisors do not receive enhanced management training. (70 schools)
Treatment 3—P4-P6 teachers, head teachers, and circuit supervisors receive targeted instruction training. Schools receive targeted instruction teaching and learning materials. Head teachers and circuit supervisors also receive enhanced management training. (70 schools)
Experimental Design Details
This randomized controlled trial is in the 20 districts in Ghana that UNICEF supports. This study will operate in 140 circuits (groups of schools) within these districts. The STARS study has two layers of randomization: circuit-level and school-level, resulting in three study arms (two treatment groups and one control group). Specifically, we first randomized each circuit to either receive the management training, or not. Then within each management circuit, we randomly selected one school to receive targeted instruction and enhanced management training. Within each non-management circuit, we randomly selected two schools—one received targeted instruction training and the other did not.

This scheme results in three treatment arms.
Treatment 1—P4-P6 teachers and head teachers continue as usual, receiving no additional training or resources. Due to program design, CSs in this arm will still receive targeted instruction training.
Treatment 2—P4-P6 teachers, head teachers, and circuit supervisors all receive targeted instruction training. Schools receive targeted instruction teaching and learning materials. Head teachers and circuit supervisors do not receive enhanced management training. (70 schools)
Treatment 3—P4-P6 teachers, head teachers, and circuit supervisors receive targeted instruction training. Schools receive targeted instruction teaching and learning materials. Head teachers and circuit supervisors also receive enhanced management training. (70 schools)
This study design is similar to a fully cross-randomized design, although we do not have any schools that receives just the management training intervention (no targeted instruction training).
Randomization Method
Randomization done in office by a computer
Randomization Unit
Circuit level randomization for management training intervention
School level randomization for targeted instruction intervention in circuits not selected for management training.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
140 circuits, 210 schools
Sample size: planned number of observations
140 Circuit supervisors, 210 Schools, 210 Head teachers, 600 Teachers, 6300 Students.
Sample size (or number of clusters) by treatment arms
70 schools in control group, 70 schools with targeted instruction treatment, 70 schools with targeted instruction + management training treatment.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
For achievement tests: With 70 schools per arm, 2 grades per school, and 15 students per grade, we will have approximately 6300 students overall in the study at baseline. Based on data from Duflo et al. (2018) we assume that 69% will respond at follow up (N= 4347). Assuming a power of 80%, a standard deviation of standardized test scores of 1, and an intra-cluster correlation of 0.17 (0.3 in English), and a 0.44 R-squared increase with the inclusion of baseline scores (0.5 for English), we are powered to detect an effect size of 0.162 for math and 0.194 for English (effect sizes are in standard deviations) comparing any two arms. For teacher attendance: With 70 schools per arm and 3 teachers per school, we will have approximately 630 teachers. Assuming that 75% will respond at follow up, total number of teachers is 473. Assuming a power of 80%, a standard deviation of teacher attendance of 1, and an intra-cluster correlation of 0.11, and a 0.15 R-squared increase with the inclusion of baseline covariates, our minimum detectable effect is 0.31 across any two arms. For other compliance outcomes (teacher in the classroom; teacher engaged with students), the minimum detectable effect is approximately the same.
IRB

Institutional Review Boards (IRBs)

IRB Name
Human Subjects Committee for Innovations for Poverty Action IRB-USA
IRB Approval Date
2018-05-11
IRB Approval Number
5678
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials