Mechanisms to Improve the Effectiveness of an Online Teacher Professional Development Program for Early Literacy

Last registered on November 01, 2023

Pre-Trial

Trial Information

General Information

Title
Mechanisms to Improve the Effectiveness of an Online Teacher Professional Development Program for Early Literacy
RCT ID
AEARCTR-0012353
Initial registration date
October 24, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
November 01, 2023, 2:51 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Affiliation
The World Bank

Other Primary Investigator(s)

PI Affiliation
Georgetown University
PI Affiliation
Center for Global Development
PI Affiliation
ESEN, The World Bank
PI Affiliation
The World Bank

Additional Trial Information

Status
On going
Start date
2023-01-02
End date
2024-12-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Learning recovery and acceleration necessitates improving the quality of teaching that students receive. Research has consistently shown that no other aspect of schooling is as important to students’ academic achievement as the quality of the teaching that they receive (Hanushek, 2011; Darling-Hammond, 2000; Araujo et.al., 2017). In fact, the value of teachers goes far beyond the effects on measured cognitive learning: research shows that teachers are essential to the socioemotional well-being and outcomes of students, often well beyond their schooling years (World Bank 2020). Despite its importance, many education systems today do not provide teachers with high-quality teacher professional development (TPD) opportunities to improve and strengthen their teaching practice. For example, an analysis of 139 TPD programs in low- and middle-income countries found that much of the professional development that teachers receive does not align with best practice of what we know works to help teachers improve their teaching practice (World Bank, 2021).

This study aims to evaluate different components of a TPD program focused on providing first grade teachers with high-quality professional development that is tailored, focused, practical and ongoing, and that will help them improve their teaching practice. Specifically, in this project we will explore the most effective way to deliver online training on teaching practices in El Salvador, a context of increasing inclusion of technology and remote learning.

To this end, we are conducting a large-scale RCT with 1,145 schools to evaluate the impact of a virtual group-based training on early learning instruction. Moreover, we aim to understand the complementarities between this virtual group-based training and other components of a TPD program. Specifically, we are evaluating three additional components: (a) one-to-one instructional coaching that aims to provide guidance to teachers on how to implement the teaching practices learned in the virtual training, (b) encouraging nudges to motivate teachers to use the tools that they learned during the virtual training, and (c) self-reflection resources and exercises component, which is a less expensive alternative to component (a) that might be more feasible in a context of lack of professional coaches.

We will measure the effects of the different combinations of TPD components on first grade teachers teaching practices (using the adapted TEACH ECE observation tool), teachers’ school attendance, and first grade students’ reading and writing skills. Moreover, we will also explore other secondary outcomes such as teachers’ wellbeing (mental health) and potential mechanisms that can explain these results, such as teachers’ motivation, experience, self-efficacy, empathy, and support from leadership.
External Link(s)

Registration Citation

Citation
Bendini, Magdalena et al. 2023. "Mechanisms to Improve the Effectiveness of an Online Teacher Professional Development Program for Early Literacy." AEA RCT Registry. November 01. https://doi.org/10.1257/rct.12353-1.0
Sponsors & Partners

Partner

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
We will evaluate four interventions:
1. Virtual group-based training on reading and writing instruction. This 10-weeks evidence-based training program aims to cover topics related to phonological awareness; knowledge of the alphabet and alphabetic principles; knowledge of vocabulary, decoding and fluency; oral and reading comprehension; lower and higher reading skills; as well as feedback and consolidation of acquired knowledge. The program is delivered over the course of 10 synchronous class sessions throughout the same number of weeks and asynchronous work during the week. These group trainings are delivered via the Google Classroom platform, by specialists trained by the World Bank team on early reading and writing instruction, coaching, and classroom observation tool. It is complemented with a package of specialized training materials, which include explanatory videos for each session, complementary training materials, assignments, and self-assessments. The entire program encourages teachers to reflect on their own practice as they incorporate new knowledge. In total, teachers will receive approximately 40 hours of group training.

2. One-to-one instructional coaching. This component consists of personalized coaching that aims to help teachers improve their teaching practice by providing them with highly individualized, tailored, practical and ongoing support from a pedagogical expert. This coaching is delivered individually to each teacher by a trained coach from the NGO FUSAL, a local non-governmental organization with extensive experience on education projects in El Salvador. Each coach was trained on a highly structured coaching methodology developed by the World Bank. The structure of the coaching used in our project is focused on developing teachers’ skills in 6 specific foundational teaching competencies: Establishing routines and norms; reinforcing routine and norms; giving clear directions; demonstrating and practicing; giving feedback; and promoting students’ critical thinking. During each coaching session, the coach is responsible for 1) conducting a classroom observation using the adapted tool based on Teach ECE (see more details below); 2) identifying a specific teaching skill that the teacher needs to improve on, based on the classroom observation; 3) delivering a structured coaching session that highlights things the teacher has done well, provides space to discuss and learn about the teaching skill, presents a short demonstration of the teaching skill in practice, and then includes time for the teacher to practice. Throughout the program, participating teachers will receive a total of 13 weekly coaching sessions, of which 4 are delivered in person and 9 are delivered virtually, by leveraging the Google Classroom platform.

3. Encouraging nudges. This component aims to motivate/encourage teachers to use and apply the concepts and practices they learned from the virtual group-based training (component i) in their classroom. These nudges will be delivered through 3 weekly WhatsApp messages over the course of 13 weeks for a total of 39 messages starting right after the conclusion of the group training (component i). These messages have been designed following principles from behavioral economics and aim to provide a combination of general information about important reading and writing skills that were taught during the training, actionable advice with specific examples of parent-child literacy activities, and encouragement or reinforcement.

4. Self-reflection resources and exercises. This component consists of a set of self-learning resources focused on helping participant teachers to reflect on their current teaching practices, learn about new teaching strategies, and implement the 6 specific foundational teaching competencies described above. These resources will be delivered through 3 weekly WhatsApp messages over the course of 13 weeks for a total of 39 messages (3 introductory messages and 36 messages with content) starting right after the conclusion of the group training (component 1). This component has been designed as an alternative to the coaching activities (component 2) that is cheaper and easier to implement in contexts where there is lack of professional coaches.
Intervention Start Date
2023-05-02
Intervention End Date
2023-09-29

Primary Outcomes

Primary Outcomes (end points)
Teaching practices, teachers' school attendance, and first grade students' reading and writing skills.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Teachers' wellbeing (mental health), teachers' motivation, experience, self-efficacy, empathy, and support from leadership.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We designed an stratified RCT, using department and if the school has only one or more than one first grade teachers as the stratification variables. We randomly allocated the 1,141 schools to one of the following groups:
T1: 230 schools were randomly allocated to participate in the virtual group-based training program on reading and writing skills instruction (component 1 only).
T2: 226 schools were randomly invited to join the virtual group-based training program and to join the one-to-one instructional coaching sessions with a specialist (components 1 and 2).
T3: 230 schools were randomly invited to join the virtual group-based training program and to receive text messages with encouraging nudges (components 1 and 3).
T4: 226 schools were randomly invited to join the virtual group-based training program and to receive text messages with self-reflection resources and exercises (components 1 and 4).
C: 229 schools were randomly allocated to the comparison group. They will not receive any of the TPD components designed for this study.
Experimental Design Details
Not available
Randomization Method
Randomization was done in office by a computer
Randomization Unit
The unit of randomization was the school.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
1,141 schools
Sample size: planned number of observations
The number of teachers included in our sample is 1,600 and the number of students from which we will collect data is 5,700 approximately.
Sample size (or number of clusters) by treatment arms
T1: 230 schools
T2: 226 schools
T3: 230 schools
T4: 226 schools
C: 229 schools
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Universidad Centroamericana Jose Simeon Canas, El Salvador
IRB Approval Date
2023-03-21
IRB Approval Number
IRB 00009046