IMPACT EVALUATION OF A SCHOOL LEADERSHIP DEVELOPMENT PROGRAMME IN WESTERN CAPE, SOUTH AFRICA

The Instructional Leadership Institute (ILI) offers a two-year leadership development program, with the goal of providing all students an equitable education. The ILI uses a rigorous, practice-based approach that aims to improve the quality of teaching and learning by providing leaders with the requisite skills to plan, manage, and support their teachers and schools. The program has five key elements: 1) data-driven instructions, 2) fostering a learning environment via changing school culture, 3) the creation of a feedback culture to foster teachers’ development, 4) leading adult professional development, and 5) planning and monitoring. Specifically, the program comprises five four to five day-long “block weeks” (group training workshops), monthly individual coaching and observation visits by ILI staff to support participants, and online modules and group assignments to reinforce content engaged with during the training.
Program participants include school leaders such as principals, vice-principals, or heads of department as well as “aspiring leaders” - teachers who are on track to take on more leadership responsibilities in their schools. Further, participants also include Circuit and District Managers, who support multiple schools in a defined area.

2022-06-01

2023-12-31

School management practices, Teaching quality, Learner performance (math tests)

School management practices: To measure school management practices, IDinsight uses the Development-World Management Survey (D-WMS). IDinsight has shortened the D-WMS questionnaire used in this study to 17 out of 23 possible sections based on feedback from the implementer about which management aspects school leaders in this context can control. Specifically, we removed sections related to salaries and talent (“people management”) as these are regulated by the Western Cape Education Department (WCED) and not by the school itself - we therefore do not expect the programme to be able to affect these dimensions of school management. The primary outcome will be an overall management index.

Teaching quality: To measure teaching quality, IDinsight uses the World Bank’s Teach Primary tool. Teach Primary includes two sections of observations: 1) the time teachers spend on learning and the extent to which students are on task, and 2) the quality of teaching practices that help develop students’ socioemotional and cognitive skills.

Learner performance: To measure student learning, IDinsight uses a shortened version of South Africa’s standardized Annual National Assessment (ANA) maths test. The full ANA maths test is about 90 minutes long. However, to optimise efficiency and reduce disruption to daily school activities, IDinsight reduced the assessment length to about 30 minutes. The final outcome is a standardized score against the control group obtained from subtracting the mean of raw scores in the control group, divided by standard deviation in the control group.

School management practices (sub-indices)

We include three sub-indices of the D-WMS tool as secondary outcomes: an overall management implementation, usage, and monitoring index. These will allow us to obtain a more nuanced understanding on where the program may contribute to improvements or changes.
To create these sub-indices, we maintain the three sub-questions within each of the 17 sections focused on process implementation, process usage, and process monitoring. We then take a simple average of all process usage and monitoring questions resulting in the three sub-indices:
1) An implementation management index - Simple average of the scores on the first sub-question of all 17 sections included within the customized D-WMS tool. This index assesses the extent to which management practices are effectively formulated, adopted, and implemented in practice.
2) An usage management index - Simple average of the scores on the second sub-question of all 17 sections. This index determines how frequently and efficiently management practices are carried out.
3) A monitoring management index - Simple average of the scores on the third sub-question of all 17 sections. This index captures the extent to which the appropriateness and efficiency of management practices are monitored.

We use a randomized controlled trial (RCT) to assess the effect of the intervention on the key outcomes of 1) school management practices, 2) teaching quality, and 3) student learning. The RCT is complemented by a process evaluation to understand whether the program has been implemented as planned. The RCT comprises a baseline (2022) and endline (2024) study.

Not available

Using Stata and stratifying on the school’s quintile group and whether the school was selected by the ILI using more stringent selection criteria than the ILI initially planned on using.

School

Yes

79 schools

School management practices: 158 (2 school leaders per school) Teaching quality: 158 "classrooms" (2 per school) Learner performance: 2 selected grade 5 classes per school at baseline with up to 16 randomly selected students per class

39 treatment and 40 control schools

School management practices: 0.29 (score points) Teaching quality: 0.36 (score points) Learner performance: 0.28 SD