Improving Early-Grade Reading in South Africa
Last registered on October 31, 2016

Pre-Trial

Trial Information
General Information
Title
Improving Early-Grade Reading in South Africa
RCT ID
AEARCTR-0001749
Initial registration date
October 31, 2016
Last updated
October 31, 2016 10:54 AM EDT
Location(s)
Primary Investigator
Affiliation
GEORGETOWN UNIVERSITY
Other Primary Investigator(s)
PI Affiliation
Human Sciences Resource Council (HSRC)
PI Affiliation
WITS University
PI Affiliation
Department of Basic Education
Additional Trial Information
Status
On going
Start date
2015-01-05
End date
2017-05-01
Secondary IDs
Abstract
Scripted lesson plans have great potential to improve teaching practice in resource- and capacity-constrained settings, but there are risks that they undermine teachers' autonomy to cater teaching to the level of the child, especially if lesson plans require adherence to an overly-ambitious curriculum. We present results of a randomized evaluation of two different programs aimed at improving early-grade reading in poor schools in South Africa. Both programs provide teachers with scripted lesson plans and supporting reading materials, but they differ in the mode of implementation. In some schools (Training) teachers receive two two-day training sessions over the course of the year. In other schools (Coaching), teachers also receive monthly visits from specialized reading coaches. We report results on reading proficiency after two years of implementation, and conduct cost-benefit analysis to determine which intervention is the most cost-effective. To better understand why these interventions worked, we also test for changed teaching practice, such as providing regular assessment and streaming pupils by ability. Finally, we measure how the impact varies across the distribution of baseline student reading proficiency, and if this is mediated by class size.
External Link(s)
Registration Citation
Citation
Cilliers, Jacobus et al. 2016. "Improving Early-Grade Reading in South Africa." AEA RCT Registry. October 31. https://www.socialscienceregistry.org/trials/1749/history/11545
Experimental Details
Interventions
Intervention(s)
This study evaluates three different interventions, all aimed at improving early-grade reading in the home language, which in the case of the North West province is Setswana. All three interventions work with children entering Grade 1 at the start of 2015 over a two-year period (thus working with grade 2 learners in 2016).

Treatments 1 and 2 aim to apply the same set of instructional practices in the teaching of home language literacy in grade 1 and 2 classrooms. Both treatments therefore provide teachers with clearly scripted lesson plans, which are aligned to the curriculum as specified in the Curriculum and Assessment Policy Statements (CAPS) for home language literacy in the Foundation Phase. The lesson plans incorporate the use of learning support materials including the government-provided workbooks as well as certain additional materials (graded reading booklets, flash cards, posters, etc.), which are provided through the EGRS. The graded reading booklets provide a key resource for the teacher to use in group-guided reading and individual work so as to facilitate reading practice at an appropriate pace and sequence of progression.

Treatment 1 trains the teachers on how to use the lesson plans and accompanying materials through central training sessions, each lasting 2 days, and occurring twice yearly. The first session was conducted in January 2015 and the second occurred in July 2015. Similar sessions are scheduled for 2016.

Treatment 2 (Reading Coaches, scripted lessons, graded readers) has exactly the same set of instructional materials (scripted lesson plans, graded reading booklets and other materials). However, instead of central training sessions, ongoing support to teachers consisting of regular (monthly) on-school coaching from specialist “reading coaches” is provided. In addition to these on-site visits, there are occasional meetings with the coach and a small cluster of nearby Treatment 2 schools. The evaluation of treatments 1 and 2 should thus shed light on whether the fairly prescriptive instructional regime has the ability to improve reading acquisition and whether the mode of teacher support is important in mediating effectiveness.

Treatment 3 (parental involvement) is designed to promote parental involvement to support their children’s reading progress. At each of the 50 schools in this treatment arm a Community Reading Coach (CRC) was recruited. The CRC was identified through communication with the school principal who recommended a suitably qualified but available person in the community. The CRCs attend a 1-day training session facilitated by the service provider (Class Act) at the start of each school term (quarterly). The CRCs are trained to deliver weekly training sessions for grade 1 parents at their respective schools. A total of 30 sessions is scheduled for each year covering a total of 10 topics. Each topic has 3 sessions where the topic is the same but the activities of the session differ. Thus a parent can attend roughly 1 in 3 sessions and still be exposed to all topics, while parents who attend more regularly can still enjoy fresh activities. For their services, CRCs are paid a stipend of R400 per month (about $35).
Intervention Start Date
2015-01-19
Intervention End Date
2016-12-01
Primary Outcomes
Primary Outcomes (end points)
Reading proficiency.
Primary Outcomes (explanation)
For reading proficiency we will measure all the intermediate steps towards comprehension: letter recognition, phonetic awareness, word recognition, fluency, and reading with comprehension. These tests will be adapted from standard tests that have already been developed for the Setswana language, such as the Early-Grade-Reading-Assessment (EGRA). As our main outcome indicator, we will also construct an aggregate indicator of learning proficiency, using principal component analysis.
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
The study is a stratified cluster RCT, randomized at the school level. 80 schools are assigned the control and 50 schools in each of the three treatment groups.

Through a process of elimination we developed a sampling frame of 230 eligible schools. Beginning with 458 primary schools registered in 2014 administrative data in the districts of Dr Kenneth Kaunda and Ngaka Modiri Molema we started by excluding relatively affluent schools (those in quintiles 4 and 5). Next, we excluded schools in which the language of instruction in the Foundation Phase was not Setswana. We excluded schools which were missing in the 2014 ANA dataset. We also excluded 8 schools that had already been selected for the purposes of piloting of instruments through the course of this project. We further excluded particularly small schools (fewer than 20 grade 1 enrolments) since many of these schools would practice multi-grade teaching rendering the scripted lesson plans less appropriate. We also excluded particularly large schools (more than 180 grade 1 enrolments) to limit intervention costs. Three more schools were excluded after the North West PED checked our list of schools and found specific problems with these schools (e.g. the school had been closed down, or a particular conflict around school management was occurring in a school). After all of these exclusions 235 eligible schools remained. Using a random number generator, we then excluded 5 schools, which we retained as possible replacement schools. Thus we obtained the sampling frame of 230 schools.
To increase power and assure balance between treatment arms, we performed stratified randomization. We created 10 strata of 23 similar schools based on school size, socio-economic status, and previous performance in the Annual National Assessments. Within each stratum, we then randomly assigned 5 schools to each treatment group and 8 to the control group. Thus we randomly assigned 50 schools to each treatment and 80 to the control. Given that we collect data on 20 grade 1 learners per school, this sample should be sufficient to identify a minimum effect size of 0.21 standard deviations when comparing a treatment group with the control group and a minimum effect size of 0.23 standard deviations when comparing two treatment groups. These calculations assume a 95% confidence interval, an alpha value of 0.8, an intra-class correlation coefficient (rho) of 0.3 and a correlation between pre- and post-test scores of 0.7.
Experimental Design Details
Randomization Method
Randomization was done by computer.
Randomization Unit
School.
Was the treatment clustered?
Yes
Experiment Characteristics
Sample size: planned number of clusters
230 schools
Sample size: planned number of observations
4600 pupil - 20 pupils per school. 230 schools
Sample size (or number of clusters) by treatment arms
80 schools in control
50 schools in each treatment group.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Given that we collect data on 20 grade 1 learners per school, this sample should be sufficient to identify a minimum effect size of 0.21 standard deviations when comparing a treatment group with the control group and a minimum effect size of 0.23 standard deviations when comparing two treatment groups. These calculations assume a 95% confidence interval, an alpha value of 0.8, an intra-class correlation coefficient (rho) of 0.3 and a correlation between pre- and post-test scores of 0.7.
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
RESEARCH ETHICS COMMITTEE
IRB Approval Date
2015-01-21
IRB Approval Number
REC 8/19/2/14​
Analysis Plan
Analysis Plan Documents
Improving Early-Grade Reading - Pre-Analysis Plan

MD5: e6362a2a2a9f2f74a0328bb1d09d705a

SHA1: 037b3049b9b84b39ca350df0bd95f5e0a7ed01c2

Uploaded At: October 31, 2016

Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
No
Is data collection complete?
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports and Papers
Preliminary Reports
Relevant Papers