AEA RC T Registr y

The American Economic Association's registry for randomized controlled trials

This study evaluates three different interventions, all aimed at improving early-grade reading in the home language, which in the case of the North West province is Setswana. All three interventions work with children entering Grade 1 at the start of 2015 over a two-year period (thus working with grade 2 learners in 2016).

Treatments 1 and 2 aim to apply the same set of instructional practices in the teaching of home language literacy in grade 1 and 2 classrooms. Both treatments therefore provide teachers with clearly scripted lesson plans, which are aligned to the curriculum as specified in the Curriculum and Assessment Policy Statements (CAPS) for home language literacy in the Foundation Phase. The lesson plans incorporate the use of learning support materials including the government-provided workbooks as well as certain additional materials (graded reading booklets, flash cards, posters, etc.), which are provided through the EGRS. The graded reading booklets provide a key resource for the teacher to use in group-guided reading and individual work so as to facilitate reading practice at an appropriate pace and sequence of progression.

Treatment 1 trains the teachers on how to use the lesson plans and accompanying materials through central training sessions, each lasting 2 days, and occurring twice yearly. The first session was conducted in January 2015 and the second occurred in July 2015. Similar sessions are scheduled for 2016.

Treatment 2 (Reading Coaches, scripted lessons, graded readers) has exactly the same set of instructional materials (scripted lesson plans, graded reading booklets and other materials). However, instead of central training sessions, ongoing support to teachers consisting of regular (monthly) on-school coaching from specialist “reading coaches” is provided. In addition to these on-site visits, there are occasional meetings with the coach and a small cluster of nearby Treatment 2 schools. The evaluation of treatments 1 and 2 should thus shed light on whether the fairly prescriptive instructional regime has the ability to improve reading acquisition and whether the mode of teacher support is important in mediating effectiveness.

Treatment 3 (parental involvement) is designed to promote parental involvement to support their children’s reading progress. At each of the 50 schools in this treatment arm a Community Reading Coach (CRC) was recruited. The CRC was identified through communication with the school principal who recommended a suitably qualified but available person in the community. The CRCs attend a 1-day training session facilitated by the service provider (Class Act) at the start of each school term (quarterly). The CRCs are trained to deliver weekly training sessions for grade 1 parents at their respective schools. A total of 30 sessions is scheduled for each year covering a total of 10 topics. Each topic has 3 sessions where the topic is the same but the activities of the session differ. Thus a parent can attend roughly 1 in 3 sessions and still be exposed to all topics, while parents who attend more regularly can still enjoy fresh activities. For their services, CRCs are paid a stipend of R400 per month (about $35).

2015-01-19

2016-12-01

Reading proficiency.

For reading proficiency we will measure all the intermediate steps towards comprehension: letter recognition, phonetic awareness, word recognition, fluency, and reading with comprehension. These tests will be adapted from standard tests that have already been developed for the Setswana language, such as the Early-Grade-Reading-Assessment (EGRA). As our main outcome indicator, we will also construct an aggregate indicator of learning proficiency, using principal component analysis.

The study is a stratified cluster RCT, randomized at the school level. 80 schools are assigned the control and 50 schools in each of the three treatment groups.

Through a process of elimination we developed a sampling frame of 230 eligible schools. Beginning with 458 primary schools registered in 2014 administrative data in the districts of Dr Kenneth Kaunda and Ngaka Modiri Molema we started by excluding relatively affluent schools (those in quintiles 4 and 5). Next, we excluded schools in which the language of instruction in the Foundation Phase was not Setswana. We excluded schools which were missing in the 2014 ANA dataset. We also excluded 8 schools that had already been selected for the purposes of piloting of instruments through the course of this project. We further excluded particularly small schools (fewer than 20 grade 1 enrolments) since many of these schools would practice multi-grade teaching rendering the scripted lesson plans less appropriate. We also excluded particularly large schools (more than 180 grade 1 enrolments) to limit intervention costs. Three more schools were excluded after the North West PED checked our list of schools and found specific problems with these schools (e.g. the school had been closed down, or a particular conflict around school management was occurring in a school). After all of these exclusions 235 eligible schools remained. Using a random number generator, we then excluded 5 schools, which we retained as possible replacement schools. Thus we obtained the sampling frame of 230 schools.
To increase power and assure balance between treatment arms, we performed stratified randomization. We created 10 strata of 23 similar schools based on school size, socio-economic status, and previous performance in the Annual National Assessments. Within each stratum, we then randomly assigned 5 schools to each treatment group and 8 to the control group. Thus we randomly assigned 50 schools to each treatment and 80 to the control. Given that we collect data on 20 grade 1 learners per school, this sample should be sufficient to identify a minimum effect size of 0.21 standard deviations when comparing a treatment group with the control group and a minimum effect size of 0.23 standard deviations when comparing two treatment groups. These calculations assume a 95% confidence interval, an alpha value of 0.8, an intra-class correlation coefficient (rho) of 0.3 and a correlation between pre- and post-test scores of 0.7.

Randomization was done by computer.

School.

Yes

230 schools

4600 pupil - 20 pupils per school. 230 schools

80 schools in control
50 schools in each treatment group.

Given that we collect data on 20 grade 1 learners per school, this sample should be sufficient to identify a minimum effect size of 0.21 standard deviations when comparing a treatment group with the control group and a minimum effect size of 0.23 standard deviations when comparing two treatment groups. These calculations assume a 95% confidence interval, an alpha value of 0.8, an intra-class correlation coefficient (rho) of 0.3 and a correlation between pre- and post-test scores of 0.7.

Improving Early-Grade Reading - Pre-Analysis Plan