Improving the teaching of English as a first additional language in South Africa.
Last registered on December 17, 2019

Pre-Trial

Trial Information
General Information
Title
Improving the teaching of English as a first additional language in South Africa.
RCT ID
AEARCTR-0005148
Initial registration date
December 13, 2019
Last updated
December 17, 2019 10:49 AM EST
Location(s)

This section is unavailable to the public. Use the button below to request access to this information.

Request Information
Primary Investigator
Affiliation
GEORGETOWN UNIVERSITY
Other Primary Investigator(s)
PI Affiliation
University of Witwatersrand
PI Affiliation
Department of Basic Education
PI Affiliation
Department of Basic Education
Additional Trial Information
Status
On going
Start date
2017-01-01
End date
2020-12-01
Secondary IDs
Abstract
This study experimentally compares two different interventions aimed at improving early grade reading in English as First Additional Language (EFAL) in poor communities in South Africa. Both interventions are designed to improve and strengthen teachers’ enactment of the existing government curriculum, and consist of the same three components: (1) detailed lesson plans, (2) integrated learning and teaching support material and (3) instructional coaching and training. The main difference between the two intervention models is in the delivery model of the lesson plans and the coaching support. In Intervention 1, the teachers receive a paper-based version of the lesson plans and benefit from regular on-site coaching. In Intervention 2, the teachers receive a tablet with an electronic version of the lesson plans, including various audio-visual resources and are supported through an ICT coaching model that includes telephone calls and cell phone messaging. The electronic lesson plans are provided on an application which is specifically developed for the study. This application contains additional electronic resources such as short training videos, sound clips of the phonics sounds, songs and rhymes and examples of learners’ work.

The program was implemented over a period of three years, tracking the same cohort of 20 students per school from the start of grade one (February, 2017) up to the end of grade three (November, 2019). Each year a different set of teachers were exposed to the program: grade one teachers in year one, grade two teachers in year two, and grade three teachers in year three. There were four rounds of data collection: once prior to the start of the program (January, 2017), and again at the end of every year of implementation (November 2017, 2018 and 2019). Another round of data collection is planned for the end of 2020.
External Link(s)
Registration Citation
Citation
CILLIERS, Jacobus et al. 2019. "Improving the teaching of English as a first additional language in South Africa. ." AEA RCT Registry. December 17. https://doi.org/10.1257/rct.5148-1.0.
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details
Interventions
Intervention(s)
Both interventions aim to apply the same set of instructional practices in the teaching of English as a First Additional Language (EFAL) in the grade 1, 2 and 3 classrooms. Both interventions provide teachers with clear, scripted lesson plans, which are aligned to the official South African government curriculum. The lesson plans incorporate the use of learning support materials, including the government provided workbooks as well as certain additional materials (graded reading booklets, flash cards, big books, posters, etc.), provided by the program. Graded reading booklets were introduced in the second year of the study (as per the recommendation of the CAPS) as a key resource for the teacher to use in group-guided reading and individual work, so as to facilitate reading practice at an appropriate pace and sequence of progression.

Intervention 1 initially introduces the teachers on how to use the lesson plans and accompanying materials through a two-day central training session at the start of the year. During the year, cluster training sessions are held for one-day prior to the start of each academic term to additionally train the teachers on the use of certain core methodologies in the lesson plans. In addition to the training sessions, ongoing support is provided to teachers through ‘specialist’ reading coaches. The on-going support takes on the form of on-site visits by the reading coach on a monthly basis to assist teachers with the implementation of the lesson plans.

Exactly the same set of instructional materials (graded reading booklets, flash cards, big books, posters, etc.) is provided to Intervention 2 schools. However, instead of providing the scripted lesson plans in a paper-based format, the scripted lesson plans are provided in an electronic format on tablets to teachers. In addition to the scripted lesson plans, the tablets also contain short video tutorials for the teachers on teaching practice, audio clips on the pronunciation of the phonics sounds, audio clips of the songs and rhymes and examples of learners’ work. Furthermore, instead of a reading coach visiting the schools on a regular basis, the intervention has a virtual coach who provides on-going support to teachers through phone calls, Whatsapp and SMS messaging. The teachers in intervention 2 are initially introduced to the lesson plans and materials through a three-day central training session at the start of the year, and also receive one-day cluster training sessions at the start of each academic term.
Intervention Start Date
2017-03-01
Intervention End Date
2019-12-01
Primary Outcomes
Primary Outcomes (end points)
1. English language proficiency.
2. English reading proficiency.
Primary Outcomes (explanation)
Both indices are aggregated using principal component analysis, and then standardized to have a control mean of zero and standard deviation of one. The indicators that constitute each index are based on oral and written assessments of students and include:
1. Language proficiency: (i) English expressive vocabulary test; and (ii) English listening comprehension test
2. Reading proficiency: (i) Oral Reading Fluency, (ii) Oral Reading Comprehension; (iii) Written comprehension.
Secondary Outcomes
Secondary Outcomes (end points)
1. Home language proficiency
2. Performance in a mathematical test.
3. Implementation fidelity
4. School support
5. Teacher behavior.
6. Skill acquisition.
3. Teachers' use of resources provided by the program
4. Total time (hours per week) dedicated to teaching English as a First Additional Language
5. Total time (hours per week) dedicated to teaching home language.
6. Teachers' pedagogical knowledge.
7. Teachers' perceived difficulty of teaching the different techniques.
8. Teachers' enjoyment of teaching English as a First Additional Language.
9. Number of written exercises completed by students in the year.
10. An index of the English print-richness of the class environment.
Secondary Outcomes (explanation)
In order to test whether the treatments had any crowding-out or spillover effects on the other subject areas, we will also look at the following outcomes:
1. Home language reading proficiency, constructed using principal component analysis with the following indicators:
a. Letter recognition
b. Oral Reading Fluency
c. Oral reading comprehension.
d. Written comprehension.
2. Performance in a simple mathematics test.

In order to test for mechanisms based on our theory of change, we will also examine the impact on a series of intermediate outcomes:

3. Implementation fidelity
a. Received training [[TRQ 6.4]
b. Frequency that a coach/mentor (i) observes a teacher in EFAL [TRQ 6.6.4], (ii) demonstrate teaching practice [TRQ 6.7.4], or (iii) provide a compliment or praise [TRQ 6.8.4]
c. Have access to graded readers in the classroom [TRQ 4.2.1]
d. Total time spent teaching:
i. English as a First Additional Language [TRQ 3.4]
ii. Home Language [TRQ 3.5]
4. School support.
a. Frequency that the school principal, head of department, or subject advisor (i) observes you teach EFAL [TRQ 6.6.1 - 6.6.3], (ii) demonstrate teaching practice [TRQ 6.7.1 – 6.7.3], or (iii) provide a compliment or praise [TRQ 6.7.1 – 6.7.3]
5. Teacher behavior
a. Use of resources provided by the program, including (i) proportion of teachers who use the Graded Readers daily [TRQ 4.2.3], (ii) proportion of teachers who use Lesson Plans provided by an NGO [TRQ 4.1.3], and (iii) print richness of the environment [TRQ 8]
b. The proportion of the teaching practices that are conducted at the correct weekly frequency [TRQ 4.7; 4.10].
c. The completion of more writing exercises [TRQ 9.3]
6. Skill acquisition: The proportion of teaching activities that a teacher finds difficult to implement [TRQ 4.8]
Experimental Design
Experimental Design
Through a process of elimination we developed a sampling frame of 180 eligible schools. Beginning with 731 primary schools registered in the 2016 administrative data in the districts of Ehlanzeni and Gert Sibande we firstly excluded relatively affluent schools (those in quintiles 4 and 5). Next, we excluded schools in which the language of instruction in the Foundation Phase was neither Siswati, nor isiZulu. We also excluded schools which were missing in the 2014 Annual National Assessment (ANA) dataset. We further excluded particularly small schools (fewer than 30 grade 1 enrolments) since many of these schools would practice multi-grade teaching rendering the grade-specific lesson plans less appropriate. We also excluded particularly large schools (more than 160 grade 1 enrolments, or more than 3 classes in grade 1, or classes with more than 60 learners in) to limit intervention costs. After all of these exclusions 193 eligible schools remained. Using a random number generator, we then excluded 3 further schools to remain with a sample of 190 schools. The 190 school sample included a sample of 10 replacement schools (1 in each of the strata) should the need arise to drop one of the sample schools. Thus we obtained the sampling frame of 190 schools.

To increase power and assure balance between treatment arms, we performed stratified randomization. We created 10 strata of 19 similar schools based on school size, socio-economic status, and previous performance in the Annual National Assessments. Within each stratum, we then randomly assigned 5 schools to each treatment group, 8 to the control group and 1 as a replacement school. Thus we randomly assigned 50 schools to each treatment and 80 to the control.
Experimental Design Details
Not available
Randomization Method
Randomization done in office on Stata.
Randomization Unit
School level.
Was the treatment clustered?
Yes
Experiment Characteristics
Sample size: planned number of clusters
180 schools.
Sample size: planned number of observations
At baseline we have a sample of 3600 students: 20 students per school.
Sample size (or number of clusters) by treatment arms
80 schools control, 50 schools each of the two treatment arms.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Given that we collect data on 20 grade 1 learners per school, this sample should be sufficient to identify a minimum effect size of 0.21 standard deviations when comparing a treatment group with the control group and a minimum effect size of 0.23 standard deviations when comparing two treatment groups. These calculations assume a 95% confidence interval, an alpha value of 0.8, an intra-class correlation coefficient (rho) of 0.3 and a correlation between pre- and post-test scores of 0.7.
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
University of Witwatersrand Human Research Ethics Committee
IRB Approval Date
2014-12-12
IRB Approval Number
N/A
Analysis Plan
Analysis Plan Documents
The Second Early Grade Reading Study in South Africa: Pre-Analysis Plan

MD5: 3c93b3ee17f8b62e734382800f286e2a

SHA1: 914916d32fe4f50fc4721f4a8ee67cb60f3a89a9

Uploaded At: December 13, 2019