Raising outcomes in primary education, a cluster randomised trial in rural Guinea Bissau

Last registered on March 15, 2019

Pre-Trial

Trial Information

General Information

Title
Raising outcomes in primary education, a cluster randomised trial in rural Guinea Bissau
RCT ID
AEARCTR-0003670
Initial registration date
February 04, 2019

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
March 15, 2019, 1:45 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
March 15, 2019, 5:41 PM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
Effective Intervention

Other Primary Investigator(s)

PI Affiliation
Teachers College, Columbia University
PI Affiliation
University of Oxford, Department of Education
PI Affiliation
Kogod School of Business, American University and Econometric Institute, Erasmus University Rotterdam
PI Affiliation
Effective Intervention

Additional Trial Information

Status
Completed
Start date
2012-12-01
End date
2018-06-30
Secondary IDs
ISRCTN67389010
Abstract
Most children in rural Guinea Bissau leave school without basic literacy and numeracy skills. Although enrolment in rural areas has increased from 24% to 60% over the past ten years, competencies in language and maths remain very low. Primary schools in rural Guinea Bissau are characterised by poorly trained, insufficient and under motivated teachers, a lack of materials for teaching and learning, poor infrastructure and inadequate didactic teaching pedagogy. Effective Intervention has designed ROPE - Raising Outcomes in Primary Education, a programme to deliver quality basic primary education for about 700 children in south rural Guinea Bissau that focus on hiring, training and monitoring teachers to provide a better quality teaching . The aim of this study is to assess the impact of this programme by comparing the test scores of children four years after the project has been implemented.
External Link(s)

Registration Citation

Citation
Boone, Peter et al. 2019. "Raising outcomes in primary education, a cluster randomised trial in rural Guinea Bissau." AEA RCT Registry. March 15. https://doi.org/10.1257/rct.3670-3.0
Former Citation
Boone, Peter et al. 2019. "Raising outcomes in primary education, a cluster randomised trial in rural Guinea Bissau." AEA RCT Registry. March 15. https://www.socialscienceregistry.org/trials/3670/history/43503
Experimental Details

Interventions

Intervention(s)
The intervention programme comprised: i) a high quality in-service teacher training with regular monitoring and educational support; ii) the development and use of learning materials for teachers and pupils from preschool to 3rd grade. The academic support to children started in February 2014 (preschool) and was given to the same group until June 2018.
Primary teachers (21 initially) were placed in the villages assigned to intervention and three supervisors were responsible for monitoring and giving educational support to them.

Supervisors and Teachers’ Training
To create a high quality teaching group, supervisors and teachers were selected based on their fluency in one of the local indigenous languages, academic level, experience in teaching, knowledge in Portuguese and mathematics, pedagogical skills and motivation.
Supervisors received an intense training in Portuguese during 3 months and were rigorously tested by the end. Only those who had achieved B2 level according to the Common European Framework of Reference for Languages continued to receive training. From February 2012 to April 2012, they received training on personal and professional development, humans-rights, child development, teacher role in the school and community, classroom management, pedagogy methods and techniques, evaluation, planning, supervision and preparative activities (preschool). Following that they had another 6 months in-service training.
Teachers were selected later through a process that included two-week training to access their skills. During August and September 2013 they received training on classroom management, pedagogy methods and techniques, evaluation, planning, supervision and preparative activities (preschool).
Every year before the academic school year started, teachers received an intensive training on scientific and pedagogical topics listed in the national curriculum for the grade they were going to teach in that year. Further, they received regular and frequent tutoring and monitoring.

Intervention Classes
Selected and trained teachers were appointed to villages randomly allocated to intervention. During the first 9 months, they worked preparing pupils (aged 4, 5 and 6 years old) to enter in 1st grade. Then, teachers worked during 3 years with the same group of children until the end of 3rd grade. Teachers conducted support classes as and when required, with those children having learning difficulties or absenteeism during the project period.

Teachers’ supervision and educational support
Throughout the study these selected teachers were closely supported and monitored by supervisors and a Quality Control Group (QCG). The QCG was composed by the trainers of teachers.
Supervisors and the QCG conducted regular classroom observation to evaluate teachers’ performance, pupils’ learning, and guide teachers’ work.
Teachers had monthly meeting for lessons' planning with supervisors, and a monthly review meeting with all the team members.
To evaluate pupils’ learning outcomes, teacher and supervisor conducted tests prepared by the QCG, monthly and quarterly.

Learning and Teaching manuals
Teaching manuals and pupils’ exercise books were developed for introductory, 1st, 2nd and 3rd grades in all subjects that are included in the National Curriculum of Guinea Bissau. These manuals helped teachers to explore all the topics found in the curriculum, structure their classes linking their lessons to the pupils’ exercise books.
Community Involvement
During July and September 2013 and end of January 2014, supervisors conducted sensitisation and mobilisation campaigns in the communities randomly selected to receive the intervention. They promoted activities to motivate children to attend academic classes and sensitise parents and community to the importance of education and their role in ensuring better outcomes.
Parent Teacher Associations (PTA) were created and fomented to promote active participation and involvement of parents and communities in the process of learning.
An annual plan of the academic classes’ activities was presented and discussed with communities. PTAs were involved in the management of the academic classes and monitoring of children’s attendance and their learning progress. Children who are not performing were given extra time after the regular teaching time and parents were sensitised to support their children.
Intervention Start Date
2014-02-01
Intervention End Date
2018-06-30

Primary Outcomes

Primary Outcomes (end points)
Primary outcome
The primary outcome of the study is a composite test score, calculated as the arithmetic mean of two separate test scores: language (early grade reading assessment - EGRA) and mathematics (early grade mathematics assessment - EGMA). The EGRA and EGMA test scores will calculated separately and considered as secondary outcomes.
Primary Outcomes (explanation)
EGRA and EGMA will be formally tested are as follows:
1. EGMA test score, to be calculated as a simple arithmetic mean of the percentage of correct answers on each of the six (some composite) subtasks, evenly weighting each task and not accounting for time remaining. The six subtasks are 1, 2, 3, 4 [the sum of 4a and 4b], 5 [the sum of 5a and 5b] and 6;

2. EGRA test score, to be calculated as a simple arithmetic mean of the percentage of correct answers on each of the seven subtasks, evenly weighting each task and not accounting for time remaining. The seven subtasks are 1, 2, 3, 4, 5a, 5b and 6

Secondary Outcomes

Secondary Outcomes (end points)
Secondary outcomes will be:
1. EGMA (mathematics) test score
2. EGRA (reading) test score
3. Timed subtask scores of EGMA and EGRA
4. Untimed subtask scores of EGMA and EGRA
5. Parent’s report of school attendance
6. Enrolment in school
7. Literacy and numeracy of next elder and younger sibling
8. Whether sibling enrolled in school
9. Residence status during the trial
Secondary Outcomes (explanation)
Explanation of how the secondary outcomes will calculated and presented:
1. EGMA (mathematics) test score - calculated as a simple arithmetic mean of the percentage of correct answers on each of the seven subtasks, evenly weighting each task and not accounting for time remaining. The seven subtasks are 1, 2, 3, 4, 5a, 5b and 6;
1. EGRA (reading) test score - calculated as a simple arithmetic mean of the percentage of correct answers on each of the seven subtasks, evenly weighting each task and not accounting for time remaining. The seven subtasks are 1, 2, 3, 4, 5a, 5b and 6;
2. For each timed subtask scores, we will present the average percentage: i) of correct answers; ii) excluding zero; iii) scored >0 but <30% of max. score; iv) scored >30 % but <60% of maximum score; v) scored >60% of maximum score; and vi) average fluency score.
3. For each untimed subtask scores, we will present the average percentage: i) of correct answers; ii) excluding zero; iii) scored >0 but <40% of max. score; iv) scored >40 % but <80% of maximum score; and v) scored >80% of maximum score;
4. Parent’s report of school attendance: number of days of school missed in past two weeks, conditional on enrolment (measured twice during the trial, in 2015 and 2017, in the middle of the academic year);
5. Enrolment in school (yes/no, grade enrolled, measured together with school attendance);

6. Literacy and numeracy of next elder and younger sibling (variable in rank order 0-4, measure in Mar 2017 survey)
7. Whether sibling enrolled in school (yes/no, measured in Mar 2017)
8. Residence status during the trial (resident/not resident/don't know, measured in Jan 2015 and Mar 2017)

Experimental Design

Experimental Design
ROPE is a cluster-randomized trial involving 49 villages in the districts of Quinara and Tombali, where a village is the unit of randomisation.
It was unblinded given that parents were aware of whether or not they were in a village that was receiving materials and teaching support. However, a strict procedure to collect data and test children was applied to avoid any bias. The research team that conduct the surveys and tested the children were in principle blind.
The design was stratified. We conducted a cluster analysis to group the villages into homogeneous strata regarding the following variables: village’s distance to road, highest grade taught by the school if the village had a school (the grade was zero if the village did not have a school), number of households in the village, proportion of mothers speaking Crioulo in the village, and third quartile of mothers’ education in the village. We selected these variables on the assumption that they are correlated with the primary outcome, as showed by the NBER education study (Boone et al. 2013). The results of the cluster analysis suggested that two strata were sufficient (the dendogram is shown in Figure 1), one with 17 villages, another one with 32. The cluster analysis was conducted in SAS Software version 9.3, using PROC CLUSTER.
Experimental Design Details
Randomization Method
The villages were randomised to intervention or control within each of the two strata formed, using an algorithm developed with SAS Software version 9.3.
There are 49 participating villages grouped in two strata, 17 in stratum 1 and 32 in stratum 2. There is a fixed number of 26 teachers (see below), who were allocated to strata proportionally to the strata sizes: 26 x 17/49=9 to stratum 1 and 26 x 32/49=17 to stratum 2. The algorithm selects villages at random without replacement, within each stratum, until reaching the fixed number of teachers assigned to that stratum. The allocation ratio for intervention and control villages was not 1:1, but determined by randomisation (roughly 1/3 in the first strata and 2/3 in the 2nd).
Randomization Unit
We had only one level of randomisation, and its unit of was the village.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
This trial was conducted in 49 clusters in the rural districts of Quinara and Tombali for four years. We used the most recent Census and field assessment to select villages with less than 400 households that were accessible by land during the dry season and 9 Km apart from each other. They are a representative sample of all villages accessible by land with the number of households in each village between 44 and 256 (approximately 270 to 1500 inhabitants).
The mean number of enumerated children per village during the enumeration survey of December 2012 to April 2013 was 43. As per our previous experience, about 17% of the children were expected to be lost to follow-up during the study (Mann et al. 2009), and as a result, an average of 35 children per village are expected to be present for the final test and therefore analysis. Forty-nine (49) villages with an average of 35 eligible children per village will provide 92% power to detect a difference in scores of 0.25 (25%) of the SD in a two-sided 5% test. An Intra cluster correlation coefficient ICC=0.03 is assumed, which gives for this trial, with 35 children per village, a design effect of DE=2.02.

Due to the limited number of teachers, there were 16 villages randomly attributed to intervention (649 children enumerated) and 33 to control (1463 children enumerated).
The limited number of teachers hired was due to a problem that we faced during the first phase, when we were identifying the teachers that would be in charge of the intervention classes. These classes were initially planned to be implemented by a group of young people who had not necessarily gone through a ‘formal’ teacher training. It was essential to have fluency in Portuguese and willingness to work in rural communities as a primary teacher. However, the first group that had been trained for one year refused to work with the conditions that had been initially offered, and a new recruitment of teachers had to be done, but this time with trained teachers. Given the design of the intervention with a fixed maximum number of children per teacher, a stringent selection process of the teachers and an urgency to start the intervention, we anticipated that it would not be feasible to hire more than 25 teachers.
Sample size: planned number of observations
We enumerated 2112 children to participate in the trial. The final test was conducted with 1644 children.
Sample size (or number of clusters) by treatment arms
16 villages randomly attributed to intervention (649 children enumerated) and 33 to control (1463 children enumerated).
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
June 01, 2018, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
December 21, 2017, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
49
Was attrition correlated with treatment status?
Final Sample Size: Total Number of Observations
Final Sample Size (or Number of Clusters) by Treatment Arms
16 clusters (villages) in the intervention arm and 33 clusters (villages) in the control arm
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials