Effects of Quality Improvement Strategies on Early Child Development in Community-based Childcare Centers in Malawi: A Cluster-Randomized Trial

Last registered on July 02, 2018

Pre-Trial

Trial Information

General Information

Title
Effects of Quality Improvement Strategies on Early Child Development in Community-based Childcare Centers in Malawi: A Cluster-Randomized Trial
RCT ID
AEARCTR-0001118
Initial registration date
March 11, 2016

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
March 11, 2016, 11:54 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
July 02, 2018, 9:51 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
The World Bank

Other Primary Investigator(s)

PI Affiliation
University of California, Berkeley

Additional Trial Information

Status
Completed
Start date
2011-09-01
End date
2018-06-30
Secondary IDs
Abstract
The majority of children who attend preschool in poor countries attend informal ones, which are often staffed by teachers with low levels of education and minimal formal training. We evaluate a government program in Malawi, which focused on improving quality at community-based childcare centers (CBCCs) and complemented these efforts with a group-based parenting support program. The study, which is a cluster-randomized controlled trial with four arms, covers 199 CBCCs in four districts of Malawi.
External Link(s)

Registration Citation

Citation
Fernald, Lia and Berk Ozler. 2018. "Effects of Quality Improvement Strategies on Early Child Development in Community-based Childcare Centers in Malawi: A Cluster-Randomized Trial." AEA RCT Registry. July 02. https://doi.org/10.1257/rct.1118-2.0
Former Citation
Fernald, Lia and Berk Ozler. 2018. "Effects of Quality Improvement Strategies on Early Child Development in Community-based Childcare Centers in Malawi: A Cluster-Randomized Trial." AEA RCT Registry. July 02. https://www.socialscienceregistry.org/trials/1118/history/31403
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
The Protecting Early Childhood Development Project (PECD), co-designed by the Ministry, World Bank officials, and academic partners from UC Berkeley to ensure both technical, scientific, and policy relevance, was a two-year project aimed to mitigate some of the negative impacts of the global economic crisis on young children’s development by testing strategies to improve the quality and stability of existing CBCCs. The interventions focused on: (a) improving the play and learning resources in CBCCs; (b) strengthening the capacity of teachers to support children’s early development and learning; (c) teaching parents about how to support development and learning activities in the home. Under PECD, the Government implemented the following interventions – in partnership with Save the Children and UNICEF:

T1. Comparison Group: Provision of play and learning materials
To address the basic developmental and learning needs of children, each center in the study received a kit of basic play and learning materials and supplies procured by UNICEF. The contents of the kit were developed by the Ministry and included items such as books, displays, balls, paint, chalk, blocks, puzzles, first aid kit, and kitchen utensils. To participate in the project, CBCCs had to make space available for secure storage of the materials; all CBCCs that were selected chose to participate.

T2. T1 + Training and mentoring of teachers
To improve the knowledge, skills, and practices of teachers in the 150 CBCCs assigned to the three treatment groups, the project tested an enhanced teacher-training package. The teacher-training component followed a cascade model, where national ECD specialists trained regional trainers, who then trained the teachers at CBCCs within their regions, and aimed to build early childhood development (ECD) capacity at the national, district, and community levels. Each CBCC in a treatment group nominated two teachers to participate in the training program. Training candidates were required to hold a Junior Certificate of Education (JCE). If none of the active teachers at the CBCC met this minimum qualification, it was recommended that the CBCC seek and nominate an eligible volunteer from the same village. However, in practice, candidates with only a primary school leaving certificate (PSLC) were nominated by the centers and were accepted for training.

The teacher-training program consisted of five weeks of residential training divided into two sessions of two weeks each and one final one-week session. The enhanced training program built on and expanded the existing 14-day training that was available to all teachers and covered the following modules: child development; play and early learning materials and equipment; learning through play; planning and organizing the learning environment; child health and care; child hygiene and environmental care; child nutrition and care; child rights and their welfare; care and development of children with special needs; early childhood care, management, and partnership. Between the three face-to-face training sessions, the trained teachers went back to their CBCCs for a few months to participate in “field practice” with support from supervisors and mentors. During these field practice periods, the teachers in training held briefing meetings with the untrained teachers at the CBCC. 310 teachers from 150 treatment CBCCs (out of a total of 468 teachers at these CBCCs) completed all three phases of training.

Mentors and supervisors were trained for three days after the first face-to-face training session. Mentors were teachers selected based on their exceptional performance, reputation in the community, and commitment to their work. Mentors provided guidance and support to teachers in their respective districts through weekly visits. Each mentor was assigned to four CBCCs translating to a ratio of one mentor to eight teachers. Supervisors were Child Protection Workers, government-employed social workers, who already worked within the area of the CBCCs. These 19 experienced Child Protection Workers were each responsible for supervising eight CBCCs on average. National and district officials, along with Save the Children staff, also conducted monitoring visits. Finally, to strengthen the capacity of Center Management Committees (CMC) in carrying out their responsibilities to managing the centers, the project included a five-day training conducted within the communities, which reached 1,499 committee members from 150 CBCCs.

T3. T2 + Teacher incentives
In each of the 49 CBCCs assigned to this group, the teachers who successfully completed the teacher-training program were given a small monthly stipend (MK 2,000) for a period of seven months to increase retention and perhaps improve motivation. Unfortunately, the intervention period coincided with an economic crisis in Malawi, which saw the value of this payment depreciate from the equivalent of $12 to $6 per month from April to November 2012. Save the Children personnel administered the payments and ensured that the trained teachers received the incentives. In several communities, CMCs raised some funds to make comparable payments to the teachers who did not receive the enhanced teacher training.

T4. T2 + Parenting education
Primary caregivers of three and four year-old children attending the 51 CBCCs assigned to this arm participated in group sessions that provided information and demonstrated practical activities that they could replicate at home to maximize the school readiness of their children. Parent educators – Child Protection Workers, trained teachers, and mentors, who received three days of training for this task – facilitated the sessions. The initial implementation of the parenting education arm, organized by Save the Children, deviated from the original project design in that the 12 parenting sessions were first conducted in 12 consecutive days instead of 12 days spread out over six weeks. To rectify this implementation error, starting approximately one month later, the facilitators held six additional “refresher” sessions of two hours each over a period of six weeks. In between these weekly sessions, parents and guardians were encouraged to practice with their children at home what they had learned.

The sessions covered the following topics: introduction to child development; physical development, mathematical and critical thinking; general knowledge and scientific thinking; language development; literacy; social and emotional development; supporting spiritual and moral development; supporting children’s approaches to learning; supporting children’s health and safety; supporting children’s nutrition and food safety. Participants used items from the kits provided to the CBCCs by UNICEF along with locally available materials, such as leaves, stones, soil, feathers, bean bags, charcoal, drums, etc.
Intervention Start Date
2012-04-01
Intervention End Date
2013-07-31

Primary Outcomes

Primary Outcomes (end points)
Child Measures:

A comprehensive battery of child development measures was used to assess language, fine motor, executive function (attention, inhibition, working memory), problem solving, social/emotional and numeracy/math skills. These measures cover abilities that typically begin to emerge and progress early in life; are encouraged through commonly recommended preschool practices; and are believed to be important for primary school success (Copple & Bredekamp, 2009; Duncan et al., 2007; Sabol & Pianta, 2012). As described below, all selected assessments had demonstrated reliability and/or validity in either Malawi or other sub-Saharan countries. Each test was translated and adapted as necessary for use in the present study. At the 36-month follow-up (Round 3), some scales were dropped because they no longer showed good variability in performance (i.e., were too easy), while other tests indicative of expanding capacities were added. For all analyses, scores were standardized to have a mean equal to 0 and standard deviation equal to 1 in the control group for ease of interpretation.
The battery included the following measures (see Appendix Table 1 for schedule of administration across rounds of data collection):
1. Malawi Developmental Assessment Tool (MDAT(Gladstone et al., 2008), a test created and validated specifically for use in rural Malawi with children 0-7 years of age. Subscales for assessing language and fine motor/perception skills were administered. Items were scored as pass or fail, and a total summed score was calculated overall, and for each subscale.
2. Peabody Picture Vocabulary Test - IV (PPVT-IV,(Dunn, 1965) a test of receptive vocabulary that measures comprehension of words through picture identification. The PPVT has been widely used throughout the world for assessing the effects of various interventions on child language, including Mozambique (TVIP, the Spanish version of PPVT, administered in the local language; (Martinez et al., 2012)) and Madagascar (Fernald et al., 2009). Specific items (both words and pictures) were modified for use in Malawi. For example, we replaced “apple” with “papaya,” a fruit that is well known throughout the country, and was estimated to be of similar difficulty as the word “apple” would be in the United States. Items were scored as pass or fail, and a summed continuous score was calculated.
3. The Leiter-R Sustained Attention (LSA) task (Roid & Miller, 1997), a language-free measure that assesses how well children can continue to maintain attention and accuracy during a timed visual search task. The measure has successfully detected group differences in performance in Madagascar (Fernald et al., 2011). Total adjusted scores were determined by subtracting the numbers of errors from the number of correct responses.
4. The Strengths and Difficulties Questionnaire (SDQ) (Goodman, 2001; Woerner et al., 2004), a brief, parent-report questionnaire that screens for both behavioral problems and pro-social (positive) behaviors. All items were translated, back-translated and approved by the test author. The SDQ has been used in several African countries, including Kenya (Oburu, 2005) and South Africa (Cluver et al., 2007). Scores were determined for the four behavior problem subscales, the pro-social subscale, and a total difficulties (problem) score.
5. Kaufman Assessment Battery-Children, 2nd Edition (KABC-II) (Kaufman & Kaufman, 2004), three scales were adopted: Hand Movements is a non-verbal, short-term motor memory task requiring children to copy increasingly difficult hand movement sequences. A total score of passed items was calculated. Number Recall is a short-term auditory memory task requiring children to repeat a series of increasingly difficult number sequences. As children learn numbers in English, no translation was required. The total score reflects the number of passed items. Finally, Triangles is a non-verbal problem-solving task that requires children to complete increasingly complex patterns and figures with plastic and foam shapes. The number of items correctly completed was summed. The Kaufman scales have been used in Kenya (Holding et al., 1999), Senegal (Boivin, 2002), and Uganda (Bangirana et al., 2009).
6. Early Grade Mathematics Assessment (EGMA) (Brombacher, 2011), a tool developed by USAID to measure early knowledge of numbers and basic math skills, validated in Malawi. A great advantage of the EGMA is that there are Malawian norms available, as well as norms from neighboring countries, allowing for easy comparison and interpretation of scores. Three subscales (number recognition, quantity discrimination, and addition) were administered. Passed items for each subscale were summed to create subscale scores.
Anthropometric measurements were made at baseline to (i) assess balance across groups, and (ii) control for any direct or indirect influences growth faltering (specifically stunting or chronic malnutrition) might have on the other child assessments. Child height and weight were measured according to the nearest 0.1 cm and 0.1 kg, respectively, following established guidelines (Cogill, 2003). Height-for-age (HAZ), weight-for-height (WHZ), and weight-for-age Z-scores (WAZ) were then calculated using the 2006 WHO growth standards (WHO Multicentre Growth Reference Study Group, 2006).
All enumerators were trained for a minimum of two weeks at each data collection time point, and all followed standardized procedures for administering each measure. Inter-rater reliability, as indicated by the correlation between scores obtained by two different testers for the same child, was estimated by having the enumerators observe and score videotaped administrations. Average inter-rater reliabilities were 0.95 (for MDAT Fine Motor at baseline and Round 2), 0.88 (for MDAT Language at baseline and Round 2), 0.94 (PPVT, baseline), and 0.96 (Triangles, Round 3).

Primary Caregiver Measures:

In addition to gathering data on household and demographic characteristics, information was gathered on the primary caregiver’s health and the home environment. At baseline and the first follow-up (18 months post-baseline, Round 2), the status of the primary guardian’s mental health functioning was assessed. At all rounds, the provision of household stimulation for learning and development and the use of positive disciplinary techniques were measured. As with the child outcomes, resulting scores were standardized. The following scales were administered to each caregiver; scales that were child specific (e.g. the Parenting Stress Index) were administered once for each child:
1. The Center for Epidemiological Studies, Depression (CESD) (Radloff, 1977), a 20-item scale that assesses depressive symptoms in adults that has been widely used throughout the world.
2. The Parenting Stress Index (PSI) (Abidin, 1990), an adapted 43-item scale that asks parents or guardians to report on their perceptions of parenting the target children in the study. Higher scores indicate more stress related to parenting this child.
3. Support for Learning and Positive Parenting (UNICEF, 2010) module was adapted from the UNICEF Multi-Indicator Cluster Surveys. Support for learning is determined by both the availability of materials (books, toys etc.) that promote development, as well as activities adults do with children to encourage learning. Typical behavior control strategies or disciplinary techniques were also measured.

CBCC Measures:

Extensive information on the characteristics of the CBCC, staff and quality of staff-child interactions was gathered at baseline and both follow-up rounds of data collection. The CBCC questionnaire and observation measure were adapted from the La Escala de Evaluación de la Calidad Educativa de Centros de Educación Preescolares (ECCP) (Martínez et al., 2004) from Mexico and a preschool quality tool developed for use in Cambodia (Rao et al., 2012). Ministry personnel and Malawian child development experts suggested additional items. Items included information on CBCC building structure and space, availability of learning materials, provision of meals, typical daily schedule of activities, number of children enrolled, and availability of water and toilets. The majority of survey questions were administered to the CBCC director, but the preschool teacher(s) also provided some information on their training, education and experience. Responses to the questionnaire were standardized with a mean of 0 and standard deviation of 1.
In addition to the CBCC survey, observations were conducted while the CBCC was operating to provide an objective account of classroom organization, activities and caregiver-child interactions. These observations are a valuable corollary to the more subjective data gathered via the questionnaire in that they provide classroom information gathered firsthand. While individual inter-rater reliability obtained during piloting was good (0.76), we learned that allowing the enumerator pairs to observe (taking brief notes) and then reach consensus together for each index was a more thorough method for capturing information about the CBCC’s overall functioning. To complete the observations, pairs of trained fieldworkers arrived unannounced just as the CBCC was opening, and observed normal center activities for one hour. The enumerators rated responses together across a variety of indices that included, for example, teachers’ style of teaching various concepts, encouragement of child participation in learning, time spent reading, time spent engaged with children (either individually or in groups), response to children’s needs, disciplinary strategies, use of small and large groups, and interactions that promote children’s social development. Scores for the observation measure were derived from a principal components analysis (PCA).
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Cluster0randomized controlled trial with one control group and three treatment arms. A sample of 199 clusters (CBCCs) containing 2,120 boys and girls aged 36-60 months.
Experimental Design Details
Randomization Method
Public Lottery
Randomization Unit
CBCC
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
199 CBCCs
Sample size: planned number of observations
2,120 children
Sample size (or number of clusters) by treatment arms
T1: 49 CBCCs
T2: 50 CBCCs
T3: 49 CBCCs
T4: 51 CBCCs
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
The minimum detectable effect size is 0.25 standard deviation improvement in primary child outcomes described above (Sample size calculations for a multi-site, cluster-randomized trial showed that the detectable difference between any two study arms for a standardized child assessment with an intra-cluster correlation of 0.1 would be approximately 0.25 standard deviations with 95% confidence and 80% power if we sampled 12 children per CBCC with 50 CBCCs allocated to each arm).
IRB

Institutional Review Boards (IRBs)

IRB Name
National Commission for Science and Technology
IRB Approval Date
2011-09-15
IRB Approval Number
RTT/2/20
IRB Name
Committee for Protection of Human Subjects, University of California, Berkeley
IRB Approval Date
2011-09-20
IRB Approval Number
2011-07-3464

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
July 30, 2013, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
March 31, 2015, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
199 CBCCs
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
2029 children
Final Sample Size (or Number of Clusters) by Treatment Arms
49, 50, 49, 51 in T1-T4, respectively
Data Publication

Data Publication

Is public data available?
Yes

Program Files

Program Files
Yes
Reports, Papers & Other Materials

Relevant Paper(s)

Abstract
We used a randomized, controlled study to evaluate a government program in Malawi, which aimed to support
child development by improving quality in community-based, informal preschools through teacher training,
financial incentives, and group-based parenting support. Children in the integrated intervention arm (teacher

training and parenting) had significantly higher scores in assessments of language and socio-emotional devel-
opment than children in preschools receiving teacher training alone at the 18-month follow-up. There were

significant improvements in classroom organization and teacher behavior at the preschools in the teacher-training
only arm, but these did not translate into improved child outcomes at 18 months. We found no effects of any
intervention on child assessments at the 36-month follow-up. Our findings suggest that, in resource-poor settings
with informal preschools, programs that integrate parenting support with preschools may be more (cost-) effective
for improving child outcomes than programs focusing simply on improving classroom quality.
Citation
Ozler et al. (2018). "Combining Pre-School Teacher Training with Parenting Education: A Cluster-Randomized Controlled Trial," Journal of Development Economics, Volume 133: 448-467.

Reports & Other Materials