School Based Programming Lean Impact Evaluation - Burundi

In Burundi, the World Food Programme (WFP), in partnership with the Government of Burundi, provides daily nutritious meals and snacks to over 660,000 school children (331572 girls and 331570 boys) in May 2022 in 873 schools through its Home-Grown School Feeding programme (HGSF). Currently, Burundi WFP Country Office’s (CO) school feeding model is based on a centralised procurement model, where WFP procures food mostly internationally and delivers it to schools. The meals comprise a combination of imported and local food such as cereals, beans and peas and parents contribute to food preparation on a rotational basis.



Starting in 2022, the Burundi CO is piloting a new decentralised school feeding procurement modality based on cash transfers to schools. Under this new Commodity Voucher (CV) procurement model the WFP will make a transfer to the Direction Provinciale de l'Education (DPE), which will purchase from local cooperatives, and cooperatives will deliver food directly to schools.

This lean impact evaluation (IE) will use a randomised controlled trial (RCT) design to assess whether the CV procurement model impacts the performance of meal distribution by schools (e.g., quantity, diversity, quality of meals). It will also compare quantities of food procured across the two treatment models and provide descriptive evidence of the efficiency and reliability of the new procurement system.



The lean IE will also include a pilot survey with cooperatives and farmers to measure productivity, revenues, and profits from farming, and a pilot survey with school children to measure individual diet diversity, meal satisfaction, school performance and cognitive ability. However, the cooperative outcomes are not evaluated experimentally because the scale of the programme and the number of available cooperatives is small. Instead, the results from the pilot survey will inform the feasibility and design of a larger scale IE with the aim to assess the impact of HGSF on farmers' income and children's learning, health, and nutritional outcomes during scale-up.

2022-11-21

2023-06-30

Service delivery; Food procurement; App experience (school connect);

Service delivery (food) examines the outcomes related to the quantity, the meal diversity and the quality of school meals.

Food procurement: CV procurement model will lead to increased school meals distributed to children. This will be explored in terms of school feeding days and quantities of food distributed in school meals.

Sales and local economy Qualitative interviews of cooperatives, NGOs, and governments will be conducted to supplement quantitative analysis. This exercise aims to understand their perceptions and experience of the CV model to document any operational challenges in the process of local procurement; including but not limited to logistics, storage, quality of produce, payments; and solutions from different stakeholders involved in the project.

The design is a randomization at the school level of the 95 schools uniquely mapped to 12 cooperatives provided by the WFP Burundi Country Office in the study. All pilot schools were in three provinces of Burundi (Bujumbura rural, Bubanza and Muyinga).

Of the 95 schools, the lean IE uses stratified randomisation where cooperatives are used as strata and 50 schools were then randomly assigned to the treatment group and 45 to control. This corresponds to 23 treatment schools in Muyinga, 3 treatments schools in Bujumbura Rural and 24 treatment schools in Bubanza.

The sampling and randomization were conducted using Stata software.

School

Yes

17 strata (provinces and cooperatives)

95 schools are sampled from the three provinces.

Treatment: 50 schools mapped to 12 cooperatives in the three provinces

Control: 45 schools receiving meals from the centralized model with WFP CO.

Power calculation is conducted using two sources of administrative data: COMET (Country Office Monitoring and Evaluation Tool) and School Connect platform. It focusses on the following key outcomes and indicators: School meal delivery: the number of feeding days Food procurement: the amount of beans and corn-soy blends delivered to schools Children’s education: student enrolment and attendance. The MDE of this IE considers 80% power and a significance level of 5%. On average, schools distributed meals to students 7.28 days per month in 2021-2022. The IE design can detect an increase of 3.75 days, and it is believed that the new CV model has the potential to generate differences as big as 3.75 days per month as while this would be 50% of the control mean, this would be 17% out of the 21 school days in a month. WFP delivered 443 kgs of beans and 1252 kgs of corn-soy blends per school term during 2020-2022. The IE can detect an increase of 208kgs for beans and 287kgs for corn-soy blends (respectively 48% and 23% of the control mean). While these are significant changes, it is believed that the new CV model should be able to support such changes due to the current import ban on cereals. Finally, on average schools had 749 students enrolled in each school in 2021-2022 and the attendance rate during this period was approximately 80% (i.e., 591 students on average). The study design can detect an average increase of 157 students enrolled per school and 24 percentage points difference for attendance rate (i.e 183 students). The MDE for attendance is bigger than that of enrolment because attendance data has a higher standard deviation.