Community advocacy forums and public service delivery - Impact, and the role of information, deliberation, and administrative placement

We tried to stay as close as possible to the baraza intervention as designed and implemented by the government. We used this baraza as a starting point and, either removed the information component or the deliberation component from the generic sub-county level baraza to test the relative importance of these components. However, to standardize the treatments, we developed detailed scripts that RDCs and facilitators were expected to follow. Furthermore, manuals for RDCs and facilitators were developed, and all were invited for a training. We will summarize the main differences between an information baraza and a deliberation baraza. Detailed information can be found in https://github.com/bjvca/baraza/blob/master/report/appendices/Training%20Manual%20for%20Baraza.pdf.

For an information baraza, templates to gather information were developed to be filled by officials and mounted at a central location in each parish of the district two weeks before the baraza. The template was designed to inform citizens about planned and actual public expenditures for the previous fiscal year, about achievements and challenges encountered during that year, and about planned expenditures and targets for the next fiscal year. This needs to be filled for each of the four sectors (agriculture, infrastructure, health and education) by the sub-county chief.While the preparation and distribution of these posters supposed to by the responsibility of the RDC, we had research assistants that closely monitored the implementation of this. We also assisted in printing of the templates. On the day of the baraza, the CAO provides a brief presentation on overall budget/finances for the fiscal year, main achievements and challenges in service delivery, and introduces local officials. After a brief intervention by the OPM, local officials responsible for each sector then present more or less the same as what was required for the templates. An information focused baraza allowed for only 10 clarifying questions to be asked, to be collected and asked by the facilitator.

For the deliberation barazas, posters are also mounted in each parish of the sub-county, but only to announce that a baraza will be held at a particular date and place. At the baraza itself, after a brief introduction by the RDC, citizens are guided to break into 5 groups by sector, discuss problems they face and draw up a list of priority issues that need to be addressed. Facilitators in each group are required to anonymously collect these issues and concerns. Facilitators are expected to focus the discussion on what was done well, and what were the problems during the past year. The discussions should also result in agreement on what should be done in the next fiscal year. After the break-out sessions, officials are asked to react to the specific comments and requests.

District level baraza were very similar to sub-county level barazas, except for the fact that district level barazas are organized at the district headquarters and all sub-county chiefs and LC3's of each sub-county are expected to attend in case questions arise related to their sub-county.

2016-07-01

2019-06-01

We assess the impact of the baraza intervention on a range of outcomes related to agriculture, health, education, and infrastructure. For each sector, key outcomes are definded, and combined into sector level indices, and in a single index following Anderson (2008). For agriculture, a first outcome looks at access to extension at home. In particular, we estimate the percentage of households in our sample who report that they were visited by an expert (e.g. crop or livestock extension agent, or community based facilitator or another experienced farmer) at the home in the last 12 months (the variable named “baraza.B2” in the end-line questionnaire). Second, we consider the proportion that reports visiting extension offices, demonstration sites or model (baraza.B3 or baraza.B3.3). A third outcome variable is the presence of NAADS/OWC supported farmer groups is also a useful indicator of agricultural service delivery in Uganda (baraza.B4.1). Further down impact pathway is actual change in the use of modern inputs by farmers in areas where agricultural related services are improved as a consequence of the baraza interventions. For instance, we also estimate the proportion of households in our sample that report to have used inorganic fertilizers (DAP, Urea, NPK, Foliar, TSP, SSP, MOP) or improved seed in the last 12 months (baraza.B1 or baraza.B1.5). We also include two outcomes that look at public services related to crop marketing. First, we estimate the proportion of households in our sample that report they received help in marketing their produce from the village procurement committee/village farmers forum in the last 12 months. (baraza.B5.2) Second, we ask a similar question to assess the proportion of households in our sample that report they received help in marketing their produce from a cooperative or association in the last 12 months. (baraza.B5.3).
For Infrastructure, a first outcome we consider is whether the household uses unprotected water source during dry season (yes/no). This is measured as the share of households that report that the main source of drinking water during the dry season is rain water, surface water, water obtained from a tube well or borehole, an unprotected dug well or and unprotected spring. (baraza.C1). Next, we look at the distance to the primary water source (baraza.C1.2) and waiting time at the water source (baraza.C1.3), both during the dry season. We also ask if there is a water user committee in the village (baraza.C2.3). We include one question related to road infrastructure. We ask how far the household is located from the nearest all weather road (in km; baraza.A6).
For the health sector, the first two outcomes we consider attempt to assess changes in access or use of public health facilities. A first indicator measures the use of public health facilities for illness. In particular, we construct an indicator that is true if the household head responds that treatment would be sought in a health center 2, 3, 4 or in a regional referral hospital if a member of your household had fever (baraza.D2). A similar indicator attempts to assess the use of the public health system for maternal health care, and asks if treatment would be sought in a health center 2, 3, 4 or in a regional referral hospital if a member of your household was to give birth (baraza.D2.4). Next, we ask if a Village Health Team is present in the village (baraza.D3). We also consider distance to the nearest government health facility, measured in km (baraza.D4.2). We then ask whether any household members were unable to work or go to school due to an illness in the past one year (baraza.D1). We then ask how long did you have to wait before being attended (in min) (baraza.D4.6). Finally, we ask if a traditional health practitioner was visited in the last year (baraza.D6).
For the education sector, impact is assessed by the following outcomes: the number of children within the households that attend public school (either Universal Primary Education (UPE; baraza.E1.2) or Universal Secondary Education (USE; baraza.E2.1)); distance to primary or secondary school (or the average if both are reported; baraza.E1.2 and baraza.E2.2); whether the primary or secondary school attended by any of their children has a complete boundary fence (baraza.E1.4 and baraza.E1.4); and whether there is a water source available in the school (baraza.E1.6 and baraza.E2.6). We also look at how the school is managed, and how stakeholders are involved. For instance, we look at whether the school has a School Management Committee (SMC) (baraza.E1.10 for primary schools, baraza.E2.10 for secondary schools) and consider the percentage of households that are informed about SMC meetings (baraza.E1.13 and baraza.E2.13). Finally, we ask households if an inspector had visited the school in the year before the survey (baraza.E1.18 and baraza.E2.18).

For continuous variables, 5 percent trimmed values will be use (2.5 percent trimming at each side of the distribution). Inverse hyperbolic sine (IHS) transformations will be used if skewness exceeds 1.96. Trimming will always be done on end results. For instance, if the outcome is yield at the plot level, then production will first be divided by plot area, after which the IHS transformation is done, and the end result is trimmed. Outcomes for which 95 percent of observations have the same value within the relevant sample will be omitted from the analysis to limit noise caused by variables with minimal variation.

Secondary outcomes, which may be used to study impact pathways include: When was the last time they spoke to a particular government official (various officials). Did you participate in elections (different levels). Did you ever contribute in cash or kind (various sectors and services).

We use a cluster randomized control trial to answer 4 research questions: (1) assess the impact of the baraza intervention, (2) estimate the relative importance of the information component of a baraza, (3) estimate the relative importance of the deliberation component of the baraza, and (4) compare effectiveness of district level to sub-county level barazas. For the last question, a parallel design is used with randomization at the district level. The first 3 questions are answered using a factorial design nested in one of the two treatment arms of the district level intervention; randomization in this factorial design happens at the district level.

randomization done in office by a computer,

Randomization units are districts and sub-counties in Uganda.This study proposed a nested, or two-step, randomization design. In a first step, we randomly allocate eligible districts to treatment and control conditions. In particular, some of the eligible districts start receiving district level barazas that contain both the information component and the deliberation component, while other districts do not receive a baraza at this level. In a second step, we proceed with all eligible sub-counties and randomly allocate each sub-county to one of four conditions in a 2 by 2 factorial design. In particular, about one quarter of all eligible sub-counties sampled from the untreated districts will serve as pure control and will not receive any baraza at any level. About one quarter will receive a sub-county level baraza that combines both information and deliberation treatment (crossed sub-county level treatment). A third quarter will receive a sub-county level baraza that consists largely of officials providing information and limited opportunity for citizens to engage. A final quarter will receive a sub-county level baraza with a focus on citizens engaging with each other and with officials, without upfront information provision. We also take a random sample of sub-counties from the districts that received the district level baraza; these sub-counties do not receive any further baraza at the district level. Within each sub-county, we sample a fixed number of households.

Yes

230

11,500

51 sub-counties that receive no barazas, 51 sub-counties that receive information baraza, 51 sub-counties that receive deliberation baraza, 53 sub-counties that receive crossed (information+deliberation) baraza; 24 sub-counties that receive baraza at the district level

The following outcomes were used in determining sample size: Weight for age z-scores - mde: 0.2, sd 1.2; Number of days unable to work as percentage of days sick -mde: 10 percentage points, sd 39.72; average number of years children go to school - mde: 1 additional year, sd 3.5; percentage of children in household attending school - mde: 5 percentage points, sd 27.72; HH visited by extension worker - mde: 5, sd 26.2 percentage points; maize yields - mde: 500kg/hectare, sd: 1998

pre-registered report

To improve public service delivery, the Government of Uganda organizes community forums-popularly known as barazas-where citizens receive information from government officials, and get the opportunity to directly engage with them. We run a cluster randomized control trial to assess the impact of this policy intervention on public service delivery in agriculture, health, education, and infrastructure. Using a factorial design, we further test the relative importance of the two main components of the intervention-information provision and citizen engagement. we also compare the effectiveness of barazas organized at the district level to the effectiveness of barazas organized at the sub-county level. Using a strictly pre-registered confirmatory analysis, we find no impact of the intervention on general public service delivery, but there are some indications that sub-county level barazas increase outcomes in the agricultural sector. A more exploratory part that looks at individual outcomes, potential mechanisms, and heterogeneous treatment effects suggests localized impacts of barazas in the areas of agricultural extension services and agricultural input distribution, access to drinking water, and school enrolment and infrastructure.

Kabunga, Nassul Ssentamu; Miehe, Caroline; Mogues, Tewodaj; and Van Campenhout, Bjorn. 2020. Community based monitoring and public service delivery: Impact, and the role of information, deliberation, and jurisdictional tier. IFPRI Discussion Paper 1933. Washington, DC: International Food Policy Research Institute (IFPRI). https://doi.org/10.2499/p15738coll2.133751

https://www.ifpri.org/publication/community-based-monitoring-and-public-service-delivery-impact-and-role-information