Understanding the Impacts of a Civic Participation Initiative in Rwanda: Experimental Findings

The Strengthening Civic Participation (SCP) program was an initiative with two focus areas: (1) supporting the efforts of civil society organizations to advocate for local issues through grant support and technical assistance and (2) supporting the efforts of local government officials to increase responsiveness to the concerns and priorities of citizens through grant support and technical assistance.

2011-03-01

2011-12-31

The baseline and follow-up surveys collected data on respondents’ civic participation levels, including awareness and perceptions of local government performance, responsiveness, and accountability. Specifically, respondents were asked about their awareness of local government meetings, familiarity with local government officials, perceived citizen influence, and knowledge and access to information about local government affairs. Respondents were also asked about their overall satisfaction with government services related to water infrastructure, local road conditions, waste collection, public schools, and health clinics.

We estimated the impacts of the Strengthening Civic Participation program for six outcome indices: awareness of local government meetings, familiarity with local government officials, knowledge about local government affairs, access to district government information, citizen influence on government, and satisfaction with local services. We constructed these outcome indices by grouping together survey questions related to the same underlying outcome using factor analysis. We did this for several reasons. First, estimating the impacts by comparing the treatment and the control groups on several survey questions is likely to result in one or more statistically significant impacts by chance when there is actually no impact. In other words, we are more likely to incorrectly reject the null hypothesis, when considering a series of hypothesis tests—a problem commonly known as the multiple comparison problem (Benjamini and Hochberg, 1995). Second, since factor analysis assumes that the observed variables are influenced by a few underlying variables or factors that are unobserved, constructing those underlying latent variables (the unobserved factors) can provide useful information about processes or behavior of the population of interest. Third, grouping survey questions into a few outcome indices helps in examining impacts in a tractable manner.

For each of the six outcome domains, we conducted factor analysis using the survey questions or observed variables that are most relevant for that domain in three steps:

(1) We used the principal-component factor method to obtain the factor solutions. For each outcome domain, we found that only one underlying factor explained the variation in the responses to the included survey questions.

(2) We then used orthogonal rotation to rotate the factor loadings and estimated factor scores using the regression method, which estimates a factor as a weighted sum of the included observed variables.

(3) Finally, we converted each of the six estimated factors to binary variables to interpret the impact estimates better. In particular, if a survey respondent’s factor score was above the mean score for the full survey sample in that year, the binary variable was coded as 1, otherwise it was coded as 0. Thus, the impact estimates compare the percentage of citizens with an above-average factor score in the treatment districts with the percentage in control districts.

The activities under the Strengthening Civic Participation component of the RTP were planned to be implemented in two separate, year-long phases, with 15 districts receiving activities in each phase. We took advantage of this plan by randomly dividing Rwanda’s 30 districts to either phase I or phase II. To do so, districts within each province were first matched in pairs or groups of three to ensure the best possible match between the two district groups. The districts were matched on the following five district characteristics:

• Population change between 2002 and 2006
• Population density
• Common Development Fund (CDF) appropriation amounts for FY 2008 (as a proxy for poverty levels)
• Share of district spending obtained through local revenues in FY 2008
• District expenditure per capita on good governance and social affairs

These matched district pairs or triplets were then used as randomization blocks. Specifically, after these blocks were created a public lottery was conducted in the presence of district officials to randomly assign districts within each block to the treatment group (districts that would receive program activities in phase I in 2011), or the control group (districts that were to receive program activities in phase II in 2012). The 15 control districts never received any program activities because the RTP was not extended beyond 2011.

Public lottery.

Clustered random assignment occurred at the district level.

Yes

30

10,000

15 districts control, 15 districts treatment

The three-year MCC program supported the Rwandan government’s efforts to strengthen civic participation, promote civil liberties and rights and improve the judicial system. The program provided training, technical support and grants to local and national civil society organizations and expanded citizen engagement by supporting independent community radio stations. The program reinforced Rwanda’s efforts to strengthen the judiciary and enact legislative reforms to strengthen civil liberties, human rights and civic participation. Finally, the program boosted training and technical assistance to journalists and the Rwandan National Police in an effort to further transparency and professionalism.

Mathematica. “Rwanda Threshold - Strengthening Justice and Promoting Political Rights: Final Analysis Report.” Washington, DC: Millennium Challenge Corporation, October 3, 2014. doi:https://doi.org/10.3886/9nys-5e23.

https://mcc.icpsr.umich.edu/evaluations/index.php/catalog/1971/versions/V1.0.1