x

We are happy to announce that all trial registrations will now be issued DOIs (digital object identifiers). For more information, see here.
Understanding the Impacts of a Civic Participation Initiative in Rwanda: Experimental Findings
Last registered on April 15, 2014

Pre-Trial

Trial Information
General Information
Title
Understanding the Impacts of a Civic Participation Initiative in Rwanda: Experimental Findings
RCT ID
AEARCTR-0000352
Initial registration date
April 15, 2014
Last updated
April 15, 2014 11:47 AM EDT
Location(s)
Region
Primary Investigator
Affiliation
Mathematica Policy Research
Other Primary Investigator(s)
Additional Trial Information
Status
Completed
Start date
2010-07-01
End date
2013-11-30
Secondary IDs
Abstract
The Millennium Challenge Corporation sponsored the Rwanda Threshold Program (RTP) to help the Government of Rwanda improve its performance on governance indicators related to citizens’ political rights and civil liberties. Mathematica Policy Research conducted an evaluation of the RTP’s Strengthening Civic Participation (SCP) component, an initiative with two focus areas: (1) supporting the efforts of civil society organizations to advocate for local issues and (2) training local government officials to increase responsiveness to the concerns and priorities of citizens. The evaluation uses a stratified random assignment design, whereby districts within each of Rwanda’s five provinces were paired based on district population and economic characteristics and then randomly assigned to either a treatment or a control group. We designed a household survey to collect data for evaluating SCP impacts, and administered this survey at baseline before the program (in early 2011) and one year after the start of the program (2012). We drew nationally representative samples of approximately 10,000 households in each data collection round. The baseline and follow-up surveys collected data on respondents’ civic participation levels, including awareness and perceptions of local government performance, responsiveness, and accountability. Specifically, respondents were asked about their awareness of local government meetings, familiarity with local government officials, perceived citizen influence, and knowledge and access to information about local government affairs. Respondents were also asked about their overall satisfaction with government services related to water infrastructure, local road conditions, waste collection, public schools, and health clinics. We use this household survey data to analyze program impacts.
External Link(s)
Registration Citation
Citation
Nichols-Barrer, Ira. 2014. "Understanding the Impacts of a Civic Participation Initiative in Rwanda: Experimental Findings." AEA RCT Registry. April 15. https://doi.org/10.1257/rct.352-2.0.
Former Citation
Nichols-Barrer, Ira. 2014. "Understanding the Impacts of a Civic Participation Initiative in Rwanda: Experimental Findings." AEA RCT Registry. April 15. https://www.socialscienceregistry.org/trials/352/history/1517.
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details
Interventions
Intervention(s)
The Strengthening Civic Participation (SCP) program was an initiative with two focus areas: (1) supporting the efforts of civil society organizations to advocate for local issues through grant support and technical assistance and (2) supporting the efforts of local government officials to increase responsiveness to the concerns and priorities of citizens through grant support and technical assistance.
Intervention Start Date
2011-03-01
Intervention End Date
2011-12-31
Primary Outcomes
Primary Outcomes (end points)
The baseline and follow-up surveys collected data on respondents’ civic participation levels, including awareness and perceptions of local government performance, responsiveness, and accountability. Specifically, respondents were asked about their awareness of local government meetings, familiarity with local government officials, perceived citizen influence, and knowledge and access to information about local government affairs. Respondents were also asked about their overall satisfaction with government services related to water infrastructure, local road conditions, waste collection, public schools, and health clinics.
Primary Outcomes (explanation)
We estimated the impacts of the Strengthening Civic Participation program for six outcome indices: awareness of local government meetings, familiarity with local government officials, knowledge about local government affairs, access to district government information, citizen influence on government, and satisfaction with local services. We constructed these outcome indices by grouping together survey questions related to the same underlying outcome using factor analysis. We did this for several reasons. First, estimating the impacts by comparing the treatment and the control groups on several survey questions is likely to result in one or more statistically significant impacts by chance when there is actually no impact. In other words, we are more likely to incorrectly reject the null hypothesis, when considering a series of hypothesis tests—a problem commonly known as the multiple comparison problem (Benjamini and Hochberg, 1995). Second, since factor analysis assumes that the observed variables are influenced by a few underlying variables or factors that are unobserved, constructing those underlying latent variables (the unobserved factors) can provide useful information about processes or behavior of the population of interest. Third, grouping survey questions into a few outcome indices helps in examining impacts in a tractable manner.

For each of the six outcome domains, we conducted factor analysis using the survey questions or observed variables that are most relevant for that domain in three steps:

(1) We used the principal-component factor method to obtain the factor solutions. For each outcome domain, we found that only one underlying factor explained the variation in the responses to the included survey questions.

(2) We then used orthogonal rotation to rotate the factor loadings and estimated factor scores using the regression method, which estimates a factor as a weighted sum of the included observed variables.

(3) Finally, we converted each of the six estimated factors to binary variables to interpret the impact estimates better. In particular, if a survey respondent’s factor score was above the mean score for the full survey sample in that year, the binary variable was coded as 1, otherwise it was coded as 0. Thus, the impact estimates compare the percentage of citizens with an above-average factor score in the treatment districts with the percentage in control districts.
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
The activities under the Strengthening Civic Participation component of the RTP were planned to be implemented in two separate, year-long phases, with 15 districts receiving activities in each phase. We took advantage of this plan by randomly dividing Rwanda’s 30 districts to either phase I or phase II. To do so, districts within each province were first matched in pairs or groups of three to ensure the best possible match between the two district groups. The districts were matched on the following five district characteristics:

• Population change between 2002 and 2006
• Population density
• Common Development Fund (CDF) appropriation amounts for FY 2008 (as a proxy for poverty levels)
• Share of district spending obtained through local revenues in FY 2008
• District expenditure per capita on good governance and social affairs

These matched district pairs or triplets were then used as randomization blocks. Specifically, after these blocks were created a public lottery was conducted in the presence of district officials to randomly assign districts within each block to the treatment group (districts that would receive program activities in phase I in 2011), or the control group (districts that were to receive program activities in phase II in 2012). The 15 control districts never received any program activities because the RTP was not extended beyond 2011.
Experimental Design Details
Randomization Method
Public lottery.
Randomization Unit
Clustered random assignment occurred at the district level.
Was the treatment clustered?
Yes
Experiment Characteristics
Sample size: planned number of clusters
30
Sample size: planned number of observations
10,000
Sample size (or number of clusters) by treatment arms
15 districts control, 15 districts treatment
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
National Institute of Statistics Rwanda
IRB Approval Date
2010-12-16
IRB Approval Number
967/2010/10/NSR
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
Yes
Intervention Completion Date
December 31, 2011, 12:00 AM +00:00
Is data collection complete?
Yes
Data Collection Completion Date
November 30, 2013, 12:00 AM +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
30
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
10,000
Final Sample Size (or Number of Clusters) by Treatment Arms
15 treatment districts, 15 control districts
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports and Papers
Preliminary Reports
Relevant Papers