Understanding the Impacts of a Civic Participation Initiative in Rwanda: Experimental Findings

Last registered on November 01, 2023

Pre-Trial

Trial Information

General Information

Title
Understanding the Impacts of a Civic Participation Initiative in Rwanda: Experimental Findings
RCT ID
AEARCTR-0000352
Initial registration date
April 15, 2014

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
April 15, 2014, 11:45 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
November 01, 2023, 10:39 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
Mathematica Policy Research

Other Primary Investigator(s)

Additional Trial Information

Status
Completed
Start date
2010-07-01
End date
2013-11-30
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
The Millennium Challenge Corporation sponsored the Rwanda Threshold Program (RTP) to help the Government of Rwanda improve its performance on governance indicators related to citizens’ political rights and civil liberties. Mathematica Policy Research conducted an evaluation of the RTP’s Strengthening Civic Participation (SCP) component, an initiative with two focus areas: (1) supporting the efforts of civil society organizations to advocate for local issues and (2) training local government officials to increase responsiveness to the concerns and priorities of citizens. The evaluation uses a stratified random assignment design, whereby districts within each of Rwanda’s five provinces were paired based on district population and economic characteristics and then randomly assigned to either a treatment or a control group. We designed a household survey to collect data for evaluating SCP impacts, and administered this survey at baseline before the program (in early 2011) and one year after the start of the program (2012). We drew nationally representative samples of approximately 10,000 households in each data collection round. The baseline and follow-up surveys collected data on respondents’ civic participation levels, including awareness and perceptions of local government performance, responsiveness, and accountability. Specifically, respondents were asked about their awareness of local government meetings, familiarity with local government officials, perceived citizen influence, and knowledge and access to information about local government affairs. Respondents were also asked about their overall satisfaction with government services related to water infrastructure, local road conditions, waste collection, public schools, and health clinics. We use this household survey data to analyze program impacts.
External Link(s)

Registration Citation

Citation
Nichols-Barrer, Ira. 2023. "Understanding the Impacts of a Civic Participation Initiative in Rwanda: Experimental Findings." AEA RCT Registry. November 01. https://doi.org/10.1257/rct.352-3.0
Former Citation
Nichols-Barrer, Ira. 2023. "Understanding the Impacts of a Civic Participation Initiative in Rwanda: Experimental Findings." AEA RCT Registry. November 01. https://www.socialscienceregistry.org/trials/352/history/198960
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
The Strengthening Civic Participation (SCP) program was an initiative with two focus areas: (1) supporting the efforts of civil society organizations to advocate for local issues through grant support and technical assistance and (2) supporting the efforts of local government officials to increase responsiveness to the concerns and priorities of citizens through grant support and technical assistance.
Intervention Start Date
2011-03-01
Intervention End Date
2011-12-31

Primary Outcomes

Primary Outcomes (end points)
The baseline and follow-up surveys collected data on respondents’ civic participation levels, including awareness and perceptions of local government performance, responsiveness, and accountability. Specifically, respondents were asked about their awareness of local government meetings, familiarity with local government officials, perceived citizen influence, and knowledge and access to information about local government affairs. Respondents were also asked about their overall satisfaction with government services related to water infrastructure, local road conditions, waste collection, public schools, and health clinics.
Primary Outcomes (explanation)
We estimated the impacts of the Strengthening Civic Participation program for six outcome indices: awareness of local government meetings, familiarity with local government officials, knowledge about local government affairs, access to district government information, citizen influence on government, and satisfaction with local services. We constructed these outcome indices by grouping together survey questions related to the same underlying outcome using factor analysis. We did this for several reasons. First, estimating the impacts by comparing the treatment and the control groups on several survey questions is likely to result in one or more statistically significant impacts by chance when there is actually no impact. In other words, we are more likely to incorrectly reject the null hypothesis, when considering a series of hypothesis tests—a problem commonly known as the multiple comparison problem (Benjamini and Hochberg, 1995). Second, since factor analysis assumes that the observed variables are influenced by a few underlying variables or factors that are unobserved, constructing those underlying latent variables (the unobserved factors) can provide useful information about processes or behavior of the population of interest. Third, grouping survey questions into a few outcome indices helps in examining impacts in a tractable manner.

For each of the six outcome domains, we conducted factor analysis using the survey questions or observed variables that are most relevant for that domain in three steps:

(1) We used the principal-component factor method to obtain the factor solutions. For each outcome domain, we found that only one underlying factor explained the variation in the responses to the included survey questions.

(2) We then used orthogonal rotation to rotate the factor loadings and estimated factor scores using the regression method, which estimates a factor as a weighted sum of the included observed variables.

(3) Finally, we converted each of the six estimated factors to binary variables to interpret the impact estimates better. In particular, if a survey respondent’s factor score was above the mean score for the full survey sample in that year, the binary variable was coded as 1, otherwise it was coded as 0. Thus, the impact estimates compare the percentage of citizens with an above-average factor score in the treatment districts with the percentage in control districts.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The activities under the Strengthening Civic Participation component of the RTP were planned to be implemented in two separate, year-long phases, with 15 districts receiving activities in each phase. We took advantage of this plan by randomly dividing Rwanda’s 30 districts to either phase I or phase II. To do so, districts within each province were first matched in pairs or groups of three to ensure the best possible match between the two district groups. The districts were matched on the following five district characteristics:

• Population change between 2002 and 2006
• Population density
• Common Development Fund (CDF) appropriation amounts for FY 2008 (as a proxy for poverty levels)
• Share of district spending obtained through local revenues in FY 2008
• District expenditure per capita on good governance and social affairs

These matched district pairs or triplets were then used as randomization blocks. Specifically, after these blocks were created a public lottery was conducted in the presence of district officials to randomly assign districts within each block to the treatment group (districts that would receive program activities in phase I in 2011), or the control group (districts that were to receive program activities in phase II in 2012). The 15 control districts never received any program activities because the RTP was not extended beyond 2011.
Experimental Design Details
Randomization Method
Public lottery.
Randomization Unit
Clustered random assignment occurred at the district level.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
30
Sample size: planned number of observations
10,000
Sample size (or number of clusters) by treatment arms
15 districts control, 15 districts treatment
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
National Institute of Statistics Rwanda
IRB Approval Date
2010-12-16
IRB Approval Number
967/2010/10/NSR

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
December 31, 2011, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
November 30, 2013, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
30
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
10,000
Final Sample Size (or Number of Clusters) by Treatment Arms
15 treatment districts, 15 control districts
Data Publication

Data Publication

Is public data available?
Yes

Program Files

Program Files
Yes
Reports, Papers & Other Materials

Relevant Paper(s)

Abstract
The three-year MCC program supported the Rwandan government’s efforts to strengthen civic participation, promote civil liberties and rights and improve the judicial system. The program provided training, technical support and grants to local and national civil society organizations and expanded citizen engagement by supporting independent community radio stations. The program reinforced Rwanda’s efforts to strengthen the judiciary and enact legislative reforms to strengthen civil liberties, human rights and civic participation. Finally, the program boosted training and technical assistance to journalists and the Rwandan National Police in an effort to further transparency and professionalism.
Citation
Mathematica. “Rwanda Threshold - Strengthening Justice and Promoting Political Rights: Final Analysis Report.” Washington, DC: Millennium Challenge Corporation, October 3, 2014. doi:https://doi.org/10.3886/9nys-5e23.

Reports & Other Materials