Facilitating Civil Service Reform on Performance Management in Liberia
Last registered on September 12, 2018

Pre-Trial

Trial Information
General Information
Title
Facilitating Civil Service Reform on Performance Management in Liberia
RCT ID
AEARCTR-0002441
Initial registration date
September 11, 2018
Last updated
September 12, 2018 12:39 PM EDT
Location(s)

This section is unavailable to the public. Use the button below to request access to this information.

Request Information
Primary Investigator
Affiliation
World Bank
Other Primary Investigator(s)
PI Affiliation
London School of Hygiene and Tropical Medicine
PI Affiliation
Northwestern University
PI Affiliation
Harvard Business School
Additional Trial Information
Status
On going
Start date
2017-01-01
End date
2019-06-30
Secondary IDs
Abstract
In large organizations, such as governments, reforms are hard to implement and frequently slowed down or cancelled due to resistance from civil servants (Gilley et al., 2009, UNDP, 2010). Designing an effective civil service reform not only requires understanding whether the reform has the potential to improve bureaucratic performance (conditional on successful implementation), but also understanding how to maximize its adoption. In collaboration with the Government of Liberia (GoL), we propose to randomize interventions accompanying the rollout of a new performance appraisal reform that aims to develop a more effective public sector.
Our study will estimate the causal effect of the reform on workers’ performance and investigate how to increase its uptake (first stage) by randomizing the way the reform is introduced in a total of approximately 450 administrative units. They will be broken down into 3 subgroups:

• In the Control units (approximately 150 units), the reform will be rolled out following the standard procedure of the Civil Service Agency (CSA): limited information provided to unit supervisors only.

• In “Information” units (approximately 100 units), both supervisors and supervisees will receive more in-depth information and training about the reform by attending a series of information training modules on the 3 components of the reform (performance planning, mid-year review, end-of year appraisal).

• The remaining “Information & Incentives” units will receive both the information/training sessions AND monetary incentives if they successfully adopt the reform. There will be two types of monetary incentives:

o Information and supervisor-only incentives (in approximately 100 units): the supervisor of each unit will be given a prize calculated using USD30 times the number of staff in their unit, if the reform is adopted by everyone in the unit (full compliance).

o Information and all-employees incentives (in approximately 100 units): a prize of USD30 will be given to each employee/supervisor if the reform is adopted by everyone in their unit (i.e., the incentives are shared equally across all staff, including but not limited to the supervisor).
External Link(s)
Registration Citation
Citation
Deserranno, Erika et al. 2018. "Facilitating Civil Service Reform on Performance Management in Liberia." AEA RCT Registry. September 12. https://www.socialscienceregistry.org/trials/2441/history/34179
Experimental Details
Interventions
Intervention(s)
Intervention Start Date
2017-07-01
Intervention End Date
2019-02-28
Primary Outcomes
Primary Outcomes (end points)
We anticipate to collect the following outcomes.

a. Outcome Variables Measuring the Adoption of the Appraisal Process: measured at unit level.
• Appraisal forms received: Indicates whether CSA received evidence of appraisal taking place/appraisal forms and extent to which they fulfill requirements
• Appraisal score distribution: Indicates use of full distribution of scores across civil servants
• Appraisal awareness amongst civil servants: Extent to which civil servants are aware of appraisal process and accuracy of their perceptions of its functions
• Perceived effectiveness of appraisal policy: Extent to which civil servants perceive appraisal process to have positive impacts on productivity/be fair assessment of reality
• Appraisal consistency with independent survey measures: Extent to which survey-defined assessments of colleagues (360-degree evaluations of performance) match appraisal assessments

b. Outcome Variables Measuring Bureaucratic Performance: measured at both civil servant and Unit level
• Survey reports of improved unit functioning: Extent to which civil servants report improved team communication, improved unit problem-solving and reduced bottlenecks to productivity
• Changes in performance scores from PMS: Extent to which performance scores allocated to civil servants by their supervisors in 2017 versus 2018 capture changes in civil servants’ work-performance and how the average total performance score across staff in a unit capture unit-level performance
• Productivity (self-reported): Self-reported measures of bureaucratic performance by respondents to civil servants survey through 360 performance review by respondent of themselves and their colleagues
• Productivity (mystery bureaucrat) Extent to which unit responds to requests from a random/‘mystery’ CSA Civil Servant
• Productivity (co-respondent): Assessment of partnering/interaction with other units and of unit-level performance as viewed by the civil servants survey respondent
• Productivity (independent audit): Audit of unit’s activities by an ex-civil servant

c. Motivation: measured at civil servant level
• Perry test of public service motivation: Repeated application of the Perry test of public sector motivation (similar tests may be included)
• Bureaucratic satisfaction: Reports by civil servants of their satisfaction in the workplace
• Locus of control: Extent to which civil servants believe they have control over their productivity
Primary Outcomes (explanation)
See above
Secondary Outcomes
Secondary Outcomes (end points)
a. Bureaucratic relationships: measured at civil servant pairs or civil servant level
• Manager-employee relationship: Extent to which civil servants report improved manager-employee relationships
• Civil servant - civil servant relationships: Extent to which civil servants report improved relationships with civil service colleagues
• Social capital/trust: Extent to which civil servants feel they are part of a unit with colleagues they bond with and trust

b. Spillovers:
• Exposure to treatment of control units: Extent to which individual officers in control group were exposed to interventions
• Spillovers: Spillover effects of the interventions on control units interacting with treatment units

c. Perception of CSA:
• Civil servant perception of CSA: Extent to which civil servants report positive views of CSA and its reform processes
Secondary Outcomes (explanation)
See above
Experimental Design
Experimental Design
Our sample of around 450 administrative units will be randomly assigned to one in three groups:

(1). Pure control group: In the ‘Control units’ (approximately 150 administrative units), we will roll out the reform following the standard CSA procedure (limited information provided to supervisors only)

(2). Pure information training group: ‘Information units’ (approximately 100 units) will receive more in‐depth information about the reform by attending three information trainings for both supervisors and supervisees

(3). Information Training & Incentives Units: the remaining units will receive monetary incentives for implementing the reform on top of the information/training sessions. They will further be randomly split into;
(i) the ‘Information and supervisor-only incentive’ subgroup (approximately 100 units), and
(ii) ‘Information and all-employees incentive’ subgroup (approximately 100 units) that are offered different types of monetary incentives.
Experimental Design Details
Not available
Randomization Method
Our randomization will be done at the unit level using Stata. We will undertake the randomization based on the census of units and staff already recorded.
Randomization Unit
Unit (bureaucratic divisions) level randomization.
Was the treatment clustered?
Yes
Experiment Characteristics
Sample size: planned number of clusters
There are about 450 units in total, which will be split between treatment and control.
Sample size: planned number of observations
Target enrollment: around 3500 civil servants, approximately 450 units. These numbers are preliminary and may be subject to change.
Sample size (or number of clusters) by treatment arms
(1). Pure control group: Approximately 150 units (ca. 1/3 of all administrative units).
(2). Pure information group: Approximately 100 Informational units.
(3). Information & Incentives Units: the remaining ~200 units. They will further be evenly split into two sub-groups offering different types of incentives:
o Information and supervisor incentive: ~ 1/2 of the remaining units.
o Information and all employees incentive: ~ 1/2 of the remaining units.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
Harvard Human Research Protection Program
IRB Approval Date
2018-04-30
IRB Approval Number
Submission Number: CR17-0116-01
IRB Name
Harvard Human Research Protection Program
IRB Approval Date
2017-09-15
IRB Approval Number
Submission Number: MOD17-0116-02
IRB Name
Harvard Human Research Protection Program
IRB Approval Date
2017-11-17
IRB Approval Number
Submission Number: MOD17-0116-01
IRB Name
University of Liberia-Pacific Institute for Research & Evaluation Institutional Review Board (UL-PIRE IRB)
IRB Approval Date
2017-05-22
IRB Approval Number
Protocol Number: 16-10-016
IRB Name
University of Liberia-Pacific Institute for Research & Evaluation Institutional Review Board (UL-PIRE IRB)
IRB Approval Date
2018-06-05
IRB Approval Number
Protocol Number: 16-10-016
IRB Name
Harvard Human Research Protection Program
IRB Approval Date
2017-04-04
IRB Approval Number
Protocol Number: IRB17-0116