Facilitating Civil Service Reform on Performance Management in Liberia

Last registered on June 17, 2019

Pre-Trial

Trial Information

General Information

Title
Facilitating Civil Service Reform on Performance Management in Liberia
RCT ID
AEARCTR-0002441
Initial registration date
September 11, 2018

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
September 12, 2018, 12:39 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
June 17, 2019, 3:01 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
World Bank

Other Primary Investigator(s)

PI Affiliation
London School of Hygiene and Tropical Medicine
PI Affiliation
Northwestern University
PI Affiliation
Harvard Business School

Additional Trial Information

Status
On going
Start date
2017-01-01
End date
2020-06-30
Secondary IDs
Abstract
In large organizations, such as governments, reforms are hard to implement and frequently slowed down or cancelled due to resistance from civil servants (Gilley et al., 2009, UNDP, 2010). Designing an effective civil service reform not only requires understanding whether the reform has the potential to improve bureaucratic performance (conditional on successful implementation), but also understanding how to maximize its adoption. In collaboration with the Government of Liberia (GoL), we propose to randomize interventions accompanying the rollout of a new performance appraisal reform that aims to develop a more effective public sector.
Our study will estimate the causal effect of the reform on workers’ performance and investigate how to increase its uptake (first stage) by randomizing the way the reform is introduced in a total of approximately 450 administrative units. They will be broken down into 3 subgroups:

• In the Control units (approximately 150 units), the reform will be rolled out following the standard procedure of the Civil Service Agency (CSA): limited information provided to unit supervisors only.

• In “Information” units (approximately 100 units), both supervisors and supervisees will receive more in-depth information and training about the reform by attending a series of information training modules on the 3 components of the reform (performance planning, mid-year review, end-of year appraisal).

• The remaining “Information & Incentives” units will receive both the information/training sessions AND monetary incentives if they successfully adopt the reform. There will be two types of monetary incentives:

o Information and supervisor-only incentives (in approximately 100 units): the supervisor of each unit will be given a prize calculated using USD30 times the number of staff in their unit, if the reform is adopted by everyone in the unit (full compliance).

o Information and all-employees incentives (in approximately 100 units): a prize of USD30 will be given to each employee/supervisor if the reform is adopted by everyone in their unit (i.e., the incentives are shared equally across all staff, including but not limited to the supervisor).
External Link(s)

Registration Citation

Citation
Deserranno, Erika et al. 2019. "Facilitating Civil Service Reform on Performance Management in Liberia." AEA RCT Registry. June 17. https://doi.org/10.1257/rct.2441-2.0
Former Citation
Deserranno, Erika et al. 2019. "Facilitating Civil Service Reform on Performance Management in Liberia." AEA RCT Registry. June 17. https://www.socialscienceregistry.org/trials/2441/history/48216
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2017-07-01
Intervention End Date
2019-02-28

Primary Outcomes

Primary Outcomes (end points)
We anticipate to collect the following outcomes.

a. Outcome Variables Measuring the Adoption of the Appraisal Process: measured at unit level.
• Appraisal forms received: Indicates whether CSA received evidence of appraisal taking place/appraisal forms and extent to which they fulfill requirements
• Appraisal score distribution: Indicates use of full distribution of scores across civil servants
• Appraisal awareness amongst civil servants: Extent to which civil servants are aware of appraisal process and accuracy of their perceptions of its functions
• Perceived effectiveness of appraisal policy: Extent to which civil servants perceive appraisal process to have positive impacts on productivity/be fair assessment of reality
• Appraisal consistency with independent survey measures: Extent to which survey-defined assessments of colleagues (360-degree evaluations of performance) match appraisal assessments

b. Outcome Variables Measuring Bureaucratic Performance: measured at both civil servant and Unit level
• Survey reports of improved unit functioning: Extent to which civil servants report improved team communication, improved unit problem-solving and reduced bottlenecks to productivity
• Changes in performance scores from PMS: Extent to which performance scores allocated to civil servants by their supervisors in 2017 versus 2018 capture changes in civil servants’ work-performance and how the average total performance score across staff in a unit capture unit-level performance
• Productivity (self-reported): Self-reported measures of bureaucratic performance by respondents to civil servants survey through 360 performance review by respondent of themselves and their colleagues
• Productivity (mystery bureaucrat) Extent to which unit responds to requests from a random/‘mystery’ CSA Civil Servant
• Productivity (co-respondent): Assessment of partnering/interaction with other units and of unit-level performance as viewed by the civil servants survey respondent
• Productivity (independent audit): Audit of unit’s activities by an ex-civil servant

c. Motivation: measured at civil servant level
• Perry test of public service motivation: Repeated application of the Perry test of public sector motivation (similar tests may be included)
• Bureaucratic satisfaction: Reports by civil servants of their satisfaction in the workplace
• Locus of control: Extent to which civil servants believe they have control over their productivity
Primary Outcomes (explanation)
See above

Secondary Outcomes

Secondary Outcomes (end points)
a. Bureaucratic relationships: measured at civil servant pairs or civil servant level
• Manager-employee relationship: Extent to which civil servants report improved manager-employee relationships
• Civil servant - civil servant relationships: Extent to which civil servants report improved relationships with civil service colleagues
• Social capital/trust: Extent to which civil servants feel they are part of a unit with colleagues they bond with and trust

b. Spillovers:
• Exposure to treatment of control units: Extent to which individual officers in control group were exposed to interventions
• Spillovers: Spillover effects of the interventions on control units interacting with treatment units

c. Perception of CSA:
• Civil servant perception of CSA: Extent to which civil servants report positive views of CSA and its reform processes
Secondary Outcomes (explanation)
See above

Experimental Design

Experimental Design
Our sample of around 450 administrative units will be randomly assigned to one in three groups:

(1). Pure control group: In the ‘Control units’ (approximately 150 administrative units), we will roll out the reform following the standard CSA procedure (limited information provided to supervisors only)

(2). Pure information training group: ‘Information units’ (approximately 100 units) will receive more in‐depth information about the reform by attending three information trainings for both supervisors and supervisees

(3). Information Training & Incentives Units: the remaining units will receive monetary incentives for implementing the reform on top of the information/training sessions. They will further be randomly split into;
(i) the ‘Information and supervisor-only incentive’ subgroup (approximately 100 units), and
(ii) ‘Information and all-employees incentive’ subgroup (approximately 100 units) that are offered different types of monetary incentives.
Experimental Design Details
a. Sample Definition
Our sample consists of Monrovia-based civil servants only. Civil servants who are based in offices outside of the capital Monrovia are excluded.

The second group to be excluded from participating are non-civil servants such as political appointees (Ministers, Deputy or Assistant Ministers, Director Generals, Deputy Director Generals or Commissioners) who may act as appraisers, but who are not being appraised themselves. The reason being that these posts are temporary and subject to political re-shuffling by the President. Political appointees are also not required to partake in the PMS processes other than in their role as supervisors of the civil servants who report directly to them.

The third group excluded in the sample are consultants or government staff who are not on the civil service payroll but may be paid through government or discretionary allowances. These persons tend to be recruited based on favor from political appointees rather than through the standardized civil service employment process. As such, they are not in the CSA’s registry and therefore not considered ‘civil servants’ who are subject to the CSA’s policies and protocols including the Performance Management System.

The fourth group to be excluded from the experiment are the public servants who work for autonomous state entities, which have their own human resource management (HRM) processes and procedures and do not adhere to CSA rules and guidelines. Some of these organizations use performance appraisal tools, while others do not. Example of such autonomous agency is the GC.

Finally, 9 of the Baseline Survey respondents from CSA and LIPA were further excluded due to their active contribution to and knowledge of the research study as PMS Trainers. Once they were excluded there were insufficient staff left in their units to include them in the study. Thus, 2 units from the CSA and 1 unit from LIPA were kept out of the experiment.

The experimental design, combing all interventions laid out in the previous sections, is summarized in the above Table 1.
Randomization Method
Our randomization will be done at the unit level using Stata. We will undertake the randomization based on the census of units and staff already recorded.
Randomization Unit
Unit (bureaucratic divisions) level randomization.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
There are about 450 units in total, which will be split between treatment and control.
Sample size: planned number of observations
Target enrollment: around 3500 civil servants, approximately 450 units. These numbers are preliminary and may be subject to change.
Sample size (or number of clusters) by treatment arms
(1). Pure control group: Approximately 150 units (ca. 1/3 of all administrative units).
(2). Pure information group: Approximately 100 Informational units.
(3). Information & Incentives Units: the remaining ~200 units. They will further be evenly split into two sub-groups offering different types of incentives:
o Information and supervisor incentive: ~ 1/2 of the remaining units.
o Information and all employees incentive: ~ 1/2 of the remaining units.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Harvard Human Research Protection Program
IRB Approval Date
2018-04-30
IRB Approval Number
Submission Number: CR17-0116-01
IRB Name
Harvard Human Research Protection Program
IRB Approval Date
2017-09-15
IRB Approval Number
Submission Number: MOD17-0116-02
IRB Name
Harvard Human Research Protection Program
IRB Approval Date
2017-11-17
IRB Approval Number
Submission Number: MOD17-0116-01
IRB Name
University of Liberia-Pacific Institute for Research & Evaluation Institutional Review Board (UL-PIRE IRB)
IRB Approval Date
2017-05-22
IRB Approval Number
Protocol Number: 16-10-016
IRB Name
University of Liberia-Pacific Institute for Research & Evaluation Institutional Review Board (UL-PIRE IRB)
IRB Approval Date
2018-06-05
IRB Approval Number
Protocol Number: 16-10-016
IRB Name
Harvard Human Research Protection Program
IRB Approval Date
2017-04-04
IRB Approval Number
Protocol Number: IRB17-0116

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials