Impact Evaluation of a Performance-Based Financing pilot in the health sector in Tajikistan

Last registered on February 22, 2018

Pre-Trial

Trial Information

General Information

Title
Impact Evaluation of a Performance-Based Financing pilot in the health sector in Tajikistan
RCT ID
AEARCTR-0002623
Initial registration date
February 22, 2018

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
February 22, 2018, 6:38 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
World Bank

Other Primary Investigator(s)

PI Affiliation
World Bank

Additional Trial Information

Status
On going
Start date
2014-11-03
End date
2019-03-13
Secondary IDs
Abstract
The study evaluates the impacts of three different interventions introduced in the context of the Health Services Improvement Project that is implemented by the government of Tajikistan with the support of the World Bank. More specifically, the impact evaluation seeks to ascertain: (i) the impact and cost-effectiveness of a performance-based financing (PBF) model implemented in Tajikistan; and (ii) whether PBF is more effective or cost-effective if implemented in conjunction with additional low cost interventions. The other interventions are Collaborative Quality Improvement (CQI) and Citizen Score Cards (CS). The PBF shceme was introduced in selected districts and will be evaluated with a difference-in-difference approach. The CQI and CSC interventions were introduced in randomly selected health centers in both PBF and control district.
External Link(s)

Registration Citation

Citation
de Walque, Damien and Gil Shapira. 2018. "Impact Evaluation of a Performance-Based Financing pilot in the health sector in Tajikistan." AEA RCT Registry. February 22. https://doi.org/10.1257/rct.2623-1.0
Former Citation
de Walque, Damien and Gil Shapira. 2018. "Impact Evaluation of a Performance-Based Financing pilot in the health sector in Tajikistan." AEA RCT Registry. February 22. https://www.socialscienceregistry.org/trials/2623/history/26049
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
Performance contracts will be signed between the Ministry of Health (MoH) and local rayon (district) administrations in the districts participating in the PBF. The MoH will purchase a set of agreed MCH and NCD services from the local district administrations which are responsible for the health services provided by RHCs and HHs. Funds will be transferred directly to the recipient health facilities, through instructions issued by the MoH to the Treasury. The RHCs will receive the performance incentives on a quarterly basis according to in how far they have met the relevant criteria determining the PBF health facility payments (in particular, the verified volume of services delivered as well as the verified quality of these services). The availability of functional facilities, basic medical equipment and trained staff are pre-requisites for implementation of the PBF scheme in the project districts.
• The pricing for each output to be purchased from health facilities will take into account both public health priorities and the need to incentivize the delivery of good quality services. The outputs to be purchased from health facilities will include service output indicators for priority MCH, NCD and general PHC services adjusted for quality and facility-level indicators of quality.
• There are going to be two levels of verification of the information reported by the participating facilities. The first level verification of the accuracy of activities reported by the PHC facilities will be done quarterly by a District PBF Verification Committee. Payments will be released only after the first level of verification, and specified sanctions will apply if fraud is uncovered during the independent verification.
• The performance-based payments will be executed to a RHC and its associated HHs to encourage better teamwork and coordination. These payments will be supplementary to the funds from the public sector budget. The use of the performance-based payments will be governed by simple spending rules designed to promote provider autonomy and ensure transparency and accountability. Within these spending rules, payments may be used for performance bonuses to facility staff, to purchase minor health facility inputs and/or for minor repairs.
• The two levels of verification will be implemented as follows: The first level of verification will be carried out by District PBF Verification Teams. The State Health Activities Supervision Service (SHASS) will lead the verification efforts. The District PBF Verification Teams could possibly also include a representative from the rayon treasury department, staff from the central rayon hospital and other independent consultants. The first level verification will be done on a quarterly basis prior to the release of any performance-based payments. The independent verification will take place after the release of the payments.
• The independent verification of the PBF scheme should be done by an independent agency, UNICEF, that has impeccable reputation and is well respected in Tajikistan and in the international donor community. It is proposed that the independent verification will be done once every six months in a sample of health facilities. The objective of the independent verification is to ensure that facilities report achieved results accurately, and to identify and penalize instances of falsified reporting and fraud
The first additional intervention is Collaborative Quality Improvement, i.e., a facilitated quality improvement approach focused on common provider-identified objectives with performance feedback and competency training for provider quality improvement teams. The quality improvement intervention responds to policy concerns that performance incentives may not produce the desired improvements if providers lack the necessary competencies and knowledge. The Collaborative Quality Improvement intervention involves three essential elements: (i) Defining a feasible set of quality improvement objectives; (ii) Forming facility quality improvement teams; (ii) Data collection and analysis by the quality improvement teams of key performance parameters that relate to this problem; (iii) Feedback on performance, and a facilitated – by a quality improvement expert facilitator – discussion sessions among quality improvement teams from multiple facilities on strategies and best practices to improve quality around the identified problem; and (iv) Integrated competency-focused training as needed. As the defined quality problem is addressed, the group moves on to identify the next priority quality problem. Variants of this Collaborative Quality Improvement approach have been applied in a number of advanced and low income health system settings. However, despite its widespread use, systematic review indicates that evidence on this approach is positive but limited making this strategy a good candidate intervention for an impact evaluation (See, Schoueten, L. M.T. et al., 2008). As the literature indicates the importance of adapting the broad principles of this approach to the local context, the specific approach to be applied in Tajikistan is currently being developed, and the Tajikistan health sector Project is in the process of hiring a consultant to do so. Details of this approach will be updated as they are available.
The second additional intervention is Citizen Report Cards, i.e., the dissemination of information on the performance of local health facilities by distributing a facility report card that benchmarks the performance of local facilities against an appropriately chosen reference. Such report cards help to inform the potential users adequately about the quality of service provided at different facilities. Two possible reference points for benchmarking include national standards for quality of care and the average quality of care in the country/ rayon/ survey sample. These performance/ benchmarking data would be obtained from surveys conducted for the Impact Evaluation (IE) and analyzed to develop facility report cards. The use of citizen report card is aimed at improving the effectiveness of PBF by increasing the accountability of health facilities. Recent evidence (Björkman-Nyqvist, de Walque and Svensson, 2013) suggests that lack of information on performance, by rendering it more difficult to identify and challenge (mis)behavior by the provider, constrains the community’s ability to hold providers to account. With access to information the community appears better able to distinguish between health workers’ effort and factors outside the health workers’ control. As such, the community is in a better position to hold health staff accountable for potential lack of performance for which the staff is directly responsible. A team from RAND Corporation has been contracted to advise on the development of the specific approach that will be implemented. That team has already conducted interviews with key stakeholder and focus groups in the PBF project regions and a version of the citizens report cards will be piloted in February 2014. The RAND team will submit a final report by March 2014. This document will be updated once the details of the approach will be finalized.
Intervention Start Date
2015-01-01
Intervention End Date
2019-03-13

Primary Outcomes

Primary Outcomes (end points)
1. Utilization of primary health services (targeted and non-targeted by the program)
2. indicators of quality of health care: Avilability of supplies and quipment, cleanliness, condition of infrastructure, knowledge of staff, adherence to protocols of care for management of child illness and hypertension

Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The Performance-Based Financing was implemented in purposely selected districts (rayons). The evaluation sample included seven districts in which the PBF is implemented and 9 in which it is not. In both PBF and treatment controls, the CSC and CQI intervenitons are randomized at the Rural health center (RHC) level such that a third of the sample receives the CQI intervention, a third receives the CSC intervention and a third receives neither. In total there are six groups of RHCs:
1. PBF
2. PBF+CQI
3. PBF+CSC
4. CQI
5. CSC
6. Status quo
Experimental Design Details
Randomization Method
Randomization was done in an office by a computer
Randomization Unit
Rural health center and its affiliated health houses. 65 our of the 218 rurl health centers did not have any health houses affiliated with them.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
216 Rural health centers
Sample size: planned number of observations
216 Rural Health centers 5,904 Households
Sample size (or number of clusters) by treatment arms
In each of the six study arms there will be 36 RHC and 984 households
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Committee on ethics under the Ministry of Health and Social Protection of Tajikistan
IRB Approval Date
2014-10-24
IRB Approval Number
1-6/4767

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials