Impact evaluation of a performance-based financing project in the health sector in the Democratic Republic of Congo

Last registered on February 23, 2024

Pre-Trial

Trial Information

General Information

Title
Impact evaluation of a performance-based financing project in the health sector in the Democratic Republic of Congo
RCT ID
AEARCTR-0002880
Initial registration date
April 11, 2018

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
April 14, 2018, 5:26 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
February 23, 2024, 1:13 PM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Primary Investigator

Affiliation
World Bank

Other Primary Investigator(s)

PI Affiliation
Swiss Tropical and Public Health Institute

Additional Trial Information

Status
Completed
Start date
2015-06-01
End date
2023-06-30
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
The study evaluates the impacts of a Performance-Based Financing (PBF) program introduced in selected districts in the Democratic Republic of Congo. The program was introduced in the context of the “Health Systems Strengthening for Better Maternal and Child Health Results” project that is financially supported by the World Bank, the Health Results Innovation Trust Fund and other partners such as USAID, GFF, and the Global Fund. The main goal of the project is to improve coverage and quality of maternal and child health services in targeted areas through providing contracted health facilities with performance payments according to defined quantity and quality indicators.

The objectives of the impact evaluation are to: (a) identify the impact of the PBF approach on service utilization, quality of care, and health outcomes, and (b) identify key factors responsible for this impact. The study will evaluate The overall impact of the PBF approach as well as compare outcomes when program design elements are varied.
External Link(s)

Registration Citation

Citation
Fink, Günther and Gil Shapira. 2024. "Impact evaluation of a performance-based financing project in the health sector in the Democratic Republic of Congo." AEA RCT Registry. February 23. https://doi.org/10.1257/rct.2880-1.1
Former Citation
Fink, Günther and Gil Shapira. 2024. "Impact evaluation of a performance-based financing project in the health sector in the Democratic Republic of Congo." AEA RCT Registry. February 23. https://www.socialscienceregistry.org/trials/2880/history/212656
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
Performance payments will be made to health service providers and health administration units contracted in the target areas. Public, quasi-public and private health facilities, including health centers and first level referral hospitals, will be targeted in rural and urban areas. The purchasing, verification, community mobilization and coaching will be organized through provincial purchasing agencies (EUPs: Etablissement d’utilité publique). PBF payments will be paid to these health facilities in proportion to, and in payment for, achieved quantity and quality results. Facility payments will be based on (i) the quantity of services defined through a package of basic and complementary activities delivered to the targeted population, and (ii) the technical quality of these services as measured through comprehensive quantified quality checklists. Facility payments will be made quarterly after service volumes have been verified and quality of technical support and care has been assessed and certified by the EUPs, and validated through special governing boards at the provincial level. After the quantity and quality of services provided are certified, payment will be released to contracted health facilities via fundholding arrangements.

Modified quality checklists (quality incentives): An innovative aspect of the PBF approach in DRC will involve the content of the quality checklist that will be used for assessing the quality of care in health centers and hospitals. The amount of money paid for the quantity of services can be increased by up to 25% according to the quarterly quality score, if the facility exceeds the threshold score of 50%. In addition to determining the payment amount to facilities, the quality checklist also provides guidelines for good practice and structures the supervision of the facilities by the staff of the health zones. The study will evaluate the use of different types of quality checklists at different sets of health facilities. All checklists will include measure of structural quality, process and content of care. Examples of measures of structural quality are: proper storage of drugs, cleanliness of the facility and existence of required registries and protocols. Measures of process and content of care include indicators such as correct entries into registries and medical records and checking these records for proper application of the protocols. In selected health facilities, ‘Vignettes Quality Checklist’ will also include healthcare providers’ responses to clinical vignettes as indicators of quality of care. The vignettes will cover cases that represent the most common causes of morbidity and mortality and will be repeated each quarter. It is theorized that the quarterly repetition of these cases will reinforce the protocol related to these common cases. However, these measures are also a limited in representing quality of care as there might be a gap between what providers know and do when they are actually treating the patients.
Intervention Start Date
2016-09-01
Intervention End Date
2022-05-15

Primary Outcomes

Primary Outcomes (end points)
1. Utilization of incentivized health services measured through household surveys
2. Measures of quality of care: facility infrastructure and availability of supplies and equipment; providers' adherence to clinical protocols; patients' satisfaction
Primary Outcomes (explanation)
Data on service utilization, health behaviors and health outcomes will be collected through household surveys conducted in the catchment areas served by the health facilities. Quality of health services will be collected through health facility assessment as well as reports by patients on the care they received. Both household and facility surveys will be conducted by a survey firm that is not involved in any aspect of the implementation of the project.

Secondary Outcomes

Secondary Outcomes (end points)
1. Utilization of non-incentivized services
2. Prices paid for services
3. Staff motivation and satisfaction
4. Providers' clinical knowledge
5. Health outcomes (anthropometric measures)
6. Household health behaviors
Secondary Outcomes (explanation)
Although the ultimate goal of the project is to improve maternal and child health outcomes, it is important to keep in mind that detecting such outcomes might require a long duration as well as a very large sample. The study is not powered to detect statistically significant changes in health outcomes over the proposed time frame. Therefore, as reflected in the research questions above, the impact evaluation will focus also on the intermediate outputs of the project: utilization and quality.

Experimental Design

Experimental Design
The research questions will be answered employing randomized assignment of the different project components. The PBF intervention will be randomly assigned at the health zone level while the quality checklist intervention will be assigned at the catchment area level (Health zones have an average of about 10 catchment area with at least one health center in each catchment area).
Experimental Design Details
Randomization Method
The randomization of the health zones into the PBF intervention has been conducted through public randomization ceremonies in the different districts. Representatives of the different health zones picked folded pieces of paper from a transparent bucket.
The randomization of the vignette quality checklist will be done in office by a computer.
Randomization Unit
The PBF intervention was randomized at the health zone level.
Variations in the PBF approach (pay-for-vignettes and community engagement approaches) will be randomized at the health center level.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
There are 100 health zones and 500 catchment areas (5 in each health zone)
The baseline and endline surveys will cover the full sample. A midline survey will be conducted in a subset of 42 health zones (and 210 health centers) in order to measure short-term changes in quality of care.
Sample size: planned number of observations
500 health centers 5000 women with recent pregnancies 2000 health providers
Sample size (or number of clusters) by treatment arms
Half of the sample (50 health zones and 250 catchment areas) is in either the PBF or control treatment arm
For the midline assessment, 21 health zones in ech of treatment and control arms will be surveyed.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Comie national d'ethique de la sante- CNES
IRB Approval Date
2015-04-30
IRB Approval Number
N/A

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
May 15, 2022, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
May 15, 2022, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
58 health zones in 6 provinces
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
Final Sample Size (or Number of Clusters) by Treatment Arms
Data Publication

Data Publication

Is public data available?
No

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Program Files

Program Files
No
Reports, Papers & Other Materials

Relevant Paper(s)

Abstract
Background
Health systems’ weakness remains one of the primary obstacles towards achieving universal access to quality healthcare in low-income settings. Performance-based financing (PBF) programs have been increasingly used to increase access to quality care in LMICs. However, evidence on the impacts of these programs remains fragmented and inconclusive. We analyze the health system impacts of the PBF program in the Democratic Republic of the Congo (DRC), one of the largest such programs introduced in LMICs to date.

Methods
We used a health systems perspective to analyze the benefits of PBF relative to unconditional financing of health facilities. Fifty-eight health zones in six provinces were randomly assigned to either a control group (28 zones) in which facilities received unconditional transfers or to a PBF program (30 zones) that started at the end of 2016. Follow-up data collection took place in 2021–2022 and included health facility assessments, health worker interviews, direct observations of consultations and deliveries, patient exit interviews, and household surveys. Using multivariate regression models, we estimated the impact of the program on 55 outcomes in seven health system domains: structural quality, technical process quality, non-technical process quality, service fees, facility management, providers’ satisfaction, and service coverage. We used random-effects meta-analysis to generate pooled average estimates within each domain.

Results
The PBF program improved the structural quality of health facilities by 4 percentage points (ppts) (95% CI 0.01–0.08), technical process quality by 5 ppts (0.03–0.07), and non-technical process by 2 ppts (0–0.04). PBF also increased coverage of priority health services by 3 ppts (0.02–0.04). Improvements were also observed for facility management (9 ppts, 0.04–0.15), service fee policies, and users’ satisfaction with service affordability (14 ppts, 0.07–0.20). Service fees and health workers’ satisfaction were not affected by the program.

Conclusions
The results suggest that well-designed PBF programs can lead to improvements in most health systems domains relative to comparable unconditional financing. However, the large persisting gaps suggest that additional changes, such as allocating more resources to the health system and reforming the human resources for health management, will be necessary in DRC to achieve the ambitious global universal health coverage and mortality goals.
Citation
Shapira, G., Clarke-Deelder, E., Booto, B.M. et al. Impacts of performance-based financing on health system performance: evidence from the Democratic Republic of Congo. BMC Med 21, 381 (2023).

Reports & Other Materials