Kenya Patient Safety Impact Evaluation
Last registered on January 11, 2018

Pre-Trial

Trial Information
General Information
Title
Kenya Patient Safety Impact Evaluation
RCT ID
AEARCTR-0001001
Initial registration date
January 11, 2018
Last updated
January 11, 2018 7:53 PM EST
Location(s)

This section is unavailable to the public. Use the button below to request access to this information.

Request Information
Primary Investigator
Affiliation
The World Bank
Other Primary Investigator(s)
PI Affiliation
The World Bank
Additional Trial Information
Status
On going
Start date
2015-04-01
End date
2019-03-29
Secondary IDs
Abstract
This study evaluates the impact of accountability mechanisms–through different models of health inspections–on quality of care and patient safety, quantity, and prices of health services in Kenya. Using health markets as the unit of intervention (clusters of health facilities where no facility is more than 4 KM from the center of the market), we experimentally allocate all private and public health facilities in three Kenyan counties to one of three arms: (1) high-intensity inspections with enforcement of warnings and sanctions for non-compliant facilities; (2) high-intensity inspections with enforcement of warnings and sanctions for non-compliant facilities and public disclosure of inspection results, and (3) “business-as-usual” low-probability inspections (the control group). The results from this study will contribute to understanding the extent to which governance and accountability mechanisms can help improve service delivery in low-income countries, particularly in this case related to improving patient safety and quality of care in the public and private sectors, and if so, how these systems may operate when they are implemented “at scale.” Another important contribution of this project is the development of a set of tools and instruments that can be broadly deployed both in surveys of quality and in the design of inspection systems across diverse low-income settings.
External Link(s)
Registration Citation
Citation
Bedoya, Guadalupe and Jishnu Das. 2018. "Kenya Patient Safety Impact Evaluation." AEA RCT Registry. January 11. https://www.socialscienceregistry.org/trials/1001/history/24878
Experimental Details
Interventions
Intervention(s)
1272 health facilities in three Kenyan counties–Kakamega, Kilifi and Meru–are experimentally allocated to three arms: (1) high-intensity inspections with enforcement of warnings and sanctions for non-compliant facilities; (2) high-intensity inspections with enforcement of warnings and sanctions for non-compliant facilities and public disclosure of inspection results; and (3) “business-as-usual” low-probability inspections (the control group).

The new regulatory framework used for this experiment was developed previous to the implementation of these pilots including: (A) a refined Joint Health Inspections Checklist (henceforth Checklist) that is easy to deploy and is focused on the fundamentals of patient safety; (B) a scoring system that allows facilities to be categorized according to the level of risk presented to patients; (C) scores that trigger warnings and sanctions to be enforced according to a facility’s level of risk.

Treatment 1 - High-intensity inspections: Every facility in this treatment group will be inspected using a checklist of safety standards that was developed in consensus and refined over 3 years. The health facilities will receive a copy of the Checklist and will be warned or sanctioned according to their risk performance.

Treatment 2 - High-intensity inspections and public disclosure of patient safety performance: Every facility in this treatment group will be inspected as in treatment 1. Following the inspection, facilities will be assigned a letter-grade to be prominently posted outside the health facility. The health facilities will receive a copy of the Checklist results with details on their assessment and grade. A campaign introducing patients to the grading system will be included as part of the accountability system.

Control - “Business-as-usual” health inspections: Health facilities in this group will operate under the current regime for health inspections, which results in inspections less than 4% of the time. These facilities did not receive routine inspections during the implementation period.
Intervention Start Date
2016-11-14
Intervention End Date
2017-12-15
Primary Outcomes
Primary Outcomes (end points)
We examine the impact of the intervention on a triad of outcome groups that are important in settings with public and private providers: Quality of care and Patient Safety, Quantity and Prices. Quality and Patient Safety will be measured through three main indicators that gauge diverse dimensions of quality and patient safety at different points in the continuum of the health care chain: (1) Adherence to a Checklist developed by the regulators and that includes indicators of quality and patient safety related to protocols, infrastructure, and equipment at the health facility level; (2) Adherence to patient safety practices related to infection prevention and control; (3) Adherence to case-specific checklists of essential and recommended care for four medical cases (subject to budget availability). Quantity (demand) of care services will be measured through patient flow records from health facilities and administrative data, and prices will be measured through patient exit surveys.
Primary Outcomes (explanation)
Below is a list of the main outcomes of interest for this analysis, as well as the source and the method of data collection used in each case.

• Adherence to a Checklist of quality of care and patient safety standards. This is a Checklist for Singular/Joint Inspections for Public and Private Medical Institutions, which has been developed by a technical working group (TWG) including the regulatory bodies under the Ministry of Health, the private sector, and other stakeholders. This Checklist includes indicators of infrastructure, equipment, staff characteristics, and protocols to measure quality and patient safety across all units of a health facility, and hence it allows measuring how facilities stand with respect to the standards set by the regulation. The results are mapped into a score and the measure of adherence we will use is the score as a percentage of the maximum score possible for each facility. We developed an electronic version of this Checklist. Data will be collected through surveys to the facility in-charge and staff, as well as through observation and verification during the surveys following closely the protocols that inspectors follow when they conduct inspections.

• Adherence to patient safety practices related to infection prevention, and control (IPC) in primary care, which are measured by direct observation of select practices in 3 outpatient sites. These outcomes will measure adherence to patient safety practices for five groups: (i) hand hygiene, (ii) injection and blood draw practices, (iii) use of gloves, (iv) disinfection of reusable equipment, and (v) waste segregation. This selection was based on their high-value (i.e. strong link to health-care infections), high frequency of opportunities (i.e., points in time during the care chain when a patient safety practice should be performed to avoid likely adverse events), and feasibility to develop valid, reliable, and generalizable indicators in the planned time and conditions. Adherence indicators are based on indications and actions and adapted to the specific violation.

• Adherence to case-specific checklists of essential and recommended care for 4 medical cases (subject to budget availability). This dimension of quality of care and patient safety is measured through Standardized Patients, which allows us to gauge the extent of correct treatment (both under and over-treatment) in these facilities. The method of Standardized Patients (SPs) was used in the first large-scale population study in India in 2008-2009 (See, e.g., Das et al., 2012). SPs are people from the local community who are extensively trained to present the same case to multiple providers. To assess the quality of care, SPs are trained to recall all history questions, examinations, and diagnoses. They are debriefed with a structured questionnaire within one hour of the interaction. The quality of medical advice is assessed by the time spent with patients, by providers’ adherence to case-specific checklists of essential and recommended care, the likelihood of correct diagnosis, and the appropriateness of treatment. SPs will undergo an exit survey where the data on their interaction with the doctor is recorded.

• Price of services at health facilities. The data on prices will come from a short survey that will be collected from patients from various demographic characteristics exiting the health facilities, as well as from SPs. In large health facilities, sampling of patients will be done randomly using a skip routine.

• Quantity (demand) of health care services: The data on demand corresponds to the number of patients in the last complete month at the time of the survey, collected at the health facility level through a survey from the forms reported to the MOH, or from the facility's alternative record system.
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
We use an stratified cluster randomized experimental design to estimate the causal impact of the interventions at the facility level, where the clusters are markets of health facilities (where facilities are within 4 km from the market center) and stratification happens at the market size and county level. We experimentally allocate around 273 markets and all facilities (around 1,272) in each market in the three evaluation counties to one of the three accountability systems (treatment 1, 2 and control). We will also present results at the market level. For this analysis, this is an stratified random experimental design where the unit of analysis is the same as the unit of randomization.
Experimental Design Details
Not available
Randomization Method
Randomization done in office by a computer.
Randomization Unit
Market of health facilities
Was the treatment clustered?
Yes
Experiment Characteristics
Sample size: planned number of clusters
273 markets of health facilities
Sample size: planned number of observations
273 markets of health facilities, 1,272 health facilities
Sample size (or number of clusters) by treatment arms
Treatment 1: 90 markets, 388 health facilities
Treatment 2: 96 markets, 473 health facilities
Control: 87 markets, 411 health facilities
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Adherence to Checklist Standards (score/max score, %) / MDE= (0.27 SD; 25%) Adherence to IPC practices (safe action/indication) (%) / MDE= (0.22 SD, 27%) Adherence to Checklist of essential recommended care (%) / MDE=(0.22 SD, 27%) - (subject to budget availability) Price per Visit (KES) / MDE = (0.27 SD, 20%) Quantity (Monthly patient flow) (Outpatients) / MDE (0.27 SD, 26%)
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
African Medical and Research Foundation
IRB Approval Date
2014-01-21
IRB Approval Number
AMREF-ESRC P94/2013