Can Feedback from a Large Language Model Improve Health Care Quality?

The quality of healthcare in low- and middle-income countries is notoriously low (Banerjee et al., 2023). In Nigeria, only a small proportion of primary care patients are seen by a Medical Officer (MO) instead of a mid-level provider such as a Community Health Extension Worker. Access to qualified medical doctors is a barrier to good health care (Okeke, 2023). This project tests whether Large Language Models (LLMs) can improve patient care in Nigerian primary care clinics. An LLM-based tool provides "second opinions'' to community health extension workers (CHEWs) at two clinics in Nigeria. These second opinions are intended to mirror what a reviewing physician might advise the CHEWs after seeing or hearing their initial report on a patient. For our main analysis, we use a within-patient comparison of two patient notes created by the CHEW; one during the initial patient consultation, and one after the LLM feedback was received. The patient is also seen by a fully trained MO who is in charge of patient care. The MO conducts a blinded review of the CHEW's patient notes to measures changes in the CHEW's care as a result of the LLM feedback.