• In the formative research phase, we will gather qualitative evidence on the potential benefits and barriers to adoption of Babyl from patients, providers, and other key players in both digital and in-person health care delivery. These insights will be used to develop an encouragement intervention, which will be tested in a pilot and ultimately brought to scale in a randomized control trial. The exogenous variation in Babyl usage introduced by our intervention will allow us to evaluate the impact of Babyl on healthcare utilization, health outcomes, and patient costs.
• In addition to data from the exogenous encouragement intervention, we will also leverage quasi-random variation in Babyl penetration across regions and over time. We will use historical administrative data to compare health care outcomes and utilization across Babyl and non-Babyl patients and to compare aggregate health outcomes and utilization across catchment areas with higher or lower levels of Babyl penetration in the market.
• Finally, we will assess quality of care at baseline, prior to the encouragement intervention. To overcome common challenges in the evaluation of health care quality, we use standardized patients, or actors who seek care with a prespecified set of symptoms. In addition to evaluating quality of care, data from standardized patients will allow us to explore aspects of clinical decision-making that are difficult to study with observational data. Specifically, we study the influence of patient suggestions and requests and whether these vary across platforms and patients.
• Because providers may differ across platforms, we will conduct a survey of providers visited by our standardized patients including measures of clinical experience, clinical knowledge assessed through vignettes, attitudes towards patients, and burnout. In our assessments of quality, we will control for these provider characteristics to isolate the impact of Babyl on quality of care for patients with different symptoms and other characteristics.
Baseline quality assessment:
At baseline, we will evaluate aspects of quality of care, clinical decision making, and costs of seeking consultation for primary care in conventional care (at 80 health facilities) vs. digital health services (through Babyl).
o We capture levels of quality of care through standardized patients ("mystery shoppers" or "fake patients"), who are individuals recruited locally and trained to portray three different conditions. SPs have been extensively trained to present their condition in the same way (same description of their symptoms, backstory, and answers to questions by the provider).
o These conditions were selected to reflect different aspects of quality of care, and to measure potential under- and over-treatment.
o In addition to the three medical conditions, we layer two additional experiments to assess clinical decision making across platforms.
First, while presenting the same conditions, patients may suggest to the provider that they may have the correct condition, an incorrect condition, or make no suggestion.
Second, patients may request an unnecessary prescription after the end of the interaction.
o Outcomes captured by the standardized patients through an exit questionnaire given to them after the interaction with providers' include: history questions asked by the provider, laboratory tests ordered, medicines dispensed or prescribed, and referrals made.
o We will also explore variation across SP gender, age, and insurance status (majority covered by the community-based health insurance, but some paying out of pocket), as well as day and time of the visit.
o To control for differences across providers in Babyl vs. conventional care, we will also field a provider survey after the SP fieldwork. This will include measures of providers’ clinical experience, knowledge of treatment for the SP conditions, caseload, burnout, altruism towards patients, personality characteristics, and attitudes about the pandemic and telemedicine.