Is an Ounce of Prevention Worth a Pound of Cure? Evidence from a Large-Scale Vaccination Experiment in China

To shed light on the long run viability of public immunization programs, we conduct a large-scale experiment in China to estimate the cost-effectiveness of influenza and pneumococcal vaccines on elderly patients (aged 60 and above) with chronic obstructive pulmonary disease (COPD). We study a sample of approximately 1,800 elderly COPD patients who were randomized into community groups that received either, both, or none, of the influenza and pneumococcal vaccines; this ensures sufficient statistical power to identify the causal effect of the vaccination program.

2013-10-28

2013-11-15

The main outcome that we look at in this paper is the total medical expenditure, which comprises three items: inpatient expenses, outpatient expenses, and prescription medicine.

In terms of COPD condition, we use two well-known indicators that track disease progression. First is the frequency of acute exacerbation (FAE) of COPD, which reflects the number of times a patient experienced acute exacerbation within the past year. Second is the COPD assessment test (CAT) score, which is an additive index that captures the severity of eight symptoms: coughing, coughing up phlegm, the sense of suppression in the chest, feeling out of breath when climbing, activities at home being affected by COPD, activities outside home being affected by COPD, poor sleep quality, and feeling fatigue. Both FAE of COPD and the CAT score were recorded by general practitioners during face-to-face interviews with the patients at baseline as well as in the follow-up stages.
CAT scores are constructed by first assigning to each symptom a score of 1-5 (mildest to severest), and then adding up all symptom-specific scores; therefore, CAT scores range from 8 to 40. According to the Chronic obstructive pulmonary disease group of Chinese Thoracic Society (2013) and GlaxoSmithKline group of companies (2016) (COPD group of Chinese Thoracic Society 2013; GlaxoSmithKline group of companies 2016), CAT scores capture the degree to which COPD impacts a patient’ normal life: a score of 0-10 reflects “slight” impact; 11-20 reflects “medium” impact; 21-30 reflects “large” impact; and 31-40 reflect “very large” impact. In addition, it is imperative to note that CAT is not the only official index that can be used to diagnose and treat COPD in China, although its biggest advantage lies in its wide coverage of symptoms (COPD group of Chinese Thoracic Society 2013). Since we have access to a patient’s complete medical records, and CAT is easily evaluated through questionnaires, we therefore prefer it over other indices that only focus on specific symptoms.

This large-scale randomized experiment was jointly conducted by Peking University’s Institute of Economic Policy Research (IEPR) and the Chinese Center for Disease Control and Prevention (CDC) in China from October 2013 to October 2014.
Two cities – Ningbo (in Zhejiang province) and Chongqing (a provincial-level municipality) – were specifically chosen as the locations for our study. There are three reasons for this choice. First, these two cities have the highest intensity of CDC branches that are “National Demonstration Units” for chronic disease management; such units have adopted a digital management system that tracks and monitors patients with various chronic diseases, making it convenient for us to identify and locate COPD patients. Second, Ningbo and Chongqing are geographically and socioeconomically representative, to the extent that they are located in the eastern (Zhejiang province) and central (Chongqing municipality) regions respectively. Thirdly, these two cities have very similar latitude – around 30 degree north – and thus face very similar influenza seasons from November to March; this implies that we can run the experiment simultaneously across the two cities.
With the permission of the CDC, we accessed the health records of three counties (a mix of rural and urban) in each city from which COPD patients were then drawn using a stratified sampling method. Specifically, within each county, we randomly selected several communities and assigned them groups that received: the influenza vaccine (hereafter, group A), the pneumococcal vaccine (hereafter, group B), both vaccines (hereafter, group C), and no vaccine (hereafter, group D, which is essentially the control group).
Within each community, our target population for the study comprises seniors aged 60 and above who have been identified as COPD patients by the local CDC authorities. To rule out possible confounding health factors, we then excluded individuals who were (i) inoculated with the influenza vaccine (within the past one year) or the pneumococcal vaccine (within the past five years), (ii) experiencing acute COPD, (iii) suffer from respiratory diseases other than COPD, (iv) undertaking hormonal therapy for cancer, (v) suspected or confirmed to be infected with pneumococcal infection.
The experiment was carried out in two stages. The first stage ran from mid-October to mid-November 2013, when we trained local general practitioners to conduct a baseline survey on all the patients in our sample. This survey collected basic demographic and socioeconomic information, physical condition, COPD status, and medical expenditure in the preceding year. Patients in the three treatment groups were then informed to visit the nearby CDC clinics to receive their allocated vaccine injection. All injections were completed in the first two weeks of November.
The second stage comprises four rounds of follow-up surveys – in the first two weeks of January, March, May, and October in 2014 – on all patients in our sample. These surveys were intended to track their COPD condition and medical expenditure across four different seasons.

Randomization done in office by a computer.

We randomized at the community rather than at the individual level. This is not because community randomization is preferable for an individual-level study but rather because individual randomization is not practical in this particular context. Indeed, a Chinese community – generally defined as a densely populated residential compound that houses approximately 1,000 households across several apartment blocks – typically features an environment in which residents, especially elderly ones, interact frequently. As such, should individuals within the community be selected for vaccination, we expect that the news will spread rapidly, which might induce voluntary vaccination among the other residents of the same community; this in turn will jeopardize our experimental design. In other words, the communal nature of Chinese communities precludes “uncontaminated” randomization at the individual level.

Yes

42 communities.

About 1840 aged above 60 COPD patients.

460 patients injecting influenza vaccine;
460 patients injecting pneumococcal vaccine;
460 patients injecting both vaccines;
460 patients injecting no vaccines.

With type-1 error probability as 0.05 and type-2 error probability as 0.1, the desirable sample size for each treatment/control group should be 157 patients. We further assume a 20% participation denial rate，20% attrition rate in the follow-up surveys, and a cluster sampling factor of 2, the final design sample size per group is approximately 460 patients.