Experimental Design
We sought to obtain experiment participants from a random sample of households that is representative at the national level excluding the conflict area of Autonomous Region of Muslim Mindanao. To achieve this, we obtained a random sample of 2950 households distributed in 590 ‘barangay’ (sub-municipality) clusters distributed across most of the Philippines. The sample households were obtained using a multi-stage cluster design. First, we stratified by broad regions (5: the National Capital Region, North-Central Luzon, South Luzon, Visayas, and Mindanao). From the broad regions, we then selected specific regions (15) using proportionate sampling. Provinces (62 out of 80) were drawn using systematic sampling. From these provinces, municipalities and cities (243 out of 1395) and the barangays (590 out of 37 165) within these municipalities and cities were likewise selected using systematic sampling. Finally, for each barangay cluster we drew five households using simple random sampling
.
In our main experiment, three quarters of the sampled municipalities within each broad region were assigned via randomization as treatment sites; the rest became control sites. The municipality of the sample household (rather than the barangay or household) was selected as the unit for the experimental cluster to reduce the risk of contamination between the treatment and control groups: treatment households could have more easily passed on the details of the information leaflets to households in the control group. We interviewed, on average, 12 households per municipality.
All sampled households in the intervention sites who were eligible for the IPP program (Group I) were given the first intervention; sampled households in the control sites, including those who are IPP-eligible (Group II), were not. We considered households eligible for IPP if the household head or the spouse claims not to be covered is unsure whether the family is covered, or claims to be covered but has not paid a premium in the preceding six months (the PhilHealth criterion to be considered an active member). Households in Group I who had still not enrolled by January 2012 (9–11 months after the baseline survey – became the sample for our sub-experiment (Group III). This sub-experiment was at the household level, not the municipality level.
Power calculations were done with a view to the main experiment only. We fixed the sampling of municipalities and households within municipalities to achieve at least 80% power at the 5% level to detect enrollment rate increases (compared to the control group rate of 10%) of 7.5 percentage points in the main experiment. This effect was chosen because it was felt to be the minimum impact that might be considered meaningful from a policy perspective; a 50% subsidy combined with an information packet is a serious policy effort, and an effect of less than 7.5 percentage points would raise serious questions about their usefulness as a mechanism to help achieve UHC. In our calculations, we assumed a PhilHealth overall coverage rate of 53% and an IPP-coverage rate of 33%2 a standard deviation in the enrollment rate of 0.25, an intra-cluster correlation coefficient (ICC) of 0.16 (the municipality is our cluster), and a ratio of intervention to control households of just under 3 (we have 243 municipalities in total of which 179 were assigned to the intervention group).
On these assumptions, we could, in fact, have sampled fewer municipalities and fewer households and still achieved 80% power to detect a 7.5 percentage point effect. Or equivalently with our sample we could detect a somewhat smaller effect with 80% power: we could reduce the target effect to 7.25 percentage points and still have 80% power; below 7.25, the number of municipalities in the intervention group becomes too small. We chose to sample so many municipalities and so many households per municipality in part because we were uncertain how many sampled households would be IPP-eligible. In the event we sampled slightly more households than necessary to achieve 80% power given our assumptions.
The full sample at baseline is 2950 households. By design three quarters of these live in intervention municipalities. Of these, 1183 (53.3%) proved ineligible for the experiment, already being an IPP member or in another PhilHealth scheme. (The fraction of the sample with some form of PhilHealth coverage already was somewhat lower in the control group: 47.5%.)
The 1037 eligible families in the intervention sites (referred as Group I) were offered Intervention I sometime during the period February–April 2011; the 383 IPP-eligible families in the control sites (referred as Group II) were not. Of the 918 families in Group I who – as of January 2012 – had not enrolled in the IPP scheme, 787 were still eligible for IPP, not having moved during the course of 2011 to another PhilHealth target group, e.g. becoming a formal-sector worker or becoming eligible for the indigent scheme. These families (referred as Group III) were randomly assigned into two groups, with 392 receiving Intervention II only, and 395 receiving both Interventions II and III.