Quantifying and Addressing Coverage Bias in Phone Surveys

With pandemic conditions making face-to-face data collection too risky or costly for most countries, global poverty researchers must rely heavily on mobile phone surveys. The main weakness of phone surveys is the difficulty achieving a representative sample (Gibson et al. 2017). Coverage bias will typically be correlated with outcomes of interest, introducing bias into most analyses of the survey data if not accounted for. This study takes advantage of a useful opportunity to test methods for addressing this sort of coverage bias in phone surveys, in which researchers are conducting a phone survey of a large sample of enrollees in a digital cash transfer program managed by a government agency. The researchers have baseline data for each potential respondent, including location, household demographics, and economic outcomes of interest from the original cash transfer program. Potential respondents are randomly placed into either high-intensity or low-intensity contact protocols, similar to the approach recommended in DiNardo et al. (2021), generating random variation in the probability that a respondent will end up in the sample, and allowing researchers to assess the marginal value of each follow-up call, given they did not answer on the previous attempt. Using the economic features in the baseline data as outcomes, we can then assess the effectiveness of various weighting schemes designed to adjust for coverage bias.