How does survey mode affect household consumption and poverty estimates: experimental evidence from in-person and telephone surveys

Remote data collection modes, such as phone surveys, have exploded in popularity in low- and middle-income countries following the COVID-19 disruptions that made traditional, face-to-face interviews both questionable ethically, difficult to train, and during lockdowns, effectively impossible. While phone survey modes are economical, quick to implement, and potentially make safeguarding anonymity easier, they also suffer drawbacks. They lack the visual aids/cues and observed respondent reactions to questions, which are often keys for collecting good quality data. Phone interviews must be much shorter to keep respondents from losing their focus, and so phone interviews may not be suitable for complex questions that are cognitively demanding or that require time to answer. The (non-random) difference in access to remote survey modes (e.g., phone ownership) and their implications on sample representativeness is also another concern that can affect the data reliability.

Despite the above-mentioned concerns on the reliability of data collected through remote survey modes, there are not yet many systematic studies that contrast responses from remote survey modes with face-to-face interviews in the context of low- and middle-income countries. In this study, we explore how survey mode affect household consumption and poverty estimates through a survey experiment that randomizes whether respondents are interviewed in person or by telephone. Specifically, we randomly assign representative sample households from Addis Ababa into two balanced groups (i.e., either to an in-person survey or a telephone survey) and administrator a “standard” food and non-food consumption survey module, to a panel of households that have been interviewed several times in the past two years (both in-person and by phone). To also be able to test whether survey fatigue differs by interview method, we cross-randomize the order of questions on food consumption into two groups.