What Drives Bias in Responses to Survey Questions about Sensitive Traits and Behaviors?

The randomized interventions are different kinds of question structures in two substantive domains.

2017-04-03

2017-05-31

In Simpser (2017), I propose a model where the probability of truthfully answering a question about having engaged in a sensitive behavior (or exhibiting a sensitive trait) is a function of:
• A shame aversion parameter
• A lying aversion parameter
• A Bayesian probability calculation based on second-order beliefs about the interviewer’s priors (about whether the respondent engaged in the sensitive behavior)

I operationalize these concepts through survey questions about the following:

A. Subjective sense of privacy after being asked a question on the sensitive item (variable reveal).
B. Shame aversion (through a battery of survey questions inspired or drawn from SHArQ) (variables shame1, shame2, shame3, [shame4])
C. Degree to which lack of privacy would translate into shame (via the variable shame, only measured conditional on reveal).
D. Lying aversion (through a battery of questions drawn from psychology) (variables lie1, [lie2], lie3, lie4, lie5)
E. Respondent second-order beliefs about the interviewer’s priors (variable expect)
F. Respondent second-order beliefs about how much the interviewer cares that the respondent might have engaged in the sensitive behavior (variable reaction)
G. Respondent second-order beliefs about how much the interviewer cares that the respondent might have engaged in the sensitive behavior (proxied as one agree/disagree question in the battery about the normative desirability of the sensitive behavior). (shame4 for voting, lie2 for coin flip).

The treatments are assigned randomly to individuals according to a pre-specified probability distribution. The estimands are the statistical associations between the variables representing model parameters, on the one hand, and treatment effect signs and magnitudes, on the other.

Question structure is independently and randomly assigned for each question topic, to each respondent. Those who receive the direct question do not subsequently receive any additional questions. However, all those assigned to the RRT or LE questions subsequently receive the direct question on the same topic.

Everyone is asked (at least) one question about voting. In the coin toss section, only those who reported having obtained 3 heads in 3 tosses are asked a question about it. All others are not exposed to the coin toss questions.

Randomization of treatments to individual is done by a computer program.

The unit is the individual respondent.

No

The randomization is not clustered.

1000 individual respondents.

The randomization procedure defines the probability that any unit will receive each of the treatments.
15% will receive the direct question, 31% the RRT, 22% the list experiment without the sensitive item, and 32% the list experiment with the sensitive item. These percentages are applied separately to each of the two substantive categories of sensitive questions used. That is, each category of question is randomized independently. This procedure is followed until the planned number of observations is reached.