Disentangling the Drivers of Taste Discrimination Using List Experiments.

2024-07-15

2025-12-31

i) The average differences-in-means between control and treatment groups from all lists for all key sensitive statements (using the list experiment data).
ii) The answers to the direct sensitive questions (using the survey data).
iii) The differences between i) and ii).
iv) The portion of an endowment that participants choose to donate to a charity versus keep for themselves and how it correlates with i) and ii).

i) Average differences-in-means between treatment and control from all lists: this is the outcome variable that comes from our double list experiment technique. As explained in more detailed in the experimental design section of our pre-analysis plan registration, for a given sensitive statement, half of the subjects will see “List A” and “List B + Key Sensitive Item” and the remaining half will see “List A + Key Sensitive Item” and “List B”. We will take the difference in means based on the answers provided to “List A” and “List A + Key Sensitive Item”, next take the difference in means based on the answers provided to “List B” and “List B + Key Sensitive Item”, and then we will calculate the average of these two differences in means. This gives us the estimated share of the population with the key sensitive attribute.
ii) The answers to the direct sensitive questions (using the survey data): these are baseline estimates of the share of the population with the key sensitive attributes (without accounting for social desirability bias).
iii) The differences between i) and ii): this is the estimated size of the social desirability bias.
iv) The portion of an endowment that participants choose to donate to a charity versus keep for themselves and how it correlates with i) and ii). Specifically, we will compare list experiment estimates and answers to the direct sensitive questions across participants as follows: (1) those who choose to donate a positive amount to the charity vs those do not; (2) the top and bottom 50th percentiles of donors; (3) the top and bottom 25th and 75th percentiles of donors. This is an estimate of how preferences elicited from the survey correlate with real-stakes choices.

We use a double list experiment, a variant of the original list experiment pioneered by Miller (1984). In a regular list experiment, participants are either assigned to a control or a treatment group. In the control group, participants are given a list with a set of statements and asked to indicate how many of those statements are true for them. In the treatment group, participants are given the same list with the same set of statements plus one additional sensitive statement (e.g., a statement about attitudes towards LGBTQ+ individuals). The difference in means between the treatment and control group is the estimated share of the population with the key sensitive attribute.

In our case, we exploit the double list experiment technique to increase power, following Glynn (2013). For each sensitive statement, we will have two lists (i.e., Lists 1A and 1B), designed to be positively correlated. Some participants will be randomized into List 1A (control) and List 1B + Key Sensitive Statement (treatment). Other participants will be randomized into List 1A + Key Sensitive Statement (treatment) and List 1B (control). Some participants will get List 1A first, others will get List 1B first (order will be randomized). The differences-in means between control and treatment groups from both lists will be averaged and that will be the share of the population with that key sensitive attribute.

Not available

As mentioned above, to randomize which lists present the key sensitive statement and the order in which lists are presented, participants will be randomized into one of the following 6 paths:
• Path 1: List 1A + Key, List 1B, List 2A + Key, List 2B, List 3A + Key, List 3B
• Path 2: List 1B + Key, List 1A, List 2B + Key, List 2A, List 3B+ Key, List 3A
• Path 3: List 2A + Key, List 2B, List 3A + Key, List 3B, List 1A + Key, List 1B
• Path 4: List 2B + Key, List 2A, List 3B + Key, List 3A, List 1B + Key, List 1A
• Path 5: List 3A + Key, List 3B, List 1A + Key, List 1B, List 2A + Key, List 2B
• Path 6: List 3B + Key, List 3A, List 1B + Key, List 1A, List 2B + Key, List 2A
Randomization will be done by a computer (Qualtrics).

Individual level randomization

No

Our goal is to run an initial pilot on online platform Prolific with approximately 800 participants. This is to check whether there are any issues with the instructions, code, or the software. Additionally, pilot participants on Prolific will be randomized into either the list experiment and the direct questions or only the direct questions. This is to check that direct question responses are not affected by the order in which they are presented with respect to the list experiments.

After the pilot phase in Prolific, our local partner in Chile, DATAVOZ, will conduct a set of cognitive interviews with 10 participants. Subsequently, DATAVOZ will run a small pilot, a soft launch, and a main sample with a total of around 4,000 participants.

Data from the initial Prolific pilot and from the cognitive interviews will not be combined with the main sample. On the other hand, if there are no issues after the small DATAVOZ pilot and soft launch, these data will be combined with the main sample.

Approximately 800 Chilean resident participants recruited via online platform Prolific. Approximately 4,000 DATAVOZ participants: 10 cognitive interview participants, 20 small pilot participants, 200 soft launch participants, and approximately 3,800-4,000 main sample participants.

We strive to have an equal number of participants randomized into the each of the six paths that vary which lists contain the sensitive statements and the order of the lists (i.e., Paths 1-6, as described above). We will use Qualtrics’s “Evenly Present Elements” feature when randomizing participants into each path.

Local partner firm, DATAVOZ, will recruit participants in Chile based on sex, region, and age quotas with the goal of achieving a representative sample of the Chilean population along these three dimensions.

Pre-Analysis Plan for “Disentangling the Drivers of Taste Discrimination Using List Experiments.”