Attitudes towards LGBT individuals and discrimination

2022-01-20

2024-01-20

i) The average differences-in-means between control and treatment groups from all lists for all sensitive statements (using the list experiment data)
ii) The answers to the direct sensitive statements (using the survey data)
iii) The differences between i) and ii)

i) The average differences-in-means between control and treatment groups from all lists for all sensitive statements (using the list experiment data):
This is the outcome variable that comes from our double list experiment technique. As explained in more detailed in the experimental design section of our AEA registration, for a given sensitive statement, half of the subjects will see “List A” and “List B + Sensitive Item” and the remaining half will see “List A + Sensitive Item” and “List B”. We will take the difference in means based on the answers provided to “List A” and “List A + Sensitive Item”, next take the difference in means based on the answers provided to “List B” and “List B + Sensitive Item”, and then we will calculate the average of these two differences in means. This gives us the estimated share of the population with the key sensitive attribute.

ii) The answers to the direct sensitive statements (using the survey data):
These provide baseline estimates of the share of population with the sensitive attributes.

iii) The differences between i) and ii):
This is the estimated size of the bias due to social desirability and underreporting of stigmatized attitudes.

We use a list experiment technique that was pioneered by Miller (1984). Our participants are either assigned to a treatment or a control group. In the control group, participants are given a list of 4 statement and asked to indicate how many of those statements are true for them. In the treatment group, participants are given the same list of statements plus a sensitive statement (e.g., a statement about attitudes towards transgender individuals). The difference in means between the treatment and control group is the estimated share of the population with the key sensitive attribute. We also directly ask questions regarding the sensitive statements toward the end of the survey. The direct questions provide baseline estimates of the share of population with the sensitive attributes. This allows us to estimate the size of the bias due to social desirability and underreporting of stigmatized attitudes.

Following Glynn (2013), to increase power, we will use a double-list experiment technique. To describe it simply, for each sensitive statement, we will have two lists (e.g., list 1A and list 1B) that are designed to be positively correlated. Half of the participants (randomly selected) will get list 1A (control) and list 1B+sensitive statement (treatment). The other half will get list 1A+sensitive statement (treatment) and list 1B (control). Some participants will get list 1A first, others will get list 1B first (order will be randomized). As suggested in Glynn (2013), the differences-in-means between control and treatment groups from both lists will be averaged and that will be the share of the population with that key sensitive attribute.

In this project, we have two sensitive statements/questions that we are interested in studying:

• "Do you think the law should prohibit employment discrimination against transgender individuals?"
• "Would you be comfortable having a transgender manager at work?"

We will use the double list experiment technique for both statements (Lists 1A and 1B for the first sensitive item and Lists 2A and 2B for the second sensitive item). This means that each participant will see a total of four lists in a randomized order described in more detail in the randomization method section.
Participants never disclose any identifying information, and the survey is completely anonymous.

We also have the following direct questions on other sensitive issues which we may study using a similar method in the future:

• “Do you think the law should prohibit housing discrimination against transgender individuals?"
• "Would you be comfortable having a transgender person as a neighbor?"
• "Would you be comfortable having an openly lesbian, gay, or bisexual manager at work?”
• "Do you think the law should prohibit employment discrimination against lesbian, gay, or bisexual individuals?"
• "Do you think the law should prohibit housing discrimination against lesbian, gay, or bisexual individuals?"
• "Would you be comfortable having an openly lesbian, gay, or bisexual person as a neighbor?"
• "Do you think that private businesses (such as cake decorators or florists) should be able to refuse service to same-sex couples or other LGBTQ+ individuals for religious reasons?"

We randomize the order of the lists such that half of the participants see the lists for the first sensitive item (Lists 1) and the other half see the lists for the second sensitive item (Lists 2) first. Also within each list group (i.e. Lists 1 and 2), we randomize the order such that half of the participants see lists A first then B and vice versa. More specifically, we created four paths that a participant will follow where SS stands for “sensitive statement”:
Path 1: 1A - 1B+SS - 2A - 2B+SS
Path 2: 1A+SS - 1B - 2A+SS - 2B
Path 3: 2B - 2A+SS - 1B - 1A+SS
Path 4: 2B+SS - 2A - 1B+SS - 1A

Participants are randomized into one of these paths. More specifically, in order to ensure that we have a balanced sample of participants in each of these paths, the path that each participant follows will be determined based on their experiment start time. For example, first participant sees the first path, second participant sees the second path … fifth participant sees the first path and so on.

Also, within each list, the order of the statements are randomly determined at the individual level.

Individual level randomization.

No

Our goal is to run an initial pilot with 300 participants first to check whether there are any issues with the instructions, code or the software when using a representative sample in Prolific. If there are no issues, this sample will be combined with the actual experiment. Then, we plan to run the experiment with 1,500 participants. 1,500 is the largest number of participants that Prolific can deliver when using a U.S. representative sample.

1,800 (300 + 1,500) participants.

For Lists A, we will have 900 participants in the treatment group and 900 participants in the control group. For Lists B, we will again have 900 participants in the treatment and control groups each. However, since we are using a double list experiment technique (which incorporates a within-subject component), participants who are in List A (List B) treatment group will naturally be in List B (List A) control group and vice versa.