Examining responses to advertising on misinformation websites

We conduct an online survey experiment to measure how people respond to information about advertising on misinformation websites. Using an incentive-compatible design, we measure how people change their consumption and voice concerns about advertising company and platform practices in response to the information provided.

For additional details, see the attached plan.

2022-08-29

2022-09-29

Our main outcome of interest is whether participants switch their gift card preference, i.e. whether participants select a different gift card after the information treatment than their top choice indicated prior to the information treatment.

See attached plan.

Participants are given the option to sign a real online petition. For this outcome, we record a series of variables. We first have a variable which indicates whether people have the intention to sign the petition. Additionally, we record whether people clicked on the petition link provided and whether they report having signed the petition. Finally, we can count the number of signatures on the petition pages.

Additionally, we record people's stated preferences and self-reported attitudes about misinformation and company practices.

The petition outcome serves two purposes. First, it allows us to measure another way in which people may respond to our information treatments other than changing their consumption behavior. Second, since participants must choose between signing either company or platform petitions, this outcome allows us to measure whether, across our treatments, people hold advertising companies more responsible than the digital ad platforms that automatically place ads for companies. Apart from measuring the proportions of our petition outcomes across our randomized groups, we will also measure the difference between the company and platform petition outcomes.

To measure stated preferences, we ask participants about their degree of agreement to general statements about misinformation on a seven-point scale ranging from "strongly agree'' to "strongly disagree''. We similarly record self-reported attitudes towards respondents' top choice gift card company. We also measure how people's beliefs about the role played by digital ad platforms change following our information treatments.

See attached plan.

After eliciting demographic information and news preferences, we inform participants that one in five (i.e. 20% of all respondents) who
complete the survey will be offered a $25 gift card from a company of their choice out of six company options. Respondents are asked to rank the six different gift card companies on a scale from their first choice (most preferred) to their sixth choice (least preferred). The order in which the six companies are presented is randomized at the respondent level. We then ask participants to confirm which gift card they would like to receive, if they are selected. This question serves to check whether respondents have consistent preferences regardless of the type of question asked to elicit their most preferred gift card. Additionally, we ask respondents to assign weights to each of the six gift card options. This question gives respondents greater flexibility by allowing them to indicate the possibility of indifference or no preference (i.e., equal weights) between any set of options. Respondents then report how frequently they have used each of the six companies in the past 12 months.

All participants in the experiment are provided with baseline information on misinformation and advertising. Participants are then randomized into five groups to receive different information treatments, which are all based on factual information from prior research and our data. We use an active control design in order to isolate the effect of providing information relevant to the practice of specific companies on people's behavior.

Our experiment includes the following five randomized groups:

(1) Control: Participants in the control group are given generic information based on prior research that is unrelated to any specific company but relevant to topic of news and misinformation.
(2) T1 (Company only): Participants are given factual information that ads from their top choice gift card company appeared on misinformation websites in the recent past.
(3) T2 (Platform only): Participants are given factual information that companies that used digital ad platforms were about 10 times more likely to appear on misinformation websites than companies that did not use such platforms in the recent past.
(4) T3 (Company and platform): This arm combines information from T1 and T2. Similar to T1, participants are given factual information that ads from their top choice gift card company appeared on misinformation websites in the recent past. Additionally, we inform participants that their top choice company used digital ad platforms and that companies that used such platforms were about 10 times more likely to appear on misinformation websites than companies that did not use digital ad platforms.
(5) T4 (Company ranking): Participants are given factual information that ads from all six gift card companies appeared on misinformation websites in the recent past along with a ranking based on the order of their intensity of advertising on misinformation websites. We personalize these rankings based on data from different years (i.e. 2019, 2020 or 2021) such that the respondents' top gift card choice company does not appear last in the ranking (i.e. is not the company that advertises least on misinformation websites) and in most cases, advertises more intensely on misinformation websites than its potential substitute in the same company category (fast food, food delivery or ride-sharing).

After the information treatment, all participants are asked to make their final gift card choice from the same six options they were shown earlier. To ensure incentive compatibility, participants are told that those who are randomly selected to receive a gift card will be offered the gift card of their choice at the end of our study. Additionally, participants are given the option to sign a real online petition and report their preferences and attitudes.

See attached plan.

Randomization will be carried by the survey software - Qualtrics.

Randomization will be at the individual level.

No

The number of clusters will be the number of individuals recruited for the survey experiment.

We expect about 3750 individuals in the study.

750 individuals in each randomized group