Examining responses to advertising on misinformation websites

In our first study, we measure how people respond to information about advertising on misinformation websites. Using an incentive-compatible design, we measure how people change their consumption and voice concerns about advertising company and platform practices in response to the information provided. For additional details about this experiment, see the attached plan.

In our second study, we measure how employees and decision-makers within companies respond to information about advertising on misinformation websites. We measure their baseline beliefs and preferences relevant to advertising on misinformation websites, and measure their behavior in response to the information provided.

2022-08-29

2023-02-15

Study 1: Our main outcome of interest is whether participants switch their gift card preference, i.e. whether participants select a different gift card after the information treatment than their top choice indicated prior to the information treatment. For additional details, see attached plan.

Study 2: Our key outcomes variables are demand for information (i.e. a binary variable that takes a value of one when participants choose to receive information on which platforms least frequently place companies' ads on misinformation websites and zero otherwise) and donation preference (i.e. a binary variable that takes a value of one when participants prefer a donation to a non-profit organization that helps digital ad platforms identify and reduce advertising on misinformation websites and zero otherwise).

Study 1: Participants are given the option to sign a real online petition. For this outcome, we record a series of variables. We first have a variable which indicates whether people have the intention to sign the petition. Additionally, we record whether people clicked on the petition link provided and whether they report having signed the petition. Finally, we can count the number of signatures on the petition pages. Additionally, we record people's stated preferences and self-reported attitudes about misinformation and company practices.

Study 2: We measure participants' posterior beliefs about the role played by digital ad platforms in placing ads on misinformation websites following our information treatment.

Study 1: The petition outcome serves two purposes. First, it allows us to measure another way in which people may respond to our information treatments other than changing their consumption behavior. Second, since participants must choose between signing either company or platform petitions, this outcome allows us to measure whether, across our treatments, people hold advertising companies more responsible than the digital ad platforms that automatically place ads for companies. Apart from measuring the proportions of our petition outcomes across our randomized groups, we will also measure the difference between the company and platform petition outcomes. To measure stated preferences, we ask participants about their degree of agreement to general statements about misinformation on a seven-point scale ranging from "strongly agree'' to "strongly disagree''. We similarly record self-reported attitudes towards respondents' top choice gift card company. We also measure people's beliefs about the role played by digital ad platforms following our information treatments on a continuous scale. See attached plan.

Study 2: Posterior beliefs are measured on a continuous scale. Participants are told about the average number of advertising companies whose ads appear per month on misinformation websites that do not use digital ad platforms. They are then asked to estimate the average number of advertising companies per month whose ads appear per month on misinformation websites that do use digital ad platforms. We will winsorize the responses collected to remove outliers.

Study 1:
After eliciting demographic information and news preferences, we inform participants that one in five (i.e. 20% of all respondents) who
complete the survey will be offered a $25 gift card from a company of their choice out of six company options. Respondents are asked to rank the six different gift card companies on a scale from their first choice (most preferred) to their sixth choice (least preferred). The order in which the six companies are presented is randomized at the respondent level. We then ask participants to confirm which gift card they would like to receive, if they are selected. This question serves to check whether respondents have consistent preferences regardless of the type of question asked to elicit their most preferred gift card. Additionally, we ask respondents to assign weights to each of the six gift card options. This question gives respondents greater flexibility by allowing them to indicate the possibility of indifference or no preference (i.e., equal weights) between any set of options. Respondents then report how frequently they have used each of the six companies in the past 12 months.

All participants in the experiment are provided with baseline information on misinformation and advertising. Participants are then randomized into five groups to receive different information treatments, which are all based on factual information from prior research and our data. We use an active control design in order to isolate the effect of providing information relevant to the practice of specific companies on people's behavior.

Our experiment includes the following five randomized groups:

(1) Control: Participants in the control group are given generic information based on prior research that is unrelated to any specific company but relevant to topic of news and misinformation.
(2) T1 (Company only): Participants are given factual information that ads from their top choice gift card company appeared on misinformation websites in the recent past.
(3) T2 (Platform only): Participants are given factual information that companies that used digital ad platforms were about 10 times more likely to appear on misinformation websites than companies that did not use such platforms in the recent past.
(4) T3 (Company and platform): This arm combines information from T1 and T2. Similar to T1, participants are given factual information that ads from their top choice gift card company appeared on misinformation websites in the recent past. Additionally, we inform participants that their top choice company used digital ad platforms and that companies that used such platforms were about 10 times more likely to appear on misinformation websites than companies that did not use digital ad platforms.
(5) T4 (Company ranking): Participants are given factual information that ads from all six gift card companies appeared on misinformation websites in the recent past along with a ranking based on the order of their intensity of advertising on misinformation websites. We personalize these rankings based on data from different years (i.e. 2019, 2020 or 2021) such that the respondents' top gift card choice company does not appear last in the ranking (i.e. is not the company that advertises least on misinformation websites) and in most cases, advertises more intensely on misinformation websites than its potential substitute in the same company category (fast food, food delivery or ride-sharing).

After the information treatment, all participants are asked to make their final gift card choice from the same six options they were shown earlier. To ensure incentive compatibility, participants are told that those who are randomly selected to receive a gift card will be offered the gift card of their choice at the end of our study. Additionally, participants are given the option to sign a real online petition and report their preferences and attitudes.

Study 2:
We first elicit participants' current employment status. All those who are working in some capacity are allowed to continue the survey whereas the rest of the participants are screened out. After asking for participants' main occupation, all participants in the experiment are provided with baseline information on misinformation and advertising. To record their stated preferences, participants are then asked to share how much they agree or disagree with a series of statements on a scale from strongly disagree to strongly agree. The order in which these statements are presented is randomized for all participants. To record their revealed preferences, participants are asked three questions in a randomized order: 1) Information demand about consumer responses, i.e. whether they would like to learn how people respond to companies whose ads appear on misinformation websites, 2) Ad check, i.e. whether they would like to know about their own company's ads appearing on misinformation websites in the recent past, and 3) Demand for solution, i.e. whether they would like to sign up for an information session on how companies can manage whether their ads appear online. For information demand about consumer responses, participants are told that they would be provided with this information at the end of our survey if they choose to receive it. For the ad check and demand for solution, participants are told that they would receive this information in a follow-up email after survey completion.

Next, we record participants' baseline beliefs about the role of companies and platforms in placing ads on misinformation websites. Participants are then randomized into a treatment group, which receives information about the role of digital ad platforms in placing ads on misinformation websites, and a control group, which does not receive this information. Following this information intervention, we inquire participants' posterior beliefs about the role played by digital ad platforms in placing ads on misinformation websites as well as their demand for information and donation preference. We then ask participants to answer a few additional questions about their background, preferences, place of work, and feedback on the survey. Participants who opted to learn more in our information demand questions are then provided with this information at the end of our survey.

For additional details, please see the attached plan.

Randomization will be carried by the survey software - Qualtrics.

Randomization will be at the individual level.

No

The number of clusters will be the number of individuals recruited for the survey experiment.

Study 1: We expect about 3750 individuals in the study. Study 2: We expect to send out email invitations to approximately 120,000 individuals. Assuming a click rate of 1% and a completion rate of 50%, we expect about 600 responses in the study.

Study 1: Participants are evenly randomized into five groups. We expect approximately 750 individuals in each randomized group.
Study 2: Participants are evenly randomized into two groups. We expect approximately 300 individuals in each randomized group.