Cookie Consent Dark Patterns and Website Competition

Last registered on January 19, 2024

Pre-Trial

Trial Information

General Information

Title
Cookie Consent Dark Patterns and Website Competition
RCT ID
AEARCTR-0012862
Initial registration date
January 19, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
January 19, 2024, 2:24 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
Boston University

Other Primary Investigator(s)

Additional Trial Information

Status
In development
Start date
2024-01-19
End date
2024-02-07
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Companies often obfuscate cookie preference policies and consent forms in order to nudge users to share their data more than they would otherwise do. These practices, known as “dark patterns,” are pervasive despite their effects being poorly understood. With the use of Webmunk, a browser extension developed for research studies of this type, we explore the effects of dark patterns on consumers’ privacy choices and the effects that such choices have on the type and quantity of ads they receive. We are particularly interested in how the effects of dark patterns differ across large and small companies, and the ensuing firms’ competitive advantage in consumer data.
External Link(s)

Registration Citation

Citation
Fradkin, Andrey. 2024. "Cookie Consent Dark Patterns and Website Competition." AEA RCT Registry. January 19. https://doi.org/10.1257/rct.12862-1.0
Experimental Details

Interventions

Intervention(s)
Intervention (Hidden)

We use a customized browser extension to experimentally vary the cookie consent banners that participants face as they browse websites, and examine whether differences in the cookie banner design change their cookie sharing decisions. There is a two stage randomization. The first randomization (of interface) occurs at the user by domain level. The second randomization (of banner frequency) occurs at the user level.

We first ask participants screening questions. We then ask them to install a browser extension that allows us to manipulate what they see while browsing, and to track their browsing activity. Participants are asked to visit a list of 20 websites we chose ex-ante. For each of the websites we ask participants to stay on the website until a cookie consent banner appears, and then indicate their cookie sharing choices among those available in the popup banner. The consent banners appear over the page. When a user selects a choice, the browser extension looks for an organic cookie banner and attempts to make that choice on the user's behalf. Note, this process is imperfect and we tell users that 'the extension will try to pass on your choices to the website.' If there is no detected organic pop-up, then the extension does not attempt to enforce the user's selection. After users complete the survey task, the second phase of the study begins. Participants are free to continue with their normal browsing activities while keeping the browser extension installed for seven days. During this second phase of the study, we display the consent popups that we design up to every 10 minutes or up to every 60 minutes, where this frequency is randomly assigned at the user level. For the rest of the time, we allow the websites to display their own version of the banners. We record the cookie sharing decisions that participants choose from both our own pop-ups and from organic pop-ups.

We ask the participants to fill out two surveys. The baseline survey measures participants' characteristics not already provided by the sample source that we use, participants' knowledge of the 20 websites that we ask them to visit, their attitudes towards our consent banners, and cookies in general. The endline survey measures their attitude towards privacy, beliefs about the information websites collect, and their browsing experiences while the extension was installed.

After participants finish the endline survey, we ask them to uninstall the browser extension and send them the compensation for completing the study.

We will recruit participants via Prolific. When recruiting participants via Prolific, we apply the following filters:
- Living in the US
- Fluent in English
- Age: over 18
- Device used for the study: laptop or desktop
- Not having participated in our previous pilots
- Computer operating system: macOS or Windows
Intervention Start Date
2024-01-19
Intervention End Date
2024-02-07

Primary Outcomes

Primary Outcomes (end points)
Consent choice: per domain, over time, by domain characteristics. In particular, the user choices that we are interested in include the following:
Accepting all cookies;
Clicking on “Cookie settings” and making more granular choices;
Rejecting all cookies;
Closing the banner without making a choice.
Accepting the 'default' (the top-most option in the banner).
Primary Outcomes (explanation)

> We are particularly interested in how consent designs affect website competition. To this end, we plan to focus on the following website characteristics to examine site-level heterogeneity in the treatment effects:
>
> - Website popularity.
> - Website category.
> - Familiarity with the website.

> We have 6 treatment conditions, described in more detail in the ``Experiment Design'' Section.
> Our Baseline specification is as follows, where we index treatments 1 through 6, with the omitted category being treatment 1 (Accept all, Reject all, Cookie settings):

$$ y_{id} = \sum_{k \in \{2, 3, 4, 5, 6\}} \tau_{k} 1(Treat_{id} = k) + \gamma_{i} + \mu (X_{d} - \bar{X}) + \epsilon_{id} $$

> In this specification, $i$ denotes a participant and $d$ denotes a website, or domain. We care about several outcomes $y$, which are dummy variables for each of the various consent choices: accepting all cookies; clicking on “Cookie settings” and making granular choices; rejecting all cookies; closing the banner without making a choice. Note that when users click on "Cookie Settings" and then make a choice consistent with either rejecting or accepting all cookies, we classify such choice as either accepting or rejecting all cookies. We include fixed effects for individuals $\gamma_{i}$ and domain characteristics $\mu (X_{d})$ (such as website category).


> Our tests of theoretical mechanisms will consist of hypothesis tests about differences among treatment arms, which correspond to differing mechanisms. The three mechanisms we are interested in evaluating are the following: removing options from the available choices; re-ordering options; emphasizing particular options through colors. The treatment arms described in the ``Experiment Design'' Section help us separate the effects of these strategies on consumer choice.

> We will conduct this analysis separately for the websites visited during the initial survey and for the websites organically browsed by the participant. For the analysis of the cookie choices for the websites visited during the initial survey, we will replace $\mu (X_{d})$ with website fixed effects.

> Our heterogeneity specifications will interact the treatment indicators in the above specification with proxies for website popularity, website category, and participant's familiarity with the website, as in the specification below. Each dimension of heterogeneity will be explored separately (i.e., in different regressions).
> We will conduct this analysis separately for the survey websites and for the organic websites that individuals browse after the survey.

$$ y_{id} = \sum_{k \in \{2, 3, 4, 5, 6\}} \tau_{k} 1(Treat_{id} = k) + \rho_{k} 1(Treat_{id} = k)(X_{d} - \bar{X}) + \gamma_{i} + \mu (X_{d} - \bar{X}) + \epsilon_{id} $$

> The specification $(X_{d} - \bar{X})$ will be simply $X_{d}$ where the dimension of heterogeneity is identified by a dummy variable.


X
ˉ
) will be simply


X
d

where the dimension of heterogeneity is identified by a dummy variable.

Secondary Outcomes

Secondary Outcomes (end points)

* Secondary analysis (end points, heterogeneity)

> We also plan to look at the following subject characteristics when examining subject-level heterogeneity:
>
> - Demographics: education, income, gender, age cohort, race and ethnicity; states that they live in (esp. if they are in a state that already enacted privacy laws)
> - Time spent online per day.
> - Stated reason for sharing vs. declining to share data with websites. This will involve classification of free form text responses in survey 2.
> - Attention while in the study: proxy measures include their time spent in answering the surveys and their response to our attention check questions.
> - Heterogeneity by order in which the choice was made in the survey.

> - Conditional on being able to correctly classify tracking cookies in our data, which we are still determining, we will look at how different treatments affect the composition of consumers willing to share data to a domain (e.g. whether they are potential or existing customers).

> We consider these to be secondary outcomes because our main focus is on site-level heterogeneity and not subject-level heterogeneity. However, to the extent that controlling for subject characteristics helps reduce the noise and improves the power of our estimate, and given the fact that participants are randomized into different treatment conditions, we will include these characteristics in our richest model specifications.

> We will also analyze the experiment by limiting the data to pairs of treatment arms, since this may be easier to explain and will likely result in similar estimates.

> To analyze the effects of frequency of pop-ups, using data from their organic browsing, we will consider whether individuals who were exogenously exposed to more frequent pop-ups made similar choices to those who were exposed to less frequent pop-ups.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The treatment arms are represented by different cookie consent banner designs, which vary in the number of options available, the color, and the order of the options.
Experimental Design Details
The treatment arms are represented by different cookie consent banner designs, which vary in the number of options available, the color, and the order of the options. Where the color is not specified, it is assumed to be blue. The treatment arms are specified as follows:

Treatment 1: three options: "Accept all" on top; "Reject all" in the middle; “Cookie settings” at the bottom.

Treatment 2: two options: “Accept all” on top; “Cookie settings” at the bottom;

Treatment 3: two options: “Reject all” on top; “Cookie settings” at the bottom;

Treatment 4: three options: “Cookie settings” on top; "Accept all" in the middle, “Reject all” at the bottom.

Treatment 5: three options: "Reject all" on top; "Accept all" in the middle; “Cookie settings” at the bottom.

Treatment 6: three options: "Accept all" on top; "Reject all" in the middle in gray; "Cookie settings" at the bottom in grey.

Status quo: these are organic banners designed by the websites themselves. Note, these banners will be shown to participants only when they are not shown our own banners. This will occur in the second phase of the study, when participants are free to browse the web while keeping the extension installed. During organic browsing, our extension will display our banners every 10 minutes or 60 minutes, depending on the randomization. Between the 10-minute or 60-minute time intervals, organic banners that websites choose to display are shown to the participant. Since not all websites in the US have cookie consent banners, the number of observed choices will be less than the number of domains browsed by a given user.

These treatment conditions allow us to test a few dimensions of dark patterns that websites engage in: which options are available, in which order, and which emphasis is placed on each of them. In particular:

Comparing Treatment 1 vs. Treatments 2 and 3 allows us to test the effect of hiding options on consumer choice.

Comparing Treatment 1 vs. 4 allows us to test the ordering effect for the “Cookie settings" option.

Comparing Treatment 1 vs. 5 allows us to test the ordering effect between accept and reject.

Comparing Treatment 1 vs. 6 allows us to test the effect of graying out options that reduce data sharing.

Lastly, comparing Treatment 1 vs. organic choices gives us the effect of nudging in the wild compared to a relatively “neutral” condition.

During this second phase of the study, we display the consent popups that we design up to every 10 minutes or up to every 60 minutes, where this frequency is randomly assigned at the user level.
Randomization Method
Computer randomization
Randomization Unit
For the banners, person by domain. For the frequency, by person.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
We will recruit 800 individuals on Prolific. We expect some leakage in the process of correctly installing the extension so that we only have browsing data for 80% of participants (640). We powered the study so that we could detect a statistically significant 5 percentage point interaction effect between treatment and site type.
Sample size: planned number of observations
We expect to have organic browsing data for 640 individuals. During phase one, we will have 21*640 = 13440 observations of consent choices. During phase two, since users are browsing organically, we can't know ex ante.
Sample size (or number of clusters) by treatment arms
We will assign arms with equal probability. This will result in a balanced number, with some slight differences due to chance.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
We powered the study so that we could detect a statistically significant 5 percentage point interaction effect between treatment and site type.
IRB

Institutional Review Boards (IRBs)

IRB Name
The Economic Implications of Digital Nudges: Applications to Privacy, Tracking, and Search Engines.
IRB Approval Date
2023-02-22
IRB Approval Number
N/A (Exempt)

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials