Cookie Consent Dark Patterns and Website Competition

Last registered on January 19, 2024

Pre-Trial

Trial Information

General Information

Title
Cookie Consent Dark Patterns and Website Competition
RCT ID
AEARCTR-0012862
Initial registration date
January 19, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
January 19, 2024, 2:24 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
Boston University

Other Primary Investigator(s)

Additional Trial Information

Status
In development
Start date
2024-01-19
End date
2024-02-07
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Companies often obfuscate cookie preference policies and consent forms in order to nudge users to share their data more than they would otherwise do. These practices, known as “dark patterns,” are pervasive despite their effects being poorly understood. With the use of Webmunk, a browser extension developed for research studies of this type, we explore the effects of dark patterns on consumers’ privacy choices and the effects that such choices have on the type and quantity of ads they receive. We are particularly interested in how the effects of dark patterns differ across large and small companies, and the ensuing firms’ competitive advantage in consumer data.
External Link(s)

Registration Citation

Citation
Fradkin, Andrey. 2024. "Cookie Consent Dark Patterns and Website Competition." AEA RCT Registry. January 19. https://doi.org/10.1257/rct.12862-1.0
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2024-01-19
Intervention End Date
2024-02-07

Primary Outcomes

Primary Outcomes (end points)
Consent choice: per domain, over time, by domain characteristics. In particular, the user choices that we are interested in include the following:
Accepting all cookies;
Clicking on “Cookie settings” and making more granular choices;
Rejecting all cookies;
Closing the banner without making a choice.
Accepting the 'default' (the top-most option in the banner).
Primary Outcomes (explanation)

> We are particularly interested in how consent designs affect website competition. To this end, we plan to focus on the following website characteristics to examine site-level heterogeneity in the treatment effects:
>
> - Website popularity.
> - Website category.
> - Familiarity with the website.

> We have 6 treatment conditions, described in more detail in the ``Experiment Design'' Section.
> Our Baseline specification is as follows, where we index treatments 1 through 6, with the omitted category being treatment 1 (Accept all, Reject all, Cookie settings):

$$ y_{id} = \sum_{k \in \{2, 3, 4, 5, 6\}} \tau_{k} 1(Treat_{id} = k) + \gamma_{i} + \mu (X_{d} - \bar{X}) + \epsilon_{id} $$

> In this specification, $i$ denotes a participant and $d$ denotes a website, or domain. We care about several outcomes $y$, which are dummy variables for each of the various consent choices: accepting all cookies; clicking on “Cookie settings” and making granular choices; rejecting all cookies; closing the banner without making a choice. Note that when users click on "Cookie Settings" and then make a choice consistent with either rejecting or accepting all cookies, we classify such choice as either accepting or rejecting all cookies. We include fixed effects for individuals $\gamma_{i}$ and domain characteristics $\mu (X_{d})$ (such as website category).


> Our tests of theoretical mechanisms will consist of hypothesis tests about differences among treatment arms, which correspond to differing mechanisms. The three mechanisms we are interested in evaluating are the following: removing options from the available choices; re-ordering options; emphasizing particular options through colors. The treatment arms described in the ``Experiment Design'' Section help us separate the effects of these strategies on consumer choice.

> We will conduct this analysis separately for the websites visited during the initial survey and for the websites organically browsed by the participant. For the analysis of the cookie choices for the websites visited during the initial survey, we will replace $\mu (X_{d})$ with website fixed effects.

> Our heterogeneity specifications will interact the treatment indicators in the above specification with proxies for website popularity, website category, and participant's familiarity with the website, as in the specification below. Each dimension of heterogeneity will be explored separately (i.e., in different regressions).
> We will conduct this analysis separately for the survey websites and for the organic websites that individuals browse after the survey.

$$ y_{id} = \sum_{k \in \{2, 3, 4, 5, 6\}} \tau_{k} 1(Treat_{id} = k) + \rho_{k} 1(Treat_{id} = k)(X_{d} - \bar{X}) + \gamma_{i} + \mu (X_{d} - \bar{X}) + \epsilon_{id} $$

> The specification $(X_{d} - \bar{X})$ will be simply $X_{d}$ where the dimension of heterogeneity is identified by a dummy variable.


X
ˉ
) will be simply


X
d

where the dimension of heterogeneity is identified by a dummy variable.

Secondary Outcomes

Secondary Outcomes (end points)

* Secondary analysis (end points, heterogeneity)

> We also plan to look at the following subject characteristics when examining subject-level heterogeneity:
>
> - Demographics: education, income, gender, age cohort, race and ethnicity; states that they live in (esp. if they are in a state that already enacted privacy laws)
> - Time spent online per day.
> - Stated reason for sharing vs. declining to share data with websites. This will involve classification of free form text responses in survey 2.
> - Attention while in the study: proxy measures include their time spent in answering the surveys and their response to our attention check questions.
> - Heterogeneity by order in which the choice was made in the survey.

> - Conditional on being able to correctly classify tracking cookies in our data, which we are still determining, we will look at how different treatments affect the composition of consumers willing to share data to a domain (e.g. whether they are potential or existing customers).

> We consider these to be secondary outcomes because our main focus is on site-level heterogeneity and not subject-level heterogeneity. However, to the extent that controlling for subject characteristics helps reduce the noise and improves the power of our estimate, and given the fact that participants are randomized into different treatment conditions, we will include these characteristics in our richest model specifications.

> We will also analyze the experiment by limiting the data to pairs of treatment arms, since this may be easier to explain and will likely result in similar estimates.

> To analyze the effects of frequency of pop-ups, using data from their organic browsing, we will consider whether individuals who were exogenously exposed to more frequent pop-ups made similar choices to those who were exposed to less frequent pop-ups.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The treatment arms are represented by different cookie consent banner designs, which vary in the number of options available, the color, and the order of the options.
Experimental Design Details
The treatment arms are represented by different cookie consent banner designs, which vary in the number of options available, the color, and the order of the options. Where the color is not specified, it is assumed to be blue. The treatment arms are specified as follows:

Treatment 1: three options: "Accept all" on top; "Reject all" in the middle; “Cookie settings” at the bottom.

Treatment 2: two options: “Accept all” on top; “Cookie settings” at the bottom;

Treatment 3: two options: “Reject all” on top; “Cookie settings” at the bottom;

Treatment 4: three options: “Cookie settings” on top; "Accept all" in the middle, “Reject all” at the bottom.

Treatment 5: three options: "Reject all" on top; "Accept all" in the middle; “Cookie settings” at the bottom.

Treatment 6: three options: "Accept all" on top; "Reject all" in the middle in gray; "Cookie settings" at the bottom in grey.

Status quo: these are organic banners designed by the websites themselves. Note, these banners will be shown to participants only when they are not shown our own banners. This will occur in the second phase of the study, when participants are free to browse the web while keeping the extension installed. During organic browsing, our extension will display our banners every 10 minutes or 60 minutes, depending on the randomization. Between the 10-minute or 60-minute time intervals, organic banners that websites choose to display are shown to the participant. Since not all websites in the US have cookie consent banners, the number of observed choices will be less than the number of domains browsed by a given user.

These treatment conditions allow us to test a few dimensions of dark patterns that websites engage in: which options are available, in which order, and which emphasis is placed on each of them. In particular:

Comparing Treatment 1 vs. Treatments 2 and 3 allows us to test the effect of hiding options on consumer choice.

Comparing Treatment 1 vs. 4 allows us to test the ordering effect for the “Cookie settings" option.

Comparing Treatment 1 vs. 5 allows us to test the ordering effect between accept and reject.

Comparing Treatment 1 vs. 6 allows us to test the effect of graying out options that reduce data sharing.

Lastly, comparing Treatment 1 vs. organic choices gives us the effect of nudging in the wild compared to a relatively “neutral” condition.

During this second phase of the study, we display the consent popups that we design up to every 10 minutes or up to every 60 minutes, where this frequency is randomly assigned at the user level.
Randomization Method
Computer randomization
Randomization Unit
For the banners, person by domain. For the frequency, by person.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
We will recruit 800 individuals on Prolific. We expect some leakage in the process of correctly installing the extension so that we only have browsing data for 80% of participants (640). We powered the study so that we could detect a statistically significant 5 percentage point interaction effect between treatment and site type.
Sample size: planned number of observations
We expect to have organic browsing data for 640 individuals. During phase one, we will have 21*640 = 13440 observations of consent choices. During phase two, since users are browsing organically, we can't know ex ante.
Sample size (or number of clusters) by treatment arms
We will assign arms with equal probability. This will result in a balanced number, with some slight differences due to chance.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
We powered the study so that we could detect a statistically significant 5 percentage point interaction effect between treatment and site type.
IRB

Institutional Review Boards (IRBs)

IRB Name
The Economic Implications of Digital Nudges: Applications to Privacy, Tracking, and Search Engines.
IRB Approval Date
2023-02-22
IRB Approval Number
N/A (Exempt)

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials