Advertising and the Cost of Search: Evidence from an Online Field Experiment

Participants were randomly assigned, at the time of enrollment, to one of three online advertising environments, which they then experienced during ordinary, unscripted browsing for a three-month period.

In the first condition, display advertising was suppressed: participants installed a browser extension that blocks advertisements, so that ads were largely absent from the pages they visited.

In the second condition, participants received advertising, but not personalized advertising: they installed an extension that blocks third-party tracking, with the result that the ads they saw were selected on the basis of the content of the page being viewed rather than on the basis of their own browsing history, inferred interests, or demographic profile.

In the third condition, participants browsed the ordinary web and received
personalized, behaviorally targeted advertising, as any user would in the absence of the study.

The intervention is therefore a change in the advertising environment itself, applied continuously to the participant's own browsing rather than to a constructed task. Participants were not instructed to shop, to search, or to behave in any particular way, and no advertisement was created, purchased, or placed by the research team. All conditions were experienced in participants' own browsers, on both desktop and mobile, over the same three-month window.

2025-05-13

2027-05-13

The object of this study is the cost of search: what a consumer expends, in effort and in elapsed time, between forming a purchase intent and completing or abandoning it. Our outcomes are organized around that interval. The first group asks whether advertising changes which intents are formed at all; the remaining groups condition on an intent having been formed and ask what it costs to carry it out.

Intent formation. 1a. Probability that a browsing session contains at least one purchase intent event. 1b. Number of distinct purchase intents formed per participant-day. 1c. Share of intents that are ad-originated, meaning the intent event is preceded within the same session by a click on an advertisement leading to the merchant on which the intent is recorded. 1d. Share of intents that are resumptions of an intent formed in an earlier session, as against newly initiated intents.
Search breadth, conditional on intent. 2a. Number of distinct search queries issued between the intent event and its resolution. 2b. Number of distinct merchant sites at which a product page is viewed over the same interval. 2c. Number of distinct websites of any kind visited over the same interval.
Search duration, conditional on intent. 3a. Active browsing time, in minutes, between the intent event and its resolution. 3b. Time spent per page visit, separately for merchant and non-merchant pages. 3c. Session duration for sessions containing an intent event.
Depth of the purchase funnel, conditional on intent. 4a. Number of stages reached in the funnel, from product page to cart to checkout to completed order. 4b. Number of distinct merchants at which the cart stage is reached.
Intent resolution. 5a. Probability that an intent ends in a completed purchase. 5b. Probability that an intent reaching the cart stage ends in a completed purchase, the complement of cart abandonment. 5c. Probability that an intent is abandoned without resolution by the end of the observation window.
Time to goal. 6a. Elapsed calendar time, in hours, between the intent event and the completed purchase that resolves it. 6b. Number of distinct browsing sessions spanned by that interval. 6c. Number of days on which activity related to the intent is observed.
Realized satisfaction. 7a. Self-reported satisfaction with the purchased product's quality, price and brand, and with the browsing experience through which it was bought, each on a seven-point Likert scale.

Each outcome is compared across the three randomly assigned conditions. We test separately the effect of advertising exposure, by contrasting each advertising condition with the ad-blocking condition, and the effect of personalization, by contrasting the targeted with the contextual condition.

The intent event is the central construct and everything else is defined relative to it. A purchase intent is recorded at the first moment a participant issues a search query through the internal search function of a merchant website. That act is observable, timestamped, and specific to a product category, and it marks the point at which the consumer has committed attention to acquiring something rather than merely browsing. Merchant status of a host is assigned by the union of an algorithmic e-commerce classifier and a curated list of known merchants, a host qualifying under either criterion being treated as a merchant. An intent is identified by the participant, the merchant category and the search terms, and it persists across sessions until it is resolved by a completed purchase of a matching item or until the observation window closes, at which point it is treated as abandoned and right-censored.

This definition is what allows the cost of search to be measured as an interval rather than as a session-level average, and it is the methodological contribution the study is meant to make. The alternative convention in the literature, which treats the browsing session as the unit and reports mean search activity within it, cannot distinguish a consumer who reaches a goal quickly from one who never had a goal, and it discards entirely the case of a goal formed on one day and completed on another. Because ninety-day windows are observed for every participant, we can follow an intent across sessions.

An intent is classified as ad-originated when the intent event is preceded, within the same session and within thirty minutes, by a page visit arriving through an advertising click, identified from the campaign and referral parameters carried in the destination URL, on the same merchant at which the intent is recorded. The distinction matters for welfare and not only for measurement: an intent that the consumer arrived with is a goal whose fulfilment counts as a benefit, whereas an intent manufactured by the advertisement encountered en route is not obviously one. We therefore report the primary outcomes overall and separately for self-originated and ad-originated intents, and we treat the share of intents that are ad-originated as a primary outcome in its own right. Because the ad-blocking condition mechanically permits very few advertising clicks, the contrast that carries information for this outcome is between targeted and contextual advertising.

A browsing session is a sequence of page visits by a single participant separated by no more than thirty minutes of inactivity, following the standard Google Analytics convention, and page visits recorded inside iframes are excluded. Time per page is the interval between consecutive recorded visits within a session, and the last visit of a session is not assigned a duration. Active browsing time within an intent interval is the sum of page durations attributable to that intent, which distinguishes effort from the calendar time in time-to-goal: a consumer may take four days and twenty minutes, or four days and four hours, and these are different search costs.

Funnel stages are assigned by URL pattern within merchant sites: product browsing, cart for paths containing cart, basket, bag or the Amazon cart path, checkout for paths containing checkout, payment or place-order, and completed purchase for paths containing thank-you or order-received markers or an order-key parameter. Purchase detection excludes a documented set of false positives, including review submission confirmations, e-book and library lending confirmations, registry thank-you pages, credit-report completions, and pages flagged as already purchased. Purchases resolving an intent are matched to the intent by merchant, product terms and timing, and corroborated against order history where the merchant is Amazon. Purchases for which no antecedent intent event can be identified are excluded from the time-to-goal outcome and reported separately, since their exclusion is potentially endogenous to treatment.

Counts and durations are strongly right-skewed. Count outcomes are analyzed with negative binomial models and, as a robustness check, in logs of one plus the count; durations are analyzed in logs of one plus the value in minutes; time-to-goal is analyzed both in logs and with duration models that treat unresolved intents as right-censored. Distributional differences are additionally assessed non-parametrically, by Kruskal-Wallis tests across the three conditions, by pairwise Mann-Whitney tests, and by comparison of the full cumulative distribution functions.

Standard errors are clustered at the participant level throughout, since treatment is assigned at that level and intents, sessions and page visits are repeated observations on the same participant. Because we specify a broad set of primary outcomes, we group them into the seven families above and report for each family both unadjusted p-values and p-values adjusted for multiple hypothesis testing within the family by the Romano-Wolf stepdown procedure. We additionally construct one standardized index per family, as the equally weighted average of its components after each has been signed so that higher values denote higher search cost and standardized to the mean and standard deviation of the ad-blocking condition. The seven family indices constitute the headline test, and the component outcomes are reported beneath them.

Estimates are reported without covariates and, for precision, with a pre-specified set of participant characteristics recorded at enrollment: age, gender, race and ethnicity indicators, education, household income, employment status, self-assessed IT skill, and mobile operating system. Covariate adjustment is not required for identification, since assignment is random.

Purchase volume and value, measured from order history at the participant-week level: number of items ordered, number of orders, total order value, and the logarithm of item price. Cancellations and returns as a share of items ordered. Composition of purchases: the share of orders placed with merchants the participant had not previously visited, and the share placed with the merchant at which the intent originated, which measures whether search that begins in one place ends in another. Session frequency, as the number of browsing sessions initiated per participant-day. Allocation of attention across site categories, as the share of page visits and of time accounted for by merchant sites, search engines, social media, publishers, entertainment and other categories. Ad-click behavior, as the number of page visits arriving through an advertising click and the share of merchant visits so originated. Heterogeneity of the primary effects by age, household income, education, self-assessed IT skill and stated privacy concern, all recorded at enrollment. Pre-post comparison of purchase behavior using the three months of order history preceding enrollment as a within-participant baseline.

Order-history outcomes are constructed from purchase records collected at months one, two and three of participation, which extend back three months before enrollment. This permits a difference-in-differences specification in which the participant's own pre-enrollment purchasing serves as the baseline and the indicator Post equals one for purchases made during the study period, with treatment effects identified by the interaction of condition with Post. That specification is reported alongside the randomization-based comparison and not in place of it: its value is precision, since randomization alone identifies the effect.

The share of orders placed with a merchant other than the one at which the intent originated is the outcome that connects search cost to market structure. If advertising raises search costs by diverting consumers, it should also displace purchases away from where the goal was formed, and the direction of that displacement, toward larger or smaller merchants, is informative about who bears the cost.

Heterogeneity analyses are exploratory. We report them with adjusted p-values and we do not treat any subgroup result as confirmatory.

The study is a randomized field experiment on the effect of online advertising on consumer search. Participants are U.S.-based internet users recruited online. At enrollment each participant installs a browser extension on desktop and, where applicable, on mobile, and is randomly assigned by the study software to one of three advertising environments, which then applies to all of their browsing for a three-month period: display advertising suppressed; contextual, non-personalized advertising only; or the ordinary web with personalized, behaviorally targeted advertising.

Participants are not assigned tasks. They browse as they otherwise would, and the extension records their browsing, the advertisements they encounter, and, periodically, their purchase history and survey responses. Because assignment is random and the observation window is identical across conditions, differences in subsequent browsing and purchasing between conditions identify the causal effect of the advertising environment.

The analysis registered here concerns the cost of search: how much effort, how many steps, and how much elapsed time separate the formation of a purchase goal from its completion, and whether these differ by advertising condition. Outcomes are measured at the level of the browsing session and of the individual purchase, with inference clustered at the participant level.

Not available

Randomization is performed on Qualtrics. At the moment a participant passes the screening survey and start the onboarding survey, Qualtrics randomly assigns them to one of the three conditions using a pseudo-random number generator, with equal assignment probabilities across the three conditions. No stratification or blocking was used, and no member of the research team was involved in or able to influence individual assignments.

The realized numbers of participants observed in each condition are not exactly equal. That imbalance arises after assignment, from differential completion of installation and from attrition during the three-month period, and not from the assignment probabilities, which are equal by construction. Enrollment, installation completion and attrition are reported by condition, and differential attrition is tested and bounded as described in the experimental design.

The individual participant. Each participant is assigned to a single advertising condition at enrollment and remains in that condition for the whole of their three-month observation window. There is only one level of randomization: no group, household or session-level assignment is used. Outcomes measured at the intent, session, page or purchase level are repeated observations on the randomized unit, and inference accounts for this by clustering standard errors at the participant level.

No

1,200 individual participants. The design is not clustered: the participant is
both the unit of randomization and the cluster for inference.

1,200 participants, each observed for three months of browsing. Outcomes are measured at several levels of aggregation beneath the participant. At the planned sample size we expect on the order of 300,000 browsing sessions (approximately 3 sessions per participant-day over a ninety-day window), of which on the order of 8,000 are product-search sessions containing at least one search query issued on a merchant website; several million individual page visits; and several thousand completed purchases with associated order-history records and satisfaction responses. These projections extrapolate observed per-participant rates in the sample enrolled to date and are indicative rather than guaranteed.

400 participants assigned to the ad-blocking condition (no display advertising)
400 participants assigned to the anti-tracking condition (contextual, non-
personalized advertising only)
400 participants assigned to the control condition (ordinary web, personalized behaviorally targeted advertising)

Assignment probabilities are equal across the three conditions. Realized arm sizes will differ from these targets because of differential completion of installation and attrition after assignment; realized counts by condition are reported with the results.

Calculations assume 400 participants per arm, two-sided tests at the five percent level, and eighty percent power, and account for the clustering of session-level observations within participants. Continuous and count outcomes measured at the session level. With approximately 7 product-search sessions per participant and an intra-participant correlation of 0.20, the design effect is 2.20 and the effective sample is approximately 1,270 sessions per arm. The minimum detectable effect for a pairwise comparison of two arms is 0.11 standard deviations of the outcome. At an intra-participant correlation of 0.10 the effective sample is approximately 1,750 sessions per arm and the minimum detectable effect is 0.10 standard deviations. Goal completion. Taking the probability that a product-search session ends in a purchase to be 7.1 percent in the ad-blocking condition, and with the same clustering assumptions, the minimum detectable difference in that probability is 2.9 percentage points, equivalent to a forty percent change relative to the ad-blocking baseline. Outcomes measured once per participant. For outcomes collapsed to the participant level, where clustering is not at issue and the effective sample is 400 per arm, the minimum detectable effect is 0.20 standard deviations for a continuous outcome and 5.1 percentage points for a binary outcome with a 7.1 percent baseline. Family indices. The standardized indices defined in the primary outcomes explanation aggregate correlated component outcomes and are correspondingly better powered; at a within-family correlation of 0.5 among six components, the minimum detectable effect on the index is approximately 0.07 standard deviations at the session level. The intra-participant correlations used here are assumptions, not estimates from the study data.