To the Depths of the Sunk Cost: an Experiment Revisiting the Elusive Fallacy (Field Study)

2022-10-01

2022-10-31

Video engagement time: we define it to be the time between (1) when the video loads after the ads and (2) the first time the user transitions out of the video page.

We will look at heterogeneity with respect to the length of the video. We will categorize observations by video length in the following way – shorter than 3 minutes, 3-6 minutes, 6-10 minutes, 10-20 minutes, longer than 20 minutes.

We will look at the subsample of videos where we exclude music videos – which might be intended to be played in the background. We will identify those videos as ones that are more than 10 minutes long, and are in the “music” category.

Additionally, we are going to test whether the duration of exposure to factors affecting the sunk cost changes the strength of the effect. To that end, we compare the effect size in the first two weeks vs. the last two weeks.

Lastly, we will collect data on whether an ad is clicked.

The experimental design is hidden until the end of the trial.

The intervention is introduced through a browser extension that all participants install. The extension was designed to improve user experience on social media and we sample participants from the pool of users who previously experienced an intervention where the researchers varied exposure to toxic content on social media.

We rely on a within-subjects design structure, randomizing at the video level. Each person will encounter all three of the treatment conditions. For each YouTube video which contains at least one skippable ad, we randomize the wait time until a skip button appears for the first skippable ad attached to the video (see Intervention section).

Primary hypothesis (existence of the sunk cost effect):
The average video engagement time in the long condition is greater than in the short condition.

Secondary hypothesis (asymmetry in the sunk cost effect):
The difference in average video engagement time between the long condition and the default condition is not equal to the difference in average average video engagement time between the default condition and the short condition.

To test the hypotheses, we will estimate the following regression specifications. First, we will regress the outcome variable on default treatment dummy (beta5) and long treatment dummy (beta10). To test the primary hypothesis, we will conduct a one sided test that beta10>0. To test the secondary hypothesis, we will conduct a likelihood ratio test that beta10 is not equal to 2*beta5. In all regressions, we will use individual fixed effects. Under the assumptions of temporal stability and causal transience, each video can be considered an independent observation. Thus, we do not cluster standard errors at an individual level.

Additionally, we will report the 2SLS specifications, where the sunk time is instrumented with the treatment condition.

In our analysis, we will exclude observations in the following cases: the intervened skippable ad did not occur at the front of the video. In rare cases in which it is clear that the extension incorrectly or incompletely recorded information about viewing a video (as a result of a technical issue), we will drop the relevant observation from the sample.

Randomization is conducted through the browser extension. For each video, the extension (using JavaScript) generates a random number between 0 and 1. If the number exceeds 2/3, we assign the long treatment (10 seconds wait time). If the number is between 1/3 and 2/3, then we assign the default treatment (5 seconds wait time). Lastly, if the number is below 1/3, we assign the short treatment (0 seconds wait time). Note that the treatment status remains unchanged when the user refreshes or accesses another video within 8 seconds of the ad starting.

A YouTube video

No

N/A

We recruit our sample for 4 weeks. We expect at least 30,000 observations depending on the frequency with which the participants watch YouTube videos with skippable ads.

We expect at least 10,000 observations per treatment condition.

For the primary hypothesis, the MDE is 0.035 standard deviation with 10,000 observations per treatment. This would be economically significant for platforms like YouTube given the number of users, and their active time on the platform.