Keep It or Skip It? Sequential Consumption of Music with Reference Effects

Last registered on August 29, 2022

Pre-Trial

Trial Information

General Information

Title
Keep It or Skip It? Sequential Consumption of Music with Reference Effects
RCT ID
AEARCTR-0009936
Initial registration date
August 28, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
August 29, 2022, 5:19 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
IESE Business School

Other Primary Investigator(s)

PI Affiliation
INSEAD
PI Affiliation
INSEAD

Additional Trial Information

Status
In development
Start date
2022-08-29
End date
2022-09-10
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Sequential consumption of experiential products gives rise to inter-temporal associations that influence the utility derived from the focal product. Developing recommendation algorithms that design experiences tailored to the users' dynamic preferences can be effective in enhancing user engagement. Using music streaming as the paradigmatic context of such interactions, where the user consumes multiple products in multiple sessions, in our paper, we construct a utility-based theoretical framework that takes into account the past consumption of users leading to references. The heterogenous responses of the users, rooted in the well-established constructs of habit formation and variety seeking, help us understand their dynamic preferences in sequential consumption, which are further influenced by the memory decay effects. This novel way of tracking the preferences of listeners allows us to develop a new class of recommendation algorithms that seeks to learn the variability in successively consumed music preferable to every listener in addition to the kind of music she likes (i.e., the genre or artist), which is different than the status quo of recommendation algorithms in the music streaming industry that focuses only on the latter (collaborative filtering-based). The counterfactual analyses that we carry out in the paper suggests that the recommendation algorithm that we propose is far better at retaining the listeners on the platform by increasing their engagement and the average duration per song that they listen to. The goal of this experiment is to test this effect in a controlled environment through randomly assigned interventions that correspond to the kind of recommendation provided to the participant. The experiment study is divided in two sessions where the first one allows us to learn the music listening preferences of the participants by recording their real-time reaction to the music. They are then randomly classified into one of the two treatment groups or a control group for the second session where Treatment Group 1 is provided recommendations based on the algorithm that we propose in our paper, Treatment Group 2 is provided recommendations based on the collaborative filtering-based algorithm, and Control group participants are provided with a randomly selected playlist of songs, The response of the participants in both the sessions is tracked through their skipping activity and the self-reported satisfaction metric. Empirical analyses are then to be carried out on this data to statistically measure the effect of recommendations based on our proposed method on the listener engagement and satisfaction in comparison to the other intervention and control group.
External Link(s)

Registration Citation

Citation
Askin, Noah, Abhishek Deshmane and Khwan Kim. 2022. "Keep It or Skip It? Sequential Consumption of Music with Reference Effects." AEA RCT Registry. August 29. https://doi.org/10.1257/rct.9936-1.0
Experimental Details

Interventions

Intervention(s)
Every participant is asked to listen to a set of songs as the experiment begins. This first playlist consists of a very specific assortment of songs which are arranged in a specific order so as to elicit the participant's response. Note that for the purposes of the experiment, we limit the recommendations to the works of the top 10 artists in the world based on the listening activity observed on Deezer (i.e., the ten most popular artists on Deezer) during the years 2018-2019. Hence, for building this initial playlist, and all the remaining playlists in the experiment, we utilize the music profile of the top 10 artists such as Drake, Ed Sheeran, Coldplay, Imagine Dragons, XXXTENTACION, etc. We make this conscious choice of restricting our study to this set of 10 artists because of their popularity and the higher likelihood of participants to be already aware of their past works. Involving relatively unknown artists in our analysis may confound the eventual results due to some participants already being aware of their music and some not.

Through this first listening session, we simultaneously measure two kinds of behavioral tendencies of each participant: (1) the amount of variability she prefers in successive songs within a listening session and (2) the kind of music she prefers. To do so, we track how long a participant listens to a given song and whether and when she skips the song to move to the next song. Based on such information, we immediately determine (1) whether a participant prefers to seek various tastes or form habitual tastes ("variety-seeking" vs. "habit-forming") as well as (2) whether a participant prefers a certain genre over the other genre in a given playlist ("pop-rock" vs. "rap"). As such, each participant, as soon as the first listening session ends, gets to have two behavioral labels regarding her variability preference and genre preference, respectively. Depending on which treatment a participant is assigned to, only one of the two behavioral labels of that participant will be used. For those who are assigned to the control group, neither information will be used.

Before beginning the second session, the participants are randomly assigned to one of the three groups which decides the kind of recommendations that will be given to them. Accordingly, we have two treatment groups and one control group:

1. Treatment Group 1 is given a new set of songs based on an algorithm that follows the proposed recommendation methodology in our paper (of the same title). This recommendation is based on the amount of variability that a participant prefers during the listening session (variability preference). Hence, for a participant who is classified into this group, we already know whether she is more inclined to be 'habituated' or 'variety seeking.' We have two separate playlists that cater to each of these distinctive tastes which is then fed to the participant in the second session based on which suits her the best. These playlists are generated from the works of the top 10 artists by the algorithm that can estimate the artistic difference between two songs using the cosine dissimilarity index as shown in the paper.

2. Treatment Group 2 gets recommended songs using content-based collaborative filtering technique that is the status quo in the music streaming industry. Here, the recommendation is more concerned with tracking the kind of music, for example, genre, that the participant prefers (genre preference). The first session already allowed us to learn whether the participant likes a certain kind of music, here: rap (for example, music from Drake, XXXTENTACION, Eminem, etc.) and pop-rock (for example, music from Coldplay, Ed Sheeran, etc.). With this understanding, we then feed the participants in treatment group 2 a playlist that best reflects their musical taste between rap music and pop-rock music. These playlists are also randomly generated from the songs released by the top 10 artists by an algorithm that seeks to classify songs based on their genres (acoustic attributes).

3. The control group is given a new set of randomly selected songs. These songs are randomly selected from the options available out of the total songs released by the top10 artists considered in our study.

The group assignment between the first and the second listening sessions is the only intervention designed for the present study.

Note that the participants in this study are all equally compensated. Hence, if there are any participants who complete the experiment by clicking on the skip button too often (without really giving any song in the playlist a reasonable listening time), we plan to exclude them from the analyses.
Intervention Start Date
2022-08-29
Intervention End Date
2022-09-10

Primary Outcomes

Primary Outcomes (end points)
Skipping frequency, average duration per song and self-reported satisfaction
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Every participant is asked to listen to a set of songs as the experiment begins. This first playlist consists of a very specific assortment of songs which are arranged in a specific order so as to elicit the participant's response. Note that for the purposes of the experiment, we limit the recommendations to the works of the top 10 artists in the world based on the listening activity observed on Deezer (i.e., the ten most popular artists on Deezer) during the years 2018-2019. Hence, for building this initial playlist, and all the remaining playlists in the experiment, we utilize the music profile of the top 10 artists such as Drake, Ed Sheeran, Coldplay, Imagine Dragons, XXXTENTACION, etc. We make this conscious choice of restricting our study to this set of 10 artists because of their popularity and the higher likelihood of participants to be already aware of their past works. Involving relatively unknown artists in our analysis may confound the eventual results due to some participants already being aware of their music and some not.

Through this first listening session, we simultaneously measure two kinds of behavioral tendencies of each participant: (1) the amount of variability she prefers in successive songs within a listening session and (2) the kind of music she prefers. To do so, we track how long a participant listens to a given song and whether and when she skips the song to move to the next song. Based on such information, we immediately determine (1) whether a participant prefers to seek various tastes or form habitual tastes ("variety-seeking" vs. "habit-forming") as well as (2) whether a participant prefers a certain genre over the other genre in a given playlist ("pop-rock" vs. "rap"). As such, each participant, as soon as the first listening session ends, gets to have two behavioral labels regarding her variability preference and genre preference, respectively. Depending on which treatment a participant is assigned to, only one of the two behavioral labels of that participant will be used. For those who are assigned to the control group, neither information will be used.

Before beginning the second session, the participants are randomly assigned to one of the three groups which decides the kind of recommendations that will be given to them. Accordingly, we have two treatment groups and one control group:

1. Treatment Group 1 is given a new set of songs based on an algorithm that follows the proposed recommendation methodology in our paper (of the same title). This recommendation is based on the amount of variability that a participant prefers during the listening session (variability preference). Hence, for a participant who is classified into this group, we already know whether she is more inclined to be 'habituated' or 'variety seeking.' We have two separate playlists that cater to each of these distinctive tastes which is then fed to the participant in the second session based on which suits her the best. These playlists are generated from the works of the top 10 artists by the algorithm that can estimate the artistic difference between two songs using the cosine dissimilarity index as shown in the paper.

2. Treatment Group 2 gets recommended songs using content-based collaborative filtering technique that is the status quo in the music streaming industry. Here, the recommendation is more concerned with tracking the kind of music, for example, genre, that the participant prefers (genre preference). The first session already allowed us to learn whether the participant likes a certain kind of music, here: rap (for example, music from Drake, XXXTENTACION, Eminem, etc.) and pop-rock (for example, music from Coldplay, Ed Sheeran, etc.). With this understanding, we then feed the participants in treatment group 2 a playlist that best reflects their musical taste between rap music and pop-rock music. These playlists are also randomly generated from the songs released by the top 10 artists by an algorithm that seeks to classify songs based on their genres (acoustic attributes).

3. The control group is given a new set of randomly selected songs. These songs are randomly selected from the options available out of the total songs released by the top10 artists considered in our study.

The group assignment between the first and the second listening sessions is the only intervention designed for the present study.

Note that the participants in this study are all equally compensated. Hence, if there are any participants who complete the experiment by clicking on the skip button too often (without really giving any song in the playlist a reasonable listening time), we plan to exclude them from the analyses.
Experimental Design Details
Randomization Method
Randomization is done by the randomization design in Qualtrics online.
Randomization Unit
Individual participants.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
NA
Sample size: planned number of observations
210
Sample size (or number of clusters) by treatment arms
70 individuals for Treatment Group 1, 70 individuals for Treatment Group 2, and 70 individuals for Control.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
The INSEAD Institutional Review Board
IRB Approval Date
2022-07-08
IRB Approval Number
2022-56
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials