|
Field
Last Published
|
Before
July 11, 2025 02:56 PM
|
After
July 18, 2025 03:57 PM
|
|
Field
Primary Outcomes (End Points)
|
Before
Whether a particular movie was chosen/clicked. The proportion of movies seen that were chosen (everyone chooses 5, so this embeds variation in how many movies were seen). How that varies by in-group status of the recommender.
|
After
Whether a particular movie was chosen/clicked. The proportion of movies seen that were chosen (everyone chooses 5, so this embeds variation in how many movies were seen). How that varies by race and/or gender of the recommender.
|
|
Field
Primary Outcomes (Explanation)
|
Before
In-group will be defined as the recommendation coming from someone with a name whose perceived race or gender matches the respondent.
|
After
We will consider how whether a movie was clicked varies with the race and/or gender of the recommender and how that compares to the respondent's race and/or gender.
In-group will be defined as the recommendation coming from someone with a name whose perceived race or gender matches the respondent.
|
|
Field
Planned Number of Clusters
|
Before
1600 respondents. Each Choose 5 movies.
|
After
1600 respondents split between control (no rush) and rushed being 5 minutes. Each Choose 5 movies.
1600 respondents split between control (no rush) and rushed being 1 minute. Each Choose 5 movies
|
|
Field
Planned Number of Observations
|
Before
8000 observations (1600 x 5 movies clicked), or ~42,000 (if we consider all movies seen)
|
After
8000 observations (1600 x 5 movies clicked), or ~42,000 (if we consider all movies seen) in 5 minutes rushed (vs control)
Same in 1 minute rushed vs control.
|
|
Field
Power calculation: Minimum Detectable Effect Size for Main Outcomes
|
Before
The main outcome is an interaction term of ingroup x rushed for the clicked choice. The minimum detectable effect size for the ingroup x rushed outcome is 2.1 percentage points.
|
After
The main outcome is an interaction term of (race and/or gender of recommender x rushed for the clicked choice and slots above mean ranking for algorithmic order
The minimum detectable effect size for the interaction is 2.1 percentage points.
|
|
Field
Intervention (Hidden)
|
Before
We will solicit white male subjects on the prolific platform. Our subjects look at a series of movie recommendations, each of which is attached to a poster by name (e.g. “Amanda A. recommends...”), and choose amongst a subset of those movies to potentially watch. The movies and posters are shown three at a time on the screen, much as results from search algorithms or social media sites might show up, and the user can click “load more” to see more. We randomly assign names to the reviews as in a traditional audit study.
The main randomized intervention is the context in which the respondents make decisions. Some are randomized into:
Rushed condition: subjects are told they have limited time to make a decision and a clearly visible countdown clock sounds in milliseconds
Non-rushed condition: subjects are told they have plenty of time and the countdown clock counts in minutes.
|
After
We will solicit white male subjects on the prolific platform. Our subjects look at a series of movie recommendations, each of which is attached to a poster by name (e.g. “Amanda A. recommends...”), and choose amongst a subset of those movies to potentially watch. The movies and posters are shown three at a time on the screen, much as results from search algorithms or social media sites might show up, and the user can click “load more” to see more. We randomly assign names to the reviews as in a traditional audit study.
The main randomized intervention is the context in which the respondents make decisions. Some are randomized into:
Rushed condition: subjects are told they have limited time to make a decision and a clearly visible countdown clock counts in milliseconds
Non-rushed condition: subjects are told they have plenty of time and the countdown clock counts in minutes.
Because we do not know how fast people need to go to induce a feeling of stress or automaticity, we plan to vary how much time is actually available in the rushed condition. In one experiment it will be 5 minutes. And in the other it will be 1 minute.
|
|
Field
Secondary Outcomes (End Points)
|
Before
|
After
How the proportion of movies seen that are by in group recommenders varies by treatment
|