|
Field
Intervention (Hidden)
|
Before
There will be multiple treatment arms, each testing a different ranking approach. For each arm, the first 50 content items displayed to a user, for each page load, for each platform, are subject to re-ranking. The algorithm processes these content items and adjusts order and selection based on the procedures described below. Items can be re-ordered, removed, or added from public content posted elsewhere on the platform.
In the control group, participants will not have their feed orders changed. Two thirds of the control group will receive in-feed survey questions, with the other one third receiving no questions.
At this stage, we pre-register two treatment arms (we will amend the pre-registration to add new arms ahead of switching them on for participants).
The algorithms below depend on a set of classifiers in Jigsaw’s Perspective API, which score text based on a variety of attributes in three categories:
BRIDGING classifiers: constructive, nuance, personal_story, affinity, compassion, respect, curiosity.
PERSUASION classifiers: fearmongering, power_appeal, generalization, scapegoating, moral_outrage, alienation.
TOXICITY classifiers: toxicity (Perspective API classic), severe toxicity, identity attack, insult, profanity, threat, sexually explicit, flirtation.
Treatment Arm 1: Upranks Bridging content
Treatment Arm 2: Upranks Bridging content and downranks both Toxicity and Persuasion
|
After
There will be multiple treatment arms, each testing a different ranking approach. For each arm, the first 50 content items displayed to a user, for each page load, for each platform, are subject to re-ranking. The algorithm processes these content items and adjusts order and selection based on the procedures described below. Items can be re-ordered, removed, or added from public content posted elsewhere on the platform.
In the control group, participants will not have their feed orders changed. Two thirds of the control group will receive in-feed survey questions, with the other one third receiving no questions.
At this stage, we pre-register two treatment arms (we will amend the pre-registration to add new arms ahead of switching them on for participants).
The algorithms below depend on a set of classifiers in Jigsaw’s Perspective API, which score text based on a variety of attributes in three categories:
BRIDGING classifiers: constructive, nuance, personal_story, affinity, compassion, respect, curiosity.
PERSUASION classifiers: fearmongering, power_appeal, generalization, scapegoating, moral_outrage, alienation.
TOXICITY classifiers: toxicity (Perspective API classic), severe toxicity, identity attack, insult, profanity, threat, sexually explicit, flirtation.
Treatment Arm 1: Upranks Bridging content
Treatment Arm 2: Upranks Bridging content and downranks both Toxicity and Persuasion
As previously indicated, we are amending the pre-registration by adding new new classifiers (three new treatment arms). The new arms are described below. Please note that the intervention for Treatments 3-5 started later than for Treatments 1-2 (which were originally pre-registered).
Tretmeant Arm 3 (Heartbreak): This algorithm recommends content based on ideological alignment of source and user, i.e., content that users likely disagree with from sources they usually agree with (and vice-versa), to study the impact on users’ perceptions of out-groups and inclination to consider other perspectives. The ideology of users is determined from nine issue position questions in the baseline survey, while the ideology of content and domains is evaluated using GPT 4o prompts.
Treatment Arm 4 (Feedspan): Detects civic content that doesn’t attract diverse engagement (not ‘bridging’) and replaces it with similar content that is expected to be more bridging, hopefully reducing divisiveness without decreasing civic content. Bridgingness is determined using a BERT-based model that is trained on LLM labels that indicate whether a post is likely to be found interesting and engaging by both average Republicans and Democrats.
Treatment Arm 5 (Personalized Quality News): Recommends news posts from credible and ideologically diverse news sources, which are tailored to a specific user's interests to increase political knowledge and make users more resilient to democratic threats.
|