How digital media markets amplify news sentiment

Journalist experiment: The randomization occurs on the incentives. There are three groups:
1. control group: receives a flat payment for the participation
2. pay per click group: receives a payment relative to the clicks the selected headlines generate.
3. pay per abo group: receives a payment relative to the newsletter abonnements the selected headlines receive.


Readers experiment: The randomization occurs on the level of the headlines which readers are exposed to. There are four main groups:
1. neutral headlines without competition: subjects see one neutral headline per topic and can select to click on it or not
2. neutral headlines with competition: subjects see two neutral headlines per topic and can select to click on one or both or none
3. emotional headlines without competition: subjects see one emotional headline per topic and can select to click on it or not
4. emotional headlines with competition: subjects see one emotional and one neutral headline per topic and can select to click on one or both or none
In the treatments with emotional headlines half of the participants are exposed to positive headlines and the other half to negative headlines.

2024-10-11

2024-11-30

Journalist experiment: emotionality and sentiment of headline suggestions, emotionality and sentiment of headline choices out of two menus of three headlines each
Readers experiment: reading duration, factual knowledge, polarization of opinions, post-feelings

Journalist experiment:
- Journalists will be asked to suggest headlines for three (randomly drawn) articles. The sentiment and emotionality of these headlines will be classified by a machiene learning algorithm and validated by human classifications.
- Jounalists will have to select a headline out of a menu of three real headlines for a given article. The menu consists of a positive, a neutral and a negative headline.
- Jounalists will have to select a headline out of a menu of three headlines which were written by ChatGPT for a given article. The menu consists of a positive, a neutral and a negative headline.

Readers experiment:
- Reading duration for each article is the time spend on an article page in seconds. These 6 measures per person will be added up to obtain the total time a participant spent reading.
- Factual knowledge is measured as the absolute distance to the correct values of the estimation questions and then converted into standard deviatons. To aggregate the 6 measures, the absolute distance (expressed in standard deviations) will be computed. Before the conversion into standard deviations, each measure will be winsorized at the 5th and 95th percentile to minimize the influence if outliers.
- Opinions are measured as agreement to polarizing statements on a scale from -2 (strongly disagree) to 2 (strongly agree). To obtain a measure for opinion polarization the absolute value of the stated opinion will be computed. Additionally, the six measures of opinion polarization will be aggregated into the average opinion polarization.
- Post-feelings are measured on a scale from 0 (feeling very bad) to 100 (feeling very good).

Journalist experiment: qualitative answers to questions on headlines choices, order of topics and assessment of topical emotionality
Readers experiment: click rates, newsletter subscriptions

Journalist experiment:
- Journalists are asked to explain the influences on their content choices outside of the experimental context in a drop-down menu and in a free text form.
- Journalists are given a list of six topics and have to say in which order they would select those topics and additionally estimate how emotional readers would react to those topics.

Readers experiment:
- Click rates are dummy variables equal to one if a person has clicked on a specific article and 0 otherwise. Additionally, the sum of the clicked articles will be computed.
- Newsletter subscriptions are dummy variables equal to one if a person has decided to subscribe to a newsletter on a specific topic and 0 otherwise. Additionally, the sum of the subscriptions will be computed.

In the journalist experiment, there will be three experimental groups who each have to complete six headline choices for two articles. The groups only differ in the way they are compensated. While the "flat pay" group is receiving a fixed amount for their participation, the remuneration of the "pay-per-click" group depends on how well the headlines they select perform with a sample of readers (=how many clicks they generate). The compensation of the "subscription pay" group in turn depends on how many readers are willing to subscribe to a newsletter to receive more information on a topic after being exposed to a headline. Fist, the journalists are asked to write a headline for a given article in a free text form themselves. Afterwards, they can two times per article decide by a menu of suggestions that differ in sentiment. In the last part of the experiment I will elicit demographic data of the journalists.

The following control variables will be used in the main regressions of the journalist experiment: age, gender, east/west, job type, political orientation, education.


Paticipants for the readers experiment are recruited via the market research company bilendi and representative for the German population in terms of age, gender and state of residence. They will be randomly exposed to the pool of headlines and articles from the journalists' experiment.

The following control variables will be used in the main regressions of the readers experiment: age, gender, federal state, income, job, political orientation, education, ex-ante knowledge on the six topics, feeling before the intervention, news consumption habits

In the readers experiment it is planned to analyze the heterogeneity of the effects for the following sub-groups:
- Age
- Ex-ante feeling
- Ex-ante opinion polarization / knowledge
- News consumption habits


Hypotheses:

Journalist experiment:
1. Journalists in the pay-per-click group select more emotional headlines than those in the control group. (between-subject)
2. Journalists in the pay-per-abo group select more emotional headlines than those in the control group. (between-subject)
3. Journalists choose more often emotional headlines if the decisions involve competitin on the headline level. (within-subject)

Readers experiment:
1. Those who read positive headlines feel better than those who read neutral or negative headlines.
2. Those who read emotional headlines spend more time reading articles than those who read neutral headlines.
3. Those who read emotional headlines have higher opinion polarization than those who read neutral headlines.
4. Those who read emotional headlines have lower factual knowledge than those who read neutral headlines.

computer

individual

No

N.A.

Planned sample size for the journalist experiment: 400. Data collection will be stopped either after 400 responses have been recorded or when the 30th of November 2024 has passed. Planned sample size for the readers experiment: 1600. Data collection will occur between 11th of October 2024 and 25th of October 2024.

Journalist experiment: 1/3 of participants in each treatment group.
Readers experiment: 400 participants per treatment group.