The Transmission of Reliable and Unreiable Information

See document.

The full experiment comprises two separate data collections that build on each other, a
transmitter experiment and a listener experiment. The two experimental collections rely on
different respondent samples.
Transmitter experiment:
Participants listen to two short recordings played consecutively and without a break, each
one of an opinion piece providing a qualitative narrative about the future path of a different
economic variable. Then, they record their own summary of these recordings, separately for
the first and second variable.
A randomly chosen 50% of transmitters will be asked their prior belief about each variable
before hearing the recordings, and all transmitters will be asked for the three beliefs
described above after recording their transmitted message for each topic.
Recording treatment arms:
Within each topic, we randomize three key features of the original recordings:
Level of variable: We randomize whether the piece argues for an increase or a decrease in
the level of the variable.
Reliability of message: Second, we randomize the reliability of the original message. We
randomly assign respondents to one of two different types of reliability manipulations:
● Naturalistic (combination of explicit statements about confidence, source quality and
speaker competence, as well as implicit markers of reliability): Respondents in the
naturalistic condition are assigned to one of the following 2 conditions: (i) Strong
reliability; (ii) Weak reliability.
● Modular (Insertion of explicit markers indicating high or low reliability (e.g., definitely
vs. possibly, will vs. might, etc.): Respondents in the modular condition are assigned
to one of the following 3 conditions: (i) Strong reliability; (ii) No reliability markers; (iii)
Weak reliability.
Sex of transmitter voice: We randomize whether the recording is a male voice or a female
voice. This is not a focus of analysis and we randomize simply for symmetry.
Randomization is stratified: each transmitter hears two recordings, one with an “increase”
and one with a “decrease,” one with “strong reliability” and one with “weak reliability,” and
one with a male voice and one with a female voice. Then, if exactly one of the two topics is
in the modular condition, that topic has a 33% chance of getting switched to “no reliability
markers.” If both topics are in the modular condition, there is a 66% chance that one of the
two topics is randomly switched to “no reliability markers.”
Respondents receive incentives for transmitting all information contained in the original
messages in a way that preserves the induced belief movements of listeners to those
messages. Respondents are informed that 1 in 10 people will be selected for a bonus of up
to $20. In particular, we tell our respondents that their task is to record a message that
Induces average belief changes that are as close as possible to the average belief changes
induced by the original message, measured over the full distribution of elicited beliefs; We
explicitly explain to respondents that beliefs are measured using a distribution.
In order to do this, they should pass on anything from the original message that they think
would be relevant for how people change their beliefs. They are told that if selected for the
bonus, their voice message will be played to some other participants, and we will measure
their belief changes after hearing the voice message. They are further told that the likelihood
of receiving the bonus payment depends on how close the average belief change induced by
their message within each bucket of the belief distribution elicitation is to the average belief
change induced within the corresponding bucket by the original messages.
Listener experiment:
This involves a separate set of respondents. For each of the two topics, respondents first
state their prior belief about the outcome variable of interest and then listen to a recording
about the variable. As before, the order of the topics is randomized. For each topic,
respondents are randomly matched to a transmitter and listen either to the same original
recording as the one the transmitter heard, or that transmitter’s transmitted recording. There
is a 30% chance of hearing the original and 70% chance of hearing the transmitted
recording.
After listening to a recording, respondents are incentivized to forecast the future
development of the variable as well as to guess the prediction of the message originator and
the reliability of the original message. (The same 3 outcomes described above, incentivized
in the same way).

2025-12-18

2025-12-19

Our experiments involve transmission of information about two unknown states: (i) the
change in home price growth in a US city and (ii) the change in revenue growth of a US
retailer, both for the upcoming year.
Each state has three key dependent variables associated with it: the respondent’s belief
movement about the unknown state as measured using the full stated belief distribution
(difference between posterior mean, elicited after the respondent listens to a message about
the state, and prior mean, elicited beforehand); the respondent’s belief about the message
originator’s prediction about the state of the world; and the respondent’s belief about the
originator’s reliability, as measured on a scale ranging from 0 (extremely unreliable) to 100
(extremely reliable).
For the state beliefs, we elicit the whole distribution of beliefs using a set of pre-specified
bins (from -10% to 10% in 2% increments).
For all three outcomes, we will z-score within topic*manipulation type quadrants and then
pool together the observations from both topics.

See document.

The full experiment comprises two separate data collections that build on each other, a
transmitter experiment and a listener experiment. The two experimental collections rely on
different respondent samples.
Transmitter experiment:
Participants listen to two short recordings played consecutively and without a break, each
one of an opinion piece providing a qualitative narrative about the future path of a different
economic variable. Then, they record their own summary of these recordings, separately for
the first and second variable.
A randomly chosen 50% of transmitters will be asked their prior belief about each variable
before hearing the recordings, and all transmitters will be asked for the three beliefs
described above after recording their transmitted message for each topic.
Recording treatment arms:
Within each topic, we randomize three key features of the original recordings:
Level of variable: We randomize whether the piece argues for an increase or a decrease in
the level of the variable.
Reliability of message: Second, we randomize the reliability of the original message. We
randomly assign respondents to one of two different types of reliability manipulations:
● Naturalistic (combination of explicit statements about confidence, source quality and
speaker competence, as well as implicit markers of reliability): Respondents in the
naturalistic condition are assigned to one of the following 2 conditions: (i) Strong
reliability; (ii) Weak reliability.
● Modular (Insertion of explicit markers indicating high or low reliability (e.g., definitely
vs. possibly, will vs. might, etc.): Respondents in the modular condition are assigned
to one of the following 3 conditions: (i) Strong reliability; (ii) No reliability markers; (iii)
Weak reliability.
Sex of transmitter voice: We randomize whether the recording is a male voice or a female
voice. This is not a focus of analysis and we randomize simply for symmetry.
Randomization is stratified: each transmitter hears two recordings, one with an “increase”
and one with a “decrease,” one with “strong reliability” and one with “weak reliability,” and
one with a male voice and one with a female voice. Then, if exactly one of the two topics is
in the modular condition, that topic has a 33% chance of getting switched to “no reliability
markers.” If both topics are in the modular condition, there is a 66% chance that one of the
two topics is randomly switched to “no reliability markers.”
Respondents receive incentives for transmitting all information contained in the original
messages in a way that preserves the induced belief movements of listeners to those
messages. Respondents are informed that 1 in 10 people will be selected for a bonus of up
to $20. In particular, we tell our respondents that their task is to record a message that
Induces average belief changes that are as close as possible to the average belief changes
induced by the original message, measured over the full distribution of elicited beliefs; We
explicitly explain to respondents that beliefs are measured using a distribution.
In order to do this, they should pass on anything from the original message that they think
would be relevant for how people change their beliefs. They are told that if selected for the
bonus, their voice message will be played to some other participants, and we will measure
their belief changes after hearing the voice message. They are further told that the likelihood
of receiving the bonus payment depends on how close the average belief change induced by
their message within each bucket of the belief distribution elicitation is to the average belief
change induced within the corresponding bucket by the original messages.
Listener experiment:
This involves a separate set of respondents. For each of the two topics, respondents first
state their prior belief about the outcome variable of interest and then listen to a recording
about the variable. As before, the order of the topics is randomized. For each topic,
respondents are randomly matched to a transmitter and listen either to the same original
recording as the one the transmitter heard, or that transmitter’s transmitted recording. There
is a 30% chance of hearing the original and 70% chance of hearing the transmitted
recording.
After listening to a recording, respondents are incentivized to forecast the future
development of the variable as well as to guess the prediction of the message originator and
the reliability of the original message. (The same 3 outcomes described above, incentivized
in the same way).

Randomization done in office by a computer

Individuals

No

200 respondents in the transmitter experiment
800 respondents in the listener experiment

200 respondents in the transmitter experiment 800 respondents in the listener experiment

TRANSMITTER EXPERIMENT
- Randomization is stratified: each transmitter hears two recordings, one with an “increase” and one with a “decrease,” one with “strong reliability” and one with “weak reliability,” and one with a male voice and one with a female voice. Then, if exactly one of the two topics is in the modular condition, that topic has a 33% chance of getting switched to “no reliability markers.” If both topics are in the modular condition, there is a 66% chance that one of the two topics is randomly switched to “no reliability markers.”
- Between subjects, we randomly assign transmitters to two variants of the incentive scheme (50% each) as explained above.
- A randomly chosen 50% of transmitters will be asked their prior belief about each variable before hearing the recordings, and all transmitters will be asked for the three beliefs described above after recording their transmitted message for each topic.


LISTENER EXPERIMENT:
- The order in which listeners see the topics is randomized.
- For each topic, respondents are randomly matched to a transmitter and listen either to the same original recording as the one the transmitter heard, or that transmitter’s transmitted recording. There is a 30% chance of hearing the original and 70% chance of hearing the transmitted recording.
- For 50% of respondents, there will be no incentive and the question will be phrased as a direct question about the message originator’s beliefs. For 50% of respondents, the question will be phrased as a second-order question (“your job is to predict what people would on average respond to the direct question”) and responses will be incentivized as described above.

Pre-registration - Belief distribution robustness