The full experiment comprises two separate data collections that build on each other, a transmitter experiment and a listener experiment. The two experimental collections rely on different respondent samples.
Participants listen to two short recordings played consecutively and without a break, each one of an opinion piece providing a qualitative narrative about the future path of a different economic variable. Then, they record their own summary of these recordings, separately for the first and second variable.
A randomly chosen 50% of transmitters will be asked their prior belief about each variable before hearing the recordings, and all transmitters will be asked for the three beliefs described above after recording their transmitted message for each topic.
Recording treatment arms:
Within each topic, we randomize three key features of the original recordings:
*Level of variable*: We randomize whether the piece argues for an increase or a decrease in the level of the variable.
*Reliability of message*: Second, we randomize the reliability of the original message. We randomly assign respondents to one of two different types of reliability manipulations:
- Naturalistic (combination of explicit statements about confidence, source quality and speaker competence, as well as implicit markers of reliability): Respondents in the naturalistic condition are assigned to one of the following 2 conditions: (i) Strong reliability; (ii) Weak reliability.
- Modular (Insertion of explicit markers indicating high or low reliability (e.g., definitely vs. possibly, will vs. might, etc.): Respondents in the modular condition are assigned to one of the following 3 conditions: (i) Strong reliability; (ii) No reliability markers; (iii) Weak reliability.
*Sex of transmitter voice*: We randomize whether the recording is a male voice or a female voice. This is not a focus of analysis and we randomize simply for symmetry.
Randomization is stratified: each transmitter hears two recordings, one with an “increase” and one with a “decrease,” one with “strong reliability” and one with “weak reliability,” and one with a male voice and one with a female voice. Then, if exactly one of the two topics is in the modular condition, that topic has a 33% chance of getting switched to “no reliability markers.” If both topics are in the modular condition, there is a 66% chance that one of the two topics is randomly switched to “no reliability markers.”
Respondents receive incentives for transmitting all information contained in the original messages. Respondents are informed that one in 10 people will be selected for bonus eligibility and that, if selected, a different group of participants will score their recordings on a scale of 0 to 10, where 0 corresponds to “Nothing conveyed in meaning” and 10 corresponds to “Everything conveyed in meaning”. If the average score their recordings receive is at least an 8, they will receive a $20 bonus payment. Between subjects, we randomly assign respondents to two variants of the incentive scheme:
In the “Joint incentive treatment” participants are given the following instructions:
“The other participants will answer the following question about your voice message. How accurately did the voice message convey the content and meaning of what the speaker said?”
In the “Separate incentive treatment”, respondents are explicitly told that the other participants will answer two questions, one about the point forecast implied by the message and one about the reliability of the message. In particular, they receive the following instruction: “The other participants will answer two questions about your voice message.
How accurately was the level of the speaker's prediction conveyed in the voice message?
How accurately was the reliability of the prediction conveyed in the voice message?”
This involves a separate set of respondents. For each of the two topics, respondents first state their prior belief about the outcome variable of interest and then listen to a recording about the variable. As before, the order of the topics is randomized. For each topic, respondents are randomly matched to a transmitter and listen either to the same original recording as the one the transmitter heard, or that transmitter’s transmitted recording. There is a 30% chance of hearing the original and 70% chance of hearing the transmitted recording.
After listening to a recording, respondents are incentivized to forecast the future development of the variable as well as to guess the prediction of the message originator and the reliability of the original message. (The same 3 outcomes described above, incentivized in the same way).