Numbers tell, but words sell: Imprecision in persuasive communication

For experimental details, see the Experimental Design (Public) section.

2024-04-07

2024-06-30

H1. We hypothesize that senders are less likely to choose to send Number messages, and are more likely to choose to send Language or Interval messages, when they face directional incentives.

H1. We conduct this test twice: Once in the Number-or-Language group and once in the Number-or-Interval group. Our main specification controls for individual and probability fixed effects. Standard errors are clustered (here and everywhere else) at the individual level.

H2. We hypothesize that, within a message format, senders slant Number, Language, and Interval messages in the direction of their directional incentives relative to the aligned, and we hypothesize that there is more directional slant for Language and Intervals relative to Number when senders have directional-high compared with directional-low incentives.

H3. We hypothesize that senders with directional incentives who see "bad news" about the true probability use Language and Intervals more than senders who see "good news."

H4. We test whether receivers are persuaded by senders; that is, we test whether their predictions move more in the direction of the sender's incentives. Then, we test whether senders with directional incentives persuade receivers more in the conditions where Language/Interval messages are sent relative to the conditions where Number messages are sent.

We also consider several exploratory analyses.

For senders, we will explore whether senders with directional incentives who choose the Language or Interval format slant more in the direction of their incentives compared with those who choose the Number format, and whether senders with directional incentives who see "bad news" about the true probability slant more in this direction than senders who see "good news." We may also explore other heterogeneities at the sender level.

For receivers, we will explore whether receivers who do not know the incentives of the sender are persuaded more, and whether receivers who do not know the incentives of the sender are driving the persuasion effects for Language/Intervals tested in H4. We may also explore how receivers generally react to different message formats, testing whether they update their beliefs more from Number messages and rating Number messages as being more informative and honest.

We may also explore other questions, such as about day-to-day usage of numbers and language, numeracy, other heterogeneities, and open-ended questions.

H2. We define the variable Slant as follows:
For Number, Slant = Number - True Probability.
For Interval, Slant = Midpoint of Interval - True Probability.
For Language, we use the sender's mapped probability of that word or phrase, and then Slant = Mapped Language - True Probability.

For Number, Interval and Mapped Language messages, we winsorize Slant at the 2% and 98% level within incentive and probability categories. For Interval messages, we will drop observations where the True Probability is not between 5 and 95 percent. The reason for this is that the midpoint of the interval, which has a width of 10 pp, must be between 5 and 95 percent.

For each question, recall that senders choose two messages (one Number, and one of Interval or Language). We define Higher Slant as (Slant for Interval/Mapped Language - Slant for Number). To explore whether there is more directional slant for Language and Intervals, we regress Higher Slant on directional-high incentives (relative to directional-low incentives), again controlling for individual and probability fixed effects. We omit senders with aligned incentives for our main analysis.

H3. We regress Language/Interval choice on the interaction of True Probability and directional-high incentives, controlling for incentives and for individual and probability fixed effects. We omit senders with aligned incentives for our main analysis.

H4. We first define the variable Overestimate to equal Receiver Prediction - True Probability. We winsorize Receiver Prediction at the 2% and 98% level within sender incentive and probability categories. We regress Overestimate on directional-high incentives, controlling for format, individual-receiver, and probability fixed effects. We omit senders with aligned incentives for this analysis.

Next, we define Persuasion to equal Receiver Prediction - True Probability for senders with directional-high incentives and -(Prediction - True Probability) for senders with directional-low incentives. We then regress Persuasion on an indicator for whether the Language/Interval message was sent, controlling for individual-receiver and probability fixed effects. We omit senders with aligned incentives for our main analysis.

This experiment is conducted in two waves: The first wave involves "Senders" and the second wave involves "Receivers." Senders and receivers each complete 8 rounds, and are randomly matched across rounds. The structure of the experiment is as follows: First, senders learn the probability of a red ball being drawn from a box (from 0-100 percent). The "True Probability" varies within subject. Then, senders choose how to communicate this probability to a participant in the receiver wave. Finally, receivers are given a sender's message and asked to predict the probability of drawing the red ball. We describe this communication process in detail below.

Receivers' payments always depend on the accuracy of their prediction. Our first main treatment arm, which varies within subject, is whether senders' payments are:
- The same as their receiver's ("aligned"),
- Increasing in their receiver's predictions ("directional-high"), or
- Decreasing in their receiver's predictions ("directional-low").

Next, we describe what messages senders can send. Messages can come in one of three "formats":
Number messages have the format: "The chance you will draw a red ball is X percent."
Language messages have the format: "It is [word/phrase] that you will draw a red ball."
Interval messages have the format: "The chance you will draw a red ball is between Y and Y+10 percent."
Senders are either assigned, between subject, to a group in which they can choose Number and Language messages, or a group in which they can choose Number and Interval messages.

Senders in the Number-or-Language (Number-or-Interval) group make two choices. They first choose the X they want to communicate for the Number message and the word (Y) they want to communicate for the Language (Interval) message. Then, they choose whether they would rather send their Number or their Language (Interval) message. One of their two messages will always be the one used, but their preferred format will sometimes not be the one used.

Receivers face these treatments as well. They are either matched with aligned, directional-high, or directional-low senders. They are either matched with Number-or-Language senders or Number-or-Interval senders. For receivers, we also vary whether they are aware of their senders' incentives or not.

After completing the main 8 rounds, subjects map a numeric probability associated to the words and phrases available in for Language messages, which we will use to map Language to Number estimates.

Finally, we will ask subjects a set of questions, including about demographics and how they use numbers and language in everyday communication, which we may use in exploratory analyses.

We will also give subjects an attention-check question, and will drop subjects from our analyses if they do not answer this question correctly.

Not available

Randomization for all treatments done by computer.

We randomize incentives (directional-high, directional-low, aligned) within subject.
We randomize whether senders and receivers are in Number-or-Language or Number-or-Interval between subject.
We randomize Probability within subject.
For receivers, we randomize whether they know their sender's incentives between subject.
If receivers see a random format from senders, the format they see is randomized within subject.

No

Senders: 500 participants
Receivers: 500 participants

Senders: 8000 message choices, 4000 message format choices Receivers: 4000 predictions after messages

Senders: 1/2 of senders choose between numbers and language, 1/2 of senders choose between numbers and intervals. 1/3 of observations involve directional-high incentives, 1/3 involve directional-low incentives, and 1/3 involve aligned incentives.

Receivers: 1/2 of receivers receive messages using numbers and language, 1/2 of receivers receive messages using numbers and intervals. 1/3 of observations involve directional-high sender incentives, 1/3 involve directional-low sender incentives, and 1/3 involve aligned sender incentives. 1/2 of receivers know their sender's incentives, 1/2 of receivers do not know their sender's incentives.