Back to History Current Version

A Word is Worth A Thousand Words: Biased Beliefs about Political Language Use

Last registered on May 23, 2022

Pre-Trial

Trial Information

General Information

Title
A Word is Worth A Thousand Words: Biased Beliefs about Political Language Use
RCT ID
AEARCTR-0009476
Initial registration date
May 22, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
May 23, 2022, 7:21 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
University of Chicago

Other Primary Investigator(s)

Additional Trial Information

Status
Completed
Start date
2021-08-25
End date
2021-08-29
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
In a time when politics seem to be polarized as ever, it isn’t surprising that language use may be polarized too. If people define words in different ways, then using those words to describe their political positions can result in being perceived as more (less) extreme than they actually are. I conducted a survey with 782 MTurk workers in the U.S., eliciting their definition of the term "Democratic Socialism". This included asking them to predict how others would define the word, as well as a treatment group that received financial incentives based on predictive accuracy. I find that there is a large discrepancy between those on the Left and Right when defining the word. Financial incentives increase predictive accuracy more for those on the Right more than those on the Left or Moderates. More importantly, people from both groups made errors when predicting how others would define the word, but the nature of these errors was asymmetric across parties. In particular, those on the Right tended to make similar predictions for others on the Left versus Right, whereas those on the Left believed others on the Left were more likely to agree with them.
External Link(s)

Registration Citation

Citation
Shah, Rohen. 2022. "A Word is Worth A Thousand Words: Biased Beliefs about Political Language Use." AEA RCT Registry. May 23. https://doi.org/10.1257/rct.9476-1.1
Experimental Details

Interventions

Intervention(s)
Treatment participants will receive a financial incentive for making accurate predictions about others' beliefs.
Intervention Start Date
2021-08-25
Intervention End Date
2021-08-29

Primary Outcomes

Primary Outcomes (end points)
Prediction accuracy
Primary Outcomes (explanation)
This is the difference between the respondent's prediction of others' beliefs and the true beliefs (as measured by our sample average).

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Participants are asked true/false questions regarding certain political words and their usage. They are then asked to predict what fraction of other survey respondents on the political left and right would answer true or false to those questions. Treatment participants are incentivized for accuracy in prediction.
Experimental Design Details
Randomization Method
randomization done in office by a computer.
Randomization Unit
Individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
782
Sample size: planned number of observations
782
Sample size (or number of clusters) by treatment arms
389 control, 393 treatment.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Social and Behavioral Sciences IRB from the University of Chicago
IRB Approval Date
2021-08-17
IRB Approval Number
IRB21-1311

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
August 29, 2021, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
August 29, 2021, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
782
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
782
Final Sample Size (or Number of Clusters) by Treatment Arms
782
Data Publication

Data Publication

Is public data available?
No

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Program Files

Program Files
No
Reports, Papers & Other Materials

Relevant Paper(s)

Abstract
In a time when politics seem to be polarized as ever, it isn’t surprising that language use may be polarized too. If people define words in different ways, then using those words to describe their political positions can result in being perceived as more (less) extreme than they actually are. I conducted a survey with 782 MTurk workers in the U.S., eliciting their definition of the term "Democratic Socialism". This included asking them to predict how others would define the word, as well as a treatment group that received financial incentives based on predictive accuracy. I find that there is a large discrepancy between those on the Left and Right when defining the word. Financial incentives increase predictive accuracy more for those on the Right more than those on the Left or Moderates. More importantly, people from both groups made errors when predicting how others would define the word, but the nature of these errors was asymmetric across parties. In particular, those on the Right tended to make similar predictions for others on the Left versus Right, whereas those on the Left believed others on the Left were more likely to agree with them. False Consensus Effect was approximately 3 times more prevalent than Pluralistic Ignorance, especially amongst others who shared your political views.
Citation
Shah, R. (2022). Word is Worth A Thousand Words: Biased Beliefs about Political Language Use. Working Paper.

Reports & Other Materials