Disbelief in Polarized Societies

Participants in South Korea and the United States take part in an online survey experiment. Some participants are given information that supporters of different political parties perform equally well on factual knowledge questions, while others receive no such information. Afterwards, all participants answer a set of judgment tasks about facts and conspiracy theories, sometimes accompanied by signals about what partisan or educational groups believe. This setup allows us to test whether correcting perceptions about partisan knowledge influences people’s beliefs, information processing, and feelings toward political groups.

The intervention is embedded in an online survey experiment administered to supporters of left- and right-wing parties in South Korea (Democratic Party of Korea, People Power Party) and the United States (Democratic Party, Republican Party). The study begins with questions on demographics, political attitudes, and baseline affective polarization, followed by two pre-treatment factual judgment tasks. After these tasks, half of the respondents are randomly assigned to the treatment group, which receives the information that the accuracy rates of right- and left-wing supporters on the pre-treatment tasks differ by less than 5%. The control group receives no such information.

All respondents then complete post-treatment tasks. These include five judgment tasks with partisan signals (three factual, two conspiracy), in which respondents see either an in-party or out-party signal about majority supporter opinions. The signals are randomized at the respondent-task level. Respondents also complete two tasks with education signals (college vs. non-college majority opinion). Across tasks, participants judge the truthfulness of statements, rate their confidence, and guess the accuracy rates of in-group and out-group supporters. These outcomes allow estimation of treatment effects on disbelief, in-group bias in information processing, and affective polarization.

The design further incorporates robustness and supplementary analyses, including: (i) effects on conspiracy theory questions, (ii) comparison with education group signals, (iii) exclusion of respondents showing experimenter demand effects (measured via changes in risk-attitude questions), and (iv) heterogeneity analyses by demographics and partisan groups. Sample restrictions are applied to respondents with baseline disbelief greater than 0.05, as treatment effects are expected primarily among those with initial bias. Power calculations indicate a target of ~4,252 respondents per country, based on pilot survey results from the United States.

See the Pre-Analysis Plan for details

2025-09-08

2025-10-15

Disbeliefs – Difference between estimated accuracy rates of in-group versus out-group supporters when judging factual statements.

In-group bias in information processing – The Extent to which respondents update their judgments more when signals come from in-party members versus out-party members.

Affective polarization – Difference in favorable versus unfavorable feelings toward in-party and out-party supporters, measured using feeling thermometers and comfort measures for social relations.

Disbeliefs: Constructed as the gap between respondents’ perceived accuracy of their own partisan in-group and the partisan out-group. We measure this before and after treatment to capture whether exposure to accurate information reduces out-group disbeliefs.

In-group bias: Measured using post-treatment tasks (j = 3–5). Each respondent receives a randomized signal about either the in-party or out-party majority opinion. We compare updates in judgments (binary correctness and continuous confidence) across in-party versus out-party signal conditions.

Affective polarization: Based on (i) 0–100 feeling thermometer ratings toward in-group and out-group supporters and (ii) binary comfort questions regarding out-party members in roles such as colleagues, friends, or spouses of children. The difference between in-party and out-party ratings forms the polarization measure.

See the Pre-Analysis Plan for details

Disbeliefs about conspiracy theories – Changes in perceived knowledge gaps across partisan groups when evaluating conspiracy theory statements.

In-group bias in processing conspiracy theory signals – Extent to which respondents weigh in-party versus out-party cues when exposed to partisan signals on conspiracy theories.

Bias in information processing with education signals – Relative responsiveness to signals from college-educated versus non-college-educated groups.

Experimenter demand effect – Sensitivity of responses to cues about the experimenter’s hypothesis (measured via repeated risk attitude questions).

Heterogeneity of treatment effects – Variation in treatment impact by demographics (age, gender, partisan groups, etc.).

Conspiracy disbeliefs: Constructed analogously to factual disbeliefs, but using tasks on conspiracy theories (j = 6, 7).

In-group bias on conspiracy theories: Measured by how much respondents update their judgments after seeing in-party versus out-party majority positions on conspiracy questions.

Education signals: Bias measured by comparing belief updates after signals from college vs. non-college groups (tasks j = 8, 9).

Experimenter demand: Identified by changes in responses to risk-attitude questions when experimenter hypotheses are made explicit; used to exclude affected respondents in robustness checks.

Heterogeneity: Examined by estimating treatment and bias outcomes across subgroups defined by demographics and partisan groups.

We conduct a randomized online survey experiment in South Korea and the United States. Participants are randomly assigned to either a treatment or control group. The treatment group receives information that supporters of different political parties perform equally well on factual knowledge questions, while the control group receives no information. After this, participants complete judgment tasks on factual and conspiracy statements, sometimes paired with signals about the majority views of partisan or education groups. This design allows us to test whether correcting perceptions of partisan knowledge influences beliefs, information processing, and affective polarization.

The experiment begins with baseline questions on demographics, political attitudes, and affective polarization, followed by two pre-treatment factual judgment tasks. Respondents are then randomly split: the treatment group is informed that partisan groups differ by less than 5% in accuracy on the factual questions, while the control group receives no information.

Post-treatment, all respondents complete five additional judgment tasks (three factual, two conspiracy), where each task randomly assigns them either an in-party or out-party signal regarding majority opinions. They also complete two additional tasks with education signals (college vs. non-college). In each task, respondents provide a true/false judgment, confidence rating, and estimates of partisan accuracy.

These tasks enable estimation of (i) treatment effects on out-group disbelief, (ii) in-group bias in information processing, and (iii) changes in affective polarization. Supplementary analyses assess the processing of conspiracy theories, compare partisan versus education signals, account for experimenter demand effects, and test heterogeneity by demographics and partisanship.

Randomization is conducted by computer within the survey platform. Respondents are randomly assigned to treatment or control groups, and for each post-treatment task, they are further randomized to receive either an in-party or out-party signal (or, for education tasks, college vs. non-college signals).

The unit of randomization is the individual respondent for each task. Randomization occurs at the individual level both for assignment to treatment versus control and for assignment to signal types within tasks.

No

No clusters; randomization is at the individual-task level (online survey).

8,504 respondents total (4,252 per country: South Korea and the United States).

See the Pre-Analysis Plan

See the Pre-Analysis Plan

Pre Analysis Plan