Does AI Deepen the Divide? Examining the Effects of AI-Generated Content on Confirmation Bias and Opinion Polarization

The intervention is an incentivized online information-exposure experiment on controversial social issues. Participants are exposed to both AI-generated and human-written content, as well as to supportive and opposing viewpoints, with the order of the six topics randomized. The study uses a Bayesian Truth Serum (BTS) incentive scheme to encourage truthful reporting.

2026-04-29

2026-05-07

The primary outcome variables are: (1) title selection, measured by which title is chosen from the four available options for each topic (AI-supportive, AI-opposing, human-supportive, human-opposing); and (2) post-exposure evaluations and attitudes following article reading, title selection, and the final related-article stage.

This study is an incentivized online randomized experiment examining how AI-generated versus human-written information affects confirmation bias and opinion polarization across controversial social issues. The design combines within-subject and between-subject variation. Participants are exposed within-subjects to content that varies by source and stance, while source-related presentation features are assigned between subjects and balanced across treatment conditions. Truthful reporting is incentivized using Bayesian Truth Serum (BTS).

The experiment consists of two main components. In the first, participants are exposed to multiple articles that vary in source and viewpoint, with randomized presentation order. In the second, participants make choices among titles that vary along similar dimensions and are later exposed to related content. The design additionally varies whether source-related features remain aligned across stages of exposure. Topic order is randomized, and treatment assignments are balanced across participants.

Not available

Randomization was implemented by computer. Topic order and balanced treatment assignments were pre-generated in Excel using random numbers. Participants were then automatically assigned by Credamo to one of the experimental conditions, with all subsequent randomized elements determined by the pre-generated assignment.

individual

No

512 individual participants

512 individual participants

128 participants in labeled-consistent, 128 in labeled-inconsistent, 128 in unlabeled-consistent, and 128 in unlabeled-inconsistent conditions. Within each treatment arm, participants are exposed to both AI-generated and human-written content and to both supportive and opposing stances.

Power calculations are based on two-sided two-sample t-tests with a significance level of 0.05 and power of 0.80. For pooled main between-subject comparisons that combine two treatment cells (e.g., labeled versus unlabeled, or consistent versus inconsistent), with 256 participants per group, the minimum detectable effect size is approximately 0.25 standard deviations (d=0.248). For comparisons between individual treatment cells, with 128 participants per group, the minimum detectable effect size is approximately 0.35 standard deviations (d=0.352). As treatment is assigned at the individual level, no clustering adjustment is required.