Real vs. Hypothetical Stimuli in Digital-Literacy Interventions

After consent and covariate collection, each participant is cross-randomized to one of four cells. The stimuli arm factor: Real (verbatim Nigerian Facebook posts) vs. Hypothetical (the same posts edited along five linguistic dimensions—length, capitalization, present-tense rate, redaction indicators, sentiment intensity—to emulate researcher-authored stimuli from prior survey experiments). The digital-literacy course factor: an Emotions course teaching recognition of fear, anger, and superiority appeals in misinformation, vs. a Facts-based active control of comparable length covering misinformation prevalence, history, and cognitive-bias findings without manipulation-detection instruction. Both courses adapt the chatbot course of Appel et al. (2026), itself adapted from Athey, Cersosimo, Koutout, and Li (2023).

2026-06-15

2026-06-30

Three participant-level outcomes, defined for both the pre-course and the immediate post-course binary sharing blocks: (1) Misinformation sharing rate: share of three misinformation stimuli the participant would share; (2) Non-misinformation sharing rate: share of three non-misinformation stimuli the participant would share; (3) Sharing discernment: non-misinformation minus misinformation sharing rate.

The first research question uses the pre-course block (Hypothetical - Real difference); the second uses the immediate post-course block as a 2x2 stimuli-by-course interaction.

Each binary sharing-block elicits six binary decisions per participant: three on misinformation stimuli and three on non-misinformation stimuli. The misinformation sharing rate is the participant's mean share decision over the three misinformation stimuli in the block, taking values in 0, 1/3, 2/3, or 1. The non-misinformation sharing rate is constructed analogously over the three non-misinformation stimuli. The discernment outcome is the non-misinformation rate minus the misinformation rate, taking values between -1 and 1, with higher values indicating greater discernment between non-misinformation and misinformation content. The same construction applies to the pre-course block (research question 1) and to the immediate post-course block (research question 2). The pre-analysis plan documents the construction in Section 5.1.

Sharing decisions from the three remaining post-course blocks: a third binary block (six decisions), a pairwise forced-choice block (three M/N pairs), and a two-stage block (three forced-choice plus six binary decisions over three M/N pairs). Subgroup heterogeneity for the first research question by digital-literacy index, pre-course misinformation and non-misinformation sharing, sex, and age. Engagement and attention-check pass rates as balance diagnostics.

The additional post-course blocks support an ancillary research question on robustness across elicitation formats. Subgroup analyses are exploratory and restricted to the first research question; the trial is not powered for three-way interactions, so no subgroup analysis is registered for the second. Balance diagnostics include attention-check pass rates and consent-to-completion shares by cell. See Sections 5.3--5.5 of the pre-analysis plan.

Between-participant 2x2 factorial cross-randomization at study entry, after pre-treatment covariates. Factors: stimuli arm (Real vs. Hypothetical) and digital-literacy course (Emotions vs. Facts-based control). Assignment is stratified on a binarization of the nine-item Digital Literacy Index (Offer-Westort, Rosenzweig, and Athey, 2024), age (cutoff 25), and sex (male vs. non-male), giving eight strata. Per-cell counters within each stratum keep the four cells balanced throughout recruitment. Sharing intentions are elicited in five blocks of six stimuli each: a pre-course binary block, an immediate post-course binary block (fixed position after the course), and three post-course blocks (a third binary block, a pairwise forced-choice block, a two-stage block) presented in uniformly randomized order at the participant level.

Within each block, three misinformation and three non-misinformation stimuli appear in randomized order with one embedded attention check at a random position. Each participant sees each of the 30 source stimuli exactly once. The first research question is identified from the pre-course block; the second from the immediate post-course block.

Not available

Computer-generated stratified randomization in Qualtrics. After the participant submits the covariates needed to compute their stratum, embedded JavaScript reads the per-cell counters for that stratum, assigns the cell with the smallest count (ties broken uniformly at random), and increments the counter. Assignment is invisible to the participant and reproducible from the participant-by-time log.

Individual participant; single level of randomization.

No

N/A

450 completed surveys, each contributing 30 sharing decisions over 30 unique stimuli across five blocks.

Balanced allocation across the four cells: ~113 participants each in Real x Emotions, Real x Facts, Hypothetical x Emotions, and Hypothetical x Facts. Total N=450, stratified across eight strata (binarized Digital Literacy Index x age x sex). Budgeting ~40% consent-to-completion attrition; the Facebook campaign targets N~750.

At N=450, analytical heteroskedasticity-robust SEs for the first research question (Hypothetical - Real OLS coefficient) at baseline rates p_mis=0.30 and p_non=0.55 are ~0.043 (misinformation), 0.047 (non-misinformation), and 0.064 (discernment). Implied 80%-power MDEs at alpha=0.10 (one-sided for the binary rates; two-sided for discernment): (1) Misinformation: 9.1 p.p.; (2) non-misinformation: 10 p.p.; discernment: 13.6 pp. For the second research question, the interaction-coefficient SE is twice the main-effect SE under balanced 2x2 allocation, giving MDEs of ~22, 23, and 32 p.p. (two-sided, alpha=0.10). Simulation-calibrated SEs in Section 6.3 of the pre-analysis plan incorporate within-participant correlation and shrink modestly. The trial is primarily powered for the main stimuli-arm effect; differential-effect tests detect only large effects on misinformation sharing.