Experimental Design
We will expose the participants to fakes and facts on Corona vaccines and on nutrition that we retrieve from Facebook.
The participants will be randomly divided into five groups:
- Participants who are not exposed to any fakes or facts at all
- Participants who are exposed to fakes and facts
- Participants who are exposed to fakes and facts and fact checks that debunk the fake news on vaccines (not the fakes on nutrition) --> Fact check intervention
- Participants who are shown Facebook’s “Tips how to spot false news” and are afterwards exposed to fakes and facts --> Media literacy intervention
- Participants who are exposed to facts and the fact checks on vaccines
To study the short-term effects of the fact check and the media literacy intervention, we will compare participants’ factual knowledge, perceived credibility of the “fake news”, and their behavioral intentions w.r.t. Corona vaccines to participants who saw the fakes and facts, but did not receive an intervention.
To examine potential spillover effects of the fact check and media literacy interven-tion, we show the participants fakes and facts on an unrelated topic (dietary sup-plements) without further interventions. Then, we will again compare factual knowledge, perceived credibility of the “fake news”, and behavioral intentions of participants in the fact check and in the media literacy group to participants who saw the fakes and facts, but did not receive an intervention.
Finally, to study the long-term effects of the fact check and media literacy interven-tion, we re-invite our participants about one week later and repeat our analyses.
Estimation procedure:
Our baseline regressions will be OLS estimations with treatment group dummies as main explanatory variables. The dependent variables are perceived credibility of the “fake news”, factual knowledge on Corona vaccines and dietary supplements, and self-reported probabilities to get vaccinated and to consume dietary supplements. Control variables include age, gender, family status, income, education, “big 5”, po-litical preferences, and prior knowledge on current events, health, and nutrition. As participants may skip over the fact checking and media literacy intervention, our baseline analysis is equivalent to an ITT analysis. To obtain estimates for the LATE of our interventions, we will also conduct an IV estimation, where we use the time spent with the fact checks or the “Tips how to spot false news” as endogenous explanatory variable and assignment to treatment as instrument.
We hypothesize the following effects:
1a) In the short-run, the fact checking intervention reduces the credibility of and increases factual knowledge about the corrected „fake news” as compared to participants without intervention.
1b) In the short-run, participants who received the fact checking intervention are more likely to state that they are willing to get vaccinated against Covid-19 than participants without intervention.
2a) In the short- and in the long-run, the media literacy intervention reduces the credibility of and increases factual knowledge about all „fake news” as compared to participants without intervention.
2b) In the short- and in the long-run, participants who received the media literacy intervention are more likely to state that they are willing to get vaccinated against Covid-19 and abstain from dietary supplements than participants without intervention.
2c) The long-run effects of the media literacy intervention are smaller than its short-run effects.
The idea is as follows:
While fact checking is likely to be an effective tool to debunk the corrected messages themselves, its effect may be limited to these messages. Thus, the intervention re-duces the credibility of the “fake news” on Corona vaccines that we display in Ses-sion 1 as compared to participants without intervention. As a result, factual knowledge on Corona vaccines is larger than in the group without intervention and participants are more likely to get vaccinated. In contrast to that, we do not expect to see spillover effects on “fake news” about dietary supplements or on “fake news” on Corona vaccines in Session 2.
The media literacy intervention conveys general skills that can be broadly applied. Hence, we expect that the intervention reduces the credibility of “fake news” on Co-rona vaccines and dietary supplements that we display in Session 1 and 2 as com-pared to participants without intervention. As a result, factual knowledge on Corona vaccines and dietary supplements is larger than in the group without intervention and participants are more likely to get vaccinated and less likely to consume sup-plements. As some participants may have forgotten the “Tips how to spot false news”, we expect the effects to be smaller in Session 2 than in Session 1.
Subgroup analyses:
To date, about half of the German population is already fully vaccinated against Covid-19 and we expect a similar fraction of our participants to be vaccinated, too. As vulnerability to “fake news” may depend on prior knowledge and experiences, we will split the sample into vaccinated and non-vaccinated participants and explore heterogeneous effects. Similarly, we will partition our sample into participants who agree with the official Corona remedies vs. those who don’t to see if and how prior beliefs and attitudes moderate the effects of our interventions. Finally, an interesting subgroup analysis will be to partition participants into those who spend much and those who spend few time on social media to see if experience and prior exposure to “fake news” from social media plays a role.