Experimental Design
Survey flow:
We invite around 300 journalists via a mailing list of the German Journalistic Association and ask if they are interested in participating in our study. Those who are interested will be randomly assigned to the treatment group or the control group. Journalists in the treatment group see our educational video (see attachment). Journalists in the control group will see the video after the trial.
Next, we ask all participants to read two news articles covering scientific studies. We also offer them to read the original scientific studies through a hyperlink. For each participant, we randomly draw two studies (plus two corresponding news articles, i.e., four news articles in total) from a pool of five studies. We display only the most relevant parts of the news articles (e.g., title, teaser, beginning), but the entire news articles are available through drop-down menues. For all news articles, the authors have pre-assessed the type and number of reporting mistakes (if any), following a codebook (see attachment). The pre-assessment is pre-registered here (see attachment), but will not be visible to our participants.
Then, for each news article, we ask our participants to assess whether there are mistakes in the reporting or not (e.g., misinterpretation of statistics), how many mistakes there are, and which types of mistakes (see "primary outcomes"). Then we ask them to suggest a hypothetical alternative title for an article covering the respective study (see "secondary outcomes").
After the trial, journalists in the control group receive access to the educational video, too.
Analysis:
In our main analysis, we will compare the treatment to the control group through simple OLS regressions with each of our outcomes on the LHS, respectively, and a dummy for being in the treatment group (or not) on the RHS. The parameter that measures the impact of the treatment indicator is our parameter of interest. Control variables include gender, age, and political orientation.