Memory and Belief Formation

2020-10-28

2021-07-01

Our primary outcome variable is respondents' beliefs about the probability that a randomly drawn image has a certain content (word vs number) conditional on receiving information (or not) about whether the image is a certain color (blue vs orange).
[Added after first set of experiments]: In the second part of the experiments, our primary outcome variable is respondent's beliefs about the probability that a randomly drawn word belongs to a certain category.

We have several secondary outcomes. We measure how confident subjects are (on a likert scale) that their probabilistic answers are correct. We also have two separate memory tasks. First, respondents are given a series of four words, and are ask "Was X (in C) among the images you were shown?" where C is a color and the word X is displayed in that color font. The correct answer to this question is always "no", and we are measuring how frequently respondents incorrectly believe that they saw the word. Our second memory task simply asks them to list all the orange words they remember having been shown.

[Added after first set of experiments]: In the second part of the experiments, we also have certainty measurements, as well as recall measurements for each category.

We present subjects with a series of images with multiple features (content vs color). We measure how beliefs about the distribution of these features vary depending on both normatively relevant and normatively irrelevant features of the dsitribution.

Respondents are shown 40 images. Each image is either a word or a number, and appears in either blue or orange font. Our primary outcome of interest is respondents answer to the following question: "Suppose the computer randomly chose an image from the images you just saw. It is orange. What is the percent chance that it is a word?" We have eight total treatments, which differ only in the distribution of words, numbers, and colors. We will certainly run the first six treatments, but we may decide against running the final two if maximum possible sample sizes at the lab are smaller than we hope.

Treatment 1: 10 orange words, 10 orange numbers, 0 blue words, 20 blue numbers.
Treatment 2: 10 orange words, 10 orange numbers, 10 blue words, 10 blue numbers.
Treatment 3: 10 orange words, 10 orange numbers, 20 blue words, 0 blue numbers.
Treatment 4: 14 orange words, 6 orange numbers, 0 blue words, 20 blue numbers.
Treatment 5: 14 orange words, 6 orange numbers, 6 blue words, 14 blue numbers.
Treatment 6: 14 orange words, 6 orange numbers, 20 blue words, 0 blue numbers.
Treatment 7: 11 orange words, 9 orange numbers, 20 blue words, 0 blue numbers.
Treatment 8: 9 orange words, 11 orange numbers, 0 blue words, 20 blue numbers.


Our theory predicts both a bias towards 50-50 and that, when there are more blue words, subjects will believe that a randomly chosen orange image is less likely to be a word, and vice versa. We first ask the probability of being a world conditional on the randomly chosen image being orange, then conditional on it being blue, then supposing they do not know it's color. Our primary outcome is the first question, and the other two are secondary. After each of these questions, respondents are asked how certain they are of their answers.

After this, respondents answer the memory questions described above.


[Added after first set of experiments]:
Respondents are shown 40 words. Each word will belong to a certain category, which will be described below (except for the random treatment). Our primary outcome of interest is respondents answer to the following question: "Suppose the computer randomly chose a word. What is the percent chance that it is a [Category]?" We have 5 total treatments, which differ in either the distribution of the categories, or the way the above question is asked, which can vary between asking about two categories (e.g. [Animals vs Others]) or three categories (e.g. [Animals, Men's names, and Women's names].

Treatment 1: 16 Land Animals, 12 Men's names, 12 Women's names, ask P(Animals vs Other)
Treatment 2: 8 Land Animals, 16 Men's names, 16 Women's names, ask P(Animals vs Other)
Treatment 3: 16 Land Animals, 12 Men's names, 12 Women's names, ask P(Animals) vs P(Men's name) vs P(Women's name)
Treatment 4: 16 Land Animals, 12 Men's names, 12 Sea Animals, ask P(Land Animals vs Other)
Treatment 5: 16 Land Animals, 24 Random Words, ask P(Land Animals vs Other)

Our primary outcome of interest is P(Land Animals), the primary category being asked. Our theory motivates the role of two forces: the way the question is asked (in terms of the precise hypothesis being cued: Treatment 1 vs 3), as well as the similarity across the elements within and across categories (Treatments 1, 4, 5: We predict Treatment 5 to be the biggest, while treatments 1&4 are ambiguous). Also, our theory implies a dampening of sensitivity to true probabilities (Treatment 1 vs 2). Furthermore, our theory in general predicts a link between recall (the number of words recalled for each category) and the probabilistic beliefs, as well as the fact that in recall, the presence of similar (but irrelevant) elements inhibits the recall of the category at hand, impacting probability beliefs.

We use qualtrics' randomization function.

Treatment is randomized at the individual level. The order of images, conditional on treatment, is randomized as well.

No

1200 individuals
[Second wave of experiments]: Also 1200 individuals

1200 individuals

150 respondents per treatment group.

[Second wave of experiments]: 240 respondents per treatment group

This sample size is sufficient to detect a 7pp difference in mean posteriors between treatments with 80% power and significance level p < 0.05. The standard deviation of these outcomes (from pilot data) is about 20pp.