Effect of Question Order on the Statistical Relationship between Investors' Recall and Their Expectations

The experiment studies the effects of question order on the statistical relationship between two pieces of data that we collect through surveys: investors’ memory of past stock market experiences and future return expectations. We conduct these surveys among retail investors in China. Respondents are randomly assigned to one of three groups, each answering the same questions but in varying sequences.

2024-03-25

2024-04-05

Recalled episode returns, Recalled own returns, Expected future returns

Recalled episode returns: in the free recall block of the survey, investors report their most memorable investment period and the returns of Shanghai composite index during that period. For simplicity, we designed the question on return as multiple choices, each choice covering 1\% (e.g. 0\% to 1\%). We then take the midpoint of each choice range as their recalled episode return.


Recalled own returns: in the probed recall block, we explicitly ask investors to recall returns of their own portfolios in the past one day, one month, or five years. Similarly, the questions are designed as multiple choices, each choice covering 1\% (e.g. 0\% to 1\%). We then take the midpoint of each choice range as their recalled own return for different horizons.

Expected future returns: in the expectation block, investors report their expected return in the future one month or one year for the overall stock market or for their own portfolio. the questions are designed as multiple choices, each choice covering 1\% (e.g. 0\% to 1\%). We then take the midpoint of each choice range as their expected returns for the future.

To collect investors’ recall of past experiences in the stock market and their expectations of the future performances of the market and one’s own portfolio, our study surveys retail investors in China.

Our survey contains four blocks, free recall block, probed recall block, the expectation block, and the demographical information block. The free recall block asks investors to recall an episode that first comes to their mind when they think about stock market movements in the past. The probed recall block asks investors to recall their own returns in the stock market in the past one day, one month, one year or five years. Respondents’ answers in these two blocks will be used to construct measures of their recall of past stock investment experiences. In the expectation block, we elicit two types of expectations, one about future market returns and one about future own returns. Respondents’ answers in this block will be used to construct measures of their return expectations for the future. At the end of the survey, we collect demographics and other information in a standard questionnaire, including age, gender, wealth, income, social activities, and so on.

To examine the effect of question order on the statistical relationship between recall and expectation, we've created three versions of our survey. Each version contains the same set of questions but arranged in different orders.

To collect investors’ recall of past experiences in the stock market and their expectations of the future performances of the market and one’s own portfolio, our study surveys retail investors in China.

Our survey contains four blocks, free recall block, probed recall block, the expectation block, and the demographical information block. The free recall block asks investors to recall an episode that first comes to their mind when they think about stock market movements in the past. The probed recall block asks investors to recall their own returns in the stock market in the past one day, one month, one year or five years. Respondents’ answers in these two blocks will be used to construct measures of their recall of past stock investment experiences. In the expectation block, we elicit two types of expectations, one about future market returns and one about future own returns. Respondents’ answers in this block will be used to construct measures of their return expectations for the future. At the end of the survey, we collect demographics and other information in a standard questionnaire, including age, gender, wealth, income, social activities, and so on.

To examine the effect of question order on the statistical relationship between recall and expectation, we've created three versions of our survey. Each version contains the same set of questions but arranged in different orders. The first version follows the sequence: free recall, probed recall, and expectation. The second version has the order: expectation, free recall, probed recall. The third version is arranged in the order of free recall, expectation, probed recall.

We will distribute three versions of the survey, each version containing the same questions but in different orders. To randomly assign investors, we require the Wenjuanxing platform (a prominent Chinese survey service provider equivalent to Amazon Mechanical Turk) to randomly send to each investor in their pool only one of the three invitations. To ensure the integrity of the randomization, we will analyze the distribution of respondents' biographical information.

Our randomization is at the individual level.

No

One cluster which is the sample pool of Wenjuanxing, a prominent Chinese survey service provider equivalent to Amazon Mechanical Turk.

3000 respondents in total.

We plan to send out three versions of the same survey. We target 1000 respondents for each version.