Are Preconceptions Postconceptions? Evidence on Motivated Political Reasoning

2021-08-05

2021-08-25

We are interested in the extent to which individuals' reported normative beliefs change in response to new information. In particular, we will compare:
(a) People's ex ante beliefs about political issues (using answers of people in the control and hypothetical groups)
(b) People's claimed responsiveness to information (using answers of people in the control and hypothetical groups)
(c) People's ex post beliefs about political issues (using answers of people in all three groups -- in particular, the wedge between the treatment and hypothetical groups)

Furthermore, we are interested in the gap between the extent to which individuals claim, hypothetically, their beliefs would change if a given piece of information was true versus the extent to which they actually change when individuals are told the information is, in fact, true.

And we are interested in the extent to which our approach of eliciting responses to hypotheticals before providing the true information can be used as a tool for de-biasing.

Depending on the exact normative question, we expect either a negative or positive relationship between beliefs and the hypothetical information. To enable standardization across categories (such as when pooling the data), we will rotate the responses accordingly. In particular, the values for the answers to the taxes, mobility, and zoning questions will be reversed.

Our survey experiment will be posted on Amazon Mechanical Turk, advertised with a base compensation of $2.50 and an incentive bonus of up to $1 for participants. Individuals who opt to take our survey will answer a series of questions about their beliefs relating to several different political issues. The issues will include police shootings of minorities, climate change, affirmative action, income taxation of the top 1%, economic mobility, zoning laws, and the Olympics.

Once an individual opts to take the survey, they will begin by answering a series of demographic questions. After this, for each of five of the above seven issue areas (randomly selected), the respondent will be randomly sorted into either (i) the control group, (ii) the information treatment group, or (iii) the hypothetical treatment group. Independently, they will be sorted into an "A" or "B" group, which receive slightly different orders of normative beliefs questions. In other words, an individual may be sorted into the control group ver. A for the climate change issue, the hypothetical group ver. B for the affirmative action issue, and the treatment group ver. A for the income taxation issue.

Regardless of which group the individual is sorted into, they will be asked for their belief on an empirical fact relating to a political issue. After this,

In the control group, respondents will be asked for their normative beliefs on the issue (both the A and B questions).

In the information treatment group, respondents will be presented with the true answer to the above empirical question corresponding to the issue. Then, they will be asked for their normative beliefs on the issue (both the A and B questions).

In the hypothetical group, respondents will be asked their normative beliefs on the issue (both the A and B questions). They will then be presented, randomly, with three hypotheticals asking what their normative beliefs would be IF the true answer to the empirical question was X, for three randomly-selected values of X (one of which is the true value). They will be asked the A version of the normative beliefs question if they were sorted into group A and the B version if they were sorted into group B.

This process is then repeated four more times for four more issues.

Comparing the extent to which individuals claim they will hypothetically update their beliefs (for the true value of X) to the extent to which they actually do update their beliefs when presented with the true information will yield information on the extent to which motivated reasoning is occurring. In addition, we will test how much people claim they will respond to new information.

In particular, we will compare:
(a) People's ex ante beliefs about political issues (using answers of people in the control and hypothetical groups)
(b) People's claimed responsiveness to information (using answers of people in the control and hypothetical groups)
(c) People's ex post beliefs about political issues (using answers of people in all three groups -- in particular, the wedge between the treatment and hypothetical groups)

At the end of the survey, for each issue on which individuals were sorted into a hypothetical group, they will receive a follow-up. They will be given the information treatment and then asked their normative beliefs on the issue (both A and B). Comparing the extent to which they then update their beliefs on A and on B will yield information on the efficacy of using the hypotheticals approach as a tool for de-biasing.

In particular, we will test:
(a) Whether the apparent degree of motivated reasoning is reduced by this process of first posing hypothetical questions
(b) Whether the apparent degree of motivated reasoning differs between the specific normative question on which the respondent previously answered the hypothetical (e.g., A) versus the normative question which they were not posed hypotheticals about (e.g., B). To the extent that de-biasing is more successful for A vis-a-vis B (in this example) this suggests a demand effect rather than a true de-biasing

In a follow-up survey one week later, we will again ask people these normative beliefs questions (both the A and B versions) in order to examine the extent to which these effects persist. The mechanical demand effects whereby people in the hypothetical group feel constrained to be consistent with their hypothetical answers (e.g., for A but not constrained for B) should be substantially alleviated one week later, and this will allow us to determine whether any de-biasing is, in fact, real.

EMPIRICAL FACT QUESTIONS

- Police shootings: In 2015, the Washington Post began to log every fatal shooting by an on-duty police officer in the United States.

For context:
As of 2019, the US population was approximately 12.8% Black.
In 2019, according to the FBI, among people arrested for violent crimes, approximately 36.4% were Black.

What is your best estimate of the percentage of people fatally shot by police (from 2015 through 2020) who were Black (amongst those whose race was known)?


- Climate change: Climate scientists at NASA's Goddard Institute for Space Studies maintain records on global average temperatures since 1880.

How many of the 20 hottest years on record would you estimate have occurred since the year 2000, according to the NASA data?


- Affirmative action: Elite colleges and universities such as Harvard have a very restricted number of slots to which they admit applicants for undergraduate study each year.

Harvard states that it "does not discriminate against applicants from any group in its admissions processes."

According to data released by Harvard University, between 1995 and 2013, the average SAT score for admitted White students was 2234 out of 2400.

What is your best estimate of the average SAT score (out of 2400) for admitted African-American students over the same time period, according to Harvard's data?


- Income taxation: Every year, the Internal Revenue Service (IRS) releases data on income and taxation in the United States. From this data, the share of federal income tax paid by the top 1% of income-earners can be computed.

For context:
According to the IRS, 21% of all income was earned by the top 1% of income-earners.

What is your best estimate of the share of federal income tax that was paid by the top 1% of income-earners in 2019, according to the IRS data?


- Economic Mobility: The Internal Revenue Service (IRS) maintains an archive of historical tax return data from individual taxpayers. This data can be linked across generations and used to compute estimates of social mobility -- such as the fraction of children born to families in the bottom 20 percentiles of income who themselves made it into the top 20 percentiles of income at the same point later in their life.

Consider the poorest 20 percentiles of children, as measured by their parents' income in early adulthood. Then:
If there was no relationship between parent and child income, 20% of these children would themselves make it into the top 20 percentiles of income at the same age later in life.
0% corresponds to none of these children making it into the top 20 percentiles of income at the same age later in life.
100% corresponds to all of these children making it into the top 20 percentiles of income at the same age later in life.

Consider the poorest 20 percentiles of children, as measured by their parents' income in early adulthood.

What is your best estimate of the share of these children who themselves make it into the top 20 percentiles of income at the same age later in life, according to the IRS data?


- Zoning: Zoning laws regulate land-use. They restrict the types of buildings and/or activities that are permitted in certain areas within communities. Common motives for zoning laws include environmental concerns, neighborhood beautification, or preferences of the existing population for maintaining the community's size and character.

Research has suggested, however, that by constraining the supply of housing, restrictive zoning laws can increase rents and home prices.

For context:
According to an index compiled by the Cato Institute which measures land-use regulation, California has the 4th-most restrictive land-use regulations in the United States while Nevada has the 26th-most restrictive land-use regulations.
According to the US Census Bureau, as of 2019, the Las Vegas metropolitan area has 0.7% of total US population while the Los Angeles metropolitan area has 5.7% of total US population.

The Census Bureau compiles data on the number of building permits issued for the construction of new single-family homes in the United States. Between 2015 and 2019, 1.2% of those permits were issued in Las Vegas.

What percentage of permits do you think were issued in Los Angeles?


- Olympics: The modern Olympics games have occurred since 1896. According to the International Olympic Committee (IOC), counting both summer and winter Olympics (through 2018), the ten athletes who have won the most gold medals have won a total of 99 gold medals between them.

Of course, some sports provide more opportunities for an individual athlete to win many medals.

What is your best estimate of the number of gold medals won by American swimmer Michael Phelps?


NORMATIVE BELIEFS QUESTIONS

Police Shootings:
Normative Belief Question A: Do you believe that police in the US are systemically racist against Black people?
Normative Belief Question B: Do you support “defunding the police,” which entails reducing funding for police departments and reallocating that funding toward social services such as counseling, healthcare, and public housing?

Climate Change:
Normative Belief Question A: Do you support stronger government action to combat climate change, even if it requires higher taxes?
Normative Belief Question B: Do you believe that the global average temperature is warming due primarily to human activity?

Affirmative Action:
Normative Belief Question A: Do you think the Supreme Court should ban universities from using race as a factor in admission decisions? (Universities argue that affirmative action is key to improving representation of minorities.)
Normative Belief Question B: Do you think that affirmative action causes lower performing and less deserving students from underrepresented minority groups to take places that otherwise would be earned by more deserving students of other races?

Income Taxation:
Normative Belief Question A: Do you support higher taxes on the top 1% of income-earners?
Normative Belief Question B: Do you think the top 1% of income-earners excessively use loopholes to avoid paying their fair share of taxes?

Economic Mobility:
Normative Belief Question A: Do you believe that hard work doesn't generally bring success; luck and connections are more important?
Normative Belief Question B: Do you think that most wealthy people are only wealthy because of inheritance and/or family privilege?

Zoning:
Normative Belief Question A: Do you think zoning laws (often promoted for their environmental, aesthetic, and other benefits) are too restrictive and should be relaxed?
Normative Belief Question B: Do you think that differences in zoning laws substantially explain why some parts of the US (e.g. San Francisco, New York City) have extremely high housing prices relative to other areas (e.g. Houston, Orlando)?

Olympics:
Normative Belief Question A: Do you believe Michael Phelps is the greatest Olympic athlete of all time?
Normative Belief Question B: Do you believe Michael Phelps is the greatest swimmer of all time?

Randomization performed by the survey software (Qualtrics).

Randomization will occur at the individual level.

No

700 individuals

700 individuals

700 individuals each answer questions on 5 of 7 issues (randomly-selected). Consequently, each issue will be presented to approximately 500 individuals, of which 1/3 will be sorted into the control group, 1/3 will be sorted into the information treatment group, and 1/3 will be sorted into the hypothetical group.