Are Preconceptions Postconceptions? Evidence on Motivated Political Reasoning

2023-11-29

2023-12-13

We study the extent to which individuals' reported normative beliefs change in response to new information. In particular, we will examine:
(a) People's ex ante beliefs about political issues (using answers of people in the control group)
(b) People's claimed responsiveness to information (using answers of people in the hypothetical group)
(c) People's ex post beliefs about political issues (using answers of people in the treatment group)
(d) People’s ex post beliefs about political issues when potentially constrained by previously-stated hypothetical beliefs (using answers of people in the hypothetical group when they are subsequently given the treatment information)

With these components, we are interested in the gap between the extent to which individuals claim, hypothetically, their beliefs would change if a given piece of information was true versus the extent to which they actually change when individuals are told the information is, in fact, true. This is evidence of motivated reasoning. Numerically, this is given by the wedge between (c) and (b), relative to the wedge between (c) and (a). (This can be measured for any given demographic or pre-treatment subgroup.)

Second, we are interested in the extent to which our approach of eliciting responses to hypotheticals before providing the true information can be used as a tool for de-biasing. Numerically, this is given by the wedge between (d) and (c) in aggregate and at the individual level.

Third, we are interested in the magnitude of the responsiveness of hypothetical beliefs to hypothetical information, relative to the cross-sectional relationship between normative beliefs and empirical beliefs in the control group. To make this more concrete, how much of the partisan difference in normative beliefs is claimed as being due to differences in information.

Our survey experiment will be posted on Prolific and distributed to their nationally-representative panel of respondents. Individuals who opt to take our survey will answer a series of questions about their beliefs relating to several different political issues. The issues will include police shootings of minorities, climate change, affirmative action, income taxation of the top 1%, economic mobility, transgender participation in sports, crime in Republican- and Democratic-run cities, gun control, and the Olympics.

Once an individual opts to take the survey, they will begin by answering a series of demographic questions. After this, for each of five of the above nine issue areas (randomly selected), the respondent will be randomly sorted into either (i) the control group, (ii) the information treatment group, or (iii) the hypothetical treatment group. Independently, they will be sorted into an "A" or "B" group, which receive slightly different orders of normative beliefs questions. In other words, an individual may be sorted into the control group ver. A for the climate change issue, the hypothetical group ver. B for the affirmative action issue, and the treatment group ver. A for the income taxation issue.

Regardless of which group the individual is sorted into, they will be asked for their belief on an empirical fact relating to a political issue. After this,

In the control group, respondents will be asked for their normative beliefs on the issue (both the A and B questions).

In the information treatment group, respondents will be presented with the true answer to the above empirical question corresponding to the issue. Then, they will be asked for their normative beliefs on the issue (both the A and B questions).

In the hypothetical group, respondents will be presented, randomly, with three hypotheticals asking what their normative beliefs would be IF the true answer to the empirical question was X, for three randomly-selected values of X (one of which is the true value). They will be asked the A version of the normative beliefs question if they were sorted into group A and the B version if they were sorted into group B.

This process is then repeated four more times for four more issues.

Comparing the extent to which individuals claim they will hypothetically update their beliefs (for the true value of X) to the extent to which they actually do update their beliefs when presented with the true information will yield information on the extent to which motivated reasoning is occurring. In addition, we will test how much people claim they will respond to new information.

Concretely, we will examine
(a) People's ex ante beliefs about political issues (using answers of people in the control group)
(b) People's claimed responsiveness to information (using answers of people in the hypothetical group)
(c) People's ex post beliefs about political issues (using answers of people in the treatment group)
(d) People’s ex post beliefs about political issues when potentially constrained by previously-stated hypothetical beliefs (using answers of people in the hypothetical group when they are subsequently given the treatment information)

At the end of the survey, for each issue on which individuals were sorted into a hypothetical group, they will receive a follow-up. They will be given the information treatment and then asked their normative beliefs on the issue (both A and B). Comparing the extent to which they then update their beliefs on A and on B will yield information on the efficacy of using the hypotheticals approach as a tool for de-biasing.

In particular, we will test:
(a) Whether the apparent degree of motivated reasoning is reduced by this process of first posing hypothetical questions
(b) Whether the apparent degree of motivated reasoning differs between the specific normative question on which the respondent previously answered the hypothetical (e.g., A) versus the normative question which they were not posed hypotheticals about (e.g., B). To the extent that de-biasing is more successful for A vis-a-vis B (in this example) this suggests a demand effect rather than a true de-biasing. To the extent that this attempted de-biasing is at least partially unsuccessful, this is indicative of the strength of motivated reasoning on political views; not only are people able to discount unfavorable information, they would have done so even in the face of information they had recently claimed would change their mind.

In a follow-up survey one week later, we will again ask people these normative beliefs questions (both the A and B versions) in order to examine the extent to which these effects persist. The mechanical demand effects whereby people in the hypothetical group feel constrained to be consistent with their hypothetical answers (e.g., for A but not constrained for B) should be substantially alleviated one week later, and this will allow us to determine whether the information treatment is persistent and any de-biasing is, in fact, real.

EMPIRICAL INFORMATION

- Police shootings: In 2015, the Washington Post began to log every fatal shooting by an on-duty police officer in the United States.

For context:
As of 2019, the US population was approximately 12.8% Black.
In 2019, according to the FBI, among people arrested for violent crimes, approximately 36.4% were Black.

What is your best estimate of the percentage of people fatally shot by police (from 2015 through 2020) who were Black (amongst those whose race was known)?


- Climate change: Climate scientists at NASA's Goddard Institute for Space Studies maintain records on global average temperatures since 1880.

How many of the 20 hottest years on record would you estimate have occurred since the year 2000, according to the NASA data?


- Affirmative action: Elite colleges and universities - such as Harvard, Princeton, and UC Berkeley - have a very restricted number of slots to which they admit applicants each year. The Supreme Court is currently in the process of deciding on the constitutionality of affirmative action - which entails taking race into consideration as part of the admissions decision.

Some states, including California and Florida, have already passed bans on affirmative action. Students applying to UC Berkeley are assigned an academic score based on their their high school grade point average (GPA) and SAT scores. The University of California system keeps data on the academic scores of its admitted undergraduate students.

In the three years prior to the affirmative action ban, the median underrepresented minority (i.e. Black, Hispanic, or Native American) student admitted to UC Berkeley had a higher academic score than what percent of White admitted students?

For context,
- If White and underrepresented minority students had identical academic score distributions, the answer would be 50%.
- If every White student had a higher academic score than every underrepresented minority student, the answer would be 0%.
- If every underrepresented minority student had a higher academic score than every White student, the answer would be 100%


- Income taxation: Every year, the Internal Revenue Service (IRS) releases data on income and taxation in the United States. From this data, the share of federal income tax paid by the top 1% of income-earners can be computed.

For context:
According to the IRS, 20.9% of all income was earned by the top 1% of income-earners.

What is your best estimate of the share of federal income tax that was paid by the top 1% of income-earners in 2018, according to the IRS data?


- Economic Mobility: The Internal Revenue Service (IRS) maintains an archive of historical tax return data from individual taxpayers. This data can be linked across generations and used to compute estimates of social mobility -- such as the fraction of children born to families in the bottom 20 percentiles of income who themselves made it into the top 20 percentiles of income during adulthood.

Consider the poorest 20 percentiles of children, as measured by their parents' income in early adulthood. Then:
If there was no relationship between parent and child income, 20% of these children would themselves make it into the top 20 percentiles of income during adulthood.
0% corresponds to none of these children making it into the top 20 percentiles of income during adulthood.
100% corresponds to all of these children making it into the top 20 percentiles of income during adulthood.

Consider the poorest 20 percentiles of children, as measured by their parents' income.

What is your best estimate of the percentage of these children who themselves make it into the top 20 percentiles of income during adulthood, according to the IRS data?


- Transgender Women in Sports: In recent years, there has been increasing social attention on questions of gender and sport. In particular, there is increasing debate about whether transgender women (that is, males who have transitioned to be women) should be allowed to compete in women's sporting events.

In 2021, the National Scholastic Athletics Foundation, an organization dedicated to the support of junior age (19 and under) and high school track and field, held the NSAF Outdoor Nationals.

Similarly, in 2022, USA Swimming held the Junior National Championships for individuals aged 18 and under.

The performances of medal-winning female athletes in the 2020 Tokyo Olympics in swimming and athletic events can be compared to the highest performing junior boys at these respective junior US national finals.

There were 84 medals on offer in women's swimming and athletics events in the Tokyo Olympics (in 2021) where junior boys (18 and under in swimming, 19 and under in athletics) competed in an identical event at these respective national junior finals in 2021 and 2022. Suppose the times/distances of the athletes were compared as if they were competing against each other. How many of the 84 medals would have been won by the Olympic women?

To make this clear, this is comparing the performance of the best female athletes in the world to the performance of junior boys in the US national finals.


- Olympics: The modern Olympics games have occurred since 1896. According to the International Olympic Committee (IOC), counting both summer and winter Olympics (through 2018), the ten athletes who have won the most gold medals have won a total of 99 gold medals between them.

Of course, some sports provide more opportunities for an individual athlete to win many medals.

What is your best estimate of the number of gold medals won by American swimmer Michael Phelps?


- Political Parties and Crime: Cities across the US vary substantially in their murder rates. According to crime data reported to the FBI by police departments in 2020, the average murder rate in the two largest cities with Democratic mayors (New York, NY and Los Angeles, CA) was 6.7 murders per 100,000 people.

What is your best guess of the average murder rate per 100,000 people in 2020 in the two largest cities with Republican mayors (Fort Worth, TX and Jacksonville, FL)?


- Gun Control: The Virgin Islands are a cluster of islands in the Caribbean. Some of the Virgin Islands are a territory of the United States (the U.S. Virgin Islands). Some of the Virgin Islands are a territory of the United Kingdom (the British Virgin Islands). Levels of income and demographics are similar for the two territories ($35,365 per-capita GDP for the USVI in 2017, $34,200 per-capita GDP for the BVI in 2017; 76.3% Black and 5.4% White in BVI in 2010, 66.1% Black and 13.5% White in USVI in 2010).

Gun laws vary between the USVI and the BVI. Because the USVI are a U.S. territory, the Second Amendment of the U.S. Constitution applies, so the right to gun ownership and a concealed-carry permit is guaranteed by law. In the BVI, the right to gun ownership is not guaranteed by law, and under the Virgin Islands Firearms and Air Guns Act of 2015, it is illegal to own many types of guns (including semi-automatic rifles, shotguns, and paintball guns).

According to the BVI police force, the murder rate in the BVI was 14.5 murders per 100,000 people in 2020-21.

What is your best guess of the murder rate per 100,000 people in 2020-21 for the USVI?


NORMATIVE BELIEFS

Police Shootings:
Normative Belief Question A: Do you believe that police in the US are systemically racist against Black people?
Normative Belief Question B: Do you support “defunding the police,” which entails reducing funding for police departments and reallocating that funding toward social services such as counseling, healthcare, and public housing?

Climate Change:
Normative Belief Question A: Do you support stronger government action to combat climate change, even if it requires higher taxes?
Normative Belief Question B: Do you believe that the global average temperature is warming due primarily to human activity?

Affirmative Action:
Normative Belief Question A: Do you think the Supreme Court should ban universities from using race as a factor in admission decisions? (Universities argue that affirmative action is key to improving representation of minorities.)
Normative Belief Question B: Do you think that affirmative action causes lower performing and less deserving students from underrepresented minority groups to take places that otherwise would be earned by more deserving students of other races?

Income Taxation:
Normative Belief Question A: Do you support higher taxes on the top 1% of income-earners?
Normative Belief Question B: Do you think the top 1% of income-earners excessively use loopholes to avoid paying their fair share of taxes?

Economic Mobility:
Normative Belief Question A: Do you believe that hard work doesn't generally bring success; luck and connections are more important?
Normative Belief Question B: Do you think that most wealthy people are only wealthy because of inheritance and/or family privilege?

Transgender Participation in Sports:
Normative Belief Question A: Do you believe transgender women should be allowed to compete in women’s sporting events?
Normative Belief Question B: Do you support transgender children having access to puberty-blocking medication?

Olympics:
Normative Belief Question A: Do you believe Michael Phelps is the greatest Olympic athlete of all time?
Normative Belief Question B: Do you believe Michael Phelps is the greatest swimmer of all time?

Political Parties and Crime:
Normative Belief Question A: Do you believe policies implemented by Democratic mayors lead to crime problems in the cities they govern?
Normative Belief Question B: Do you support criminal justice policies that aim to reduce incarceration rates and focus on rehabilitation instead of punishment?

Gun Control:
Normative Belief Question A: Do you believe gun control laws are an effective way of reducing violent crime?
Normative Belief Question B: Do you support a federal assault weapons ban?

Randomization performed by the survey software (Qualtrics).
For each issue, the arm (control, treatment, hypothetical) that a given individual is sorted into is randomized independently.
For each issue, the order in which Question A and Question B are asked is randomized independently.
For each issue where the individual was in the hypothetical arm, the order in which the follow-up information treatment questions are asked is randomized independently.

Randomization will occur at the individual level.

No

2000 individuals

2000 individuals

2000 individuals each answer questions on 5 of 9 issues (randomly-selected). Consequently, each issue will be presented to approximately 1100 individuals, of which 1/4 will be sorted into the control group, 1/4 will be sorted into the information treatment group, and 1/2 will be sorted into the hypothetical group.