Experimental Design
We posted the survey on Amazon Mechanical Turk (mTurk) with a description stating that the survey would take roughly 10 minutes, and respondents would receive $1.50 for completing the survey (a $9.00 hourly wage, approximately). Respondents were free to drop out at any time or take up to 24 hours to work on all the questions. We restricted our sample to those workers Amazon has verified as U.S. residents. We did not, however, choose to work with only “masters qualified” mTurk workers to avoid frequent (if otherwise reliable) survey experiment participants in favor of less experienced respondents. We launched our survey during U.S. business hours to discourage ineligible international respondents. Respondents only receive payment contingent upon completing the survey and are required to provide a unique password that is only visible at completion. Finally, aside from the voluntarily answered question on whether to write a letter to their U.S. Senator, we required subjects to answer all questions, and pop-up windows reminded them to complete all questions in each section before continuing.
Our experimental design includes eight treatments (two “information” treatments and six “argument” treatments) and a control condition. In all of these, respondents begin by indicating whether they favor or oppose six policies that one or more political actors (elected official or advocacy groups) have linked to improving economic mobility. These six polices are: 1) raising the minimum wage, 2) increasing cash assistance to the poor, 3) providing housing vouchers to move poor people into middle class neighborhoods, 4) universal pre-kindergarten, 5) marriage tax credits to encourage two-parent families for children, and 6) reducing immigration (both legal and illegal). These are not necessarily policies developed by economists nor found to be effective to promote intergenerational economic mobility, but have been framed in terms of current or intergenerational mobility by one or more political entrepreneurs. Following these questions, respondents are randomly assigned into one of the nine conditions.
The information treatments—Relative and Absolute—use a short interactive task to elicit respondents’ beliefs about either relative mobility or absolute mobility, respectively, though they do not elicit any information on the respondents themselves. The Relative task asks respondents to indicate what fraction of children born in the bottom 20 percent of the income distribution in the 1980s end up in each income quintile as an adult today, and also what fraction of children born in the top 20 percent end up in each income quintile today. The Absolute task, in contrast, asks respondents to estimate what fraction of children born in each income quintile in the 1980s earn more today than their parents earned when the children were born. Following the respondents’ guesses, we show them how their estimates compare to the actual data from Chetty et al. (2014) for relative mobility and Chetty et al. (2017) for absolute mobility, with the data presented both textually and graphically.
In addition to the two information treatments, we have six argument treatments. Each treatment is an explicit argument for one of the six policies styled similarly to how proponents of that policy tie it to economic mobility or poverty reduction in policy briefs, op-eds, or speeches, alongside a graphical presentation of evidence for the policy drawn from policy entrepreneurs’ arguments. Note that we are not claiming these policies will have the effect of increasing social mobility or decreasing income inequality. Rather, there are proponents of these policies who have framed arguments for each policy in these terms, and we have adapted their arguments and evidence into six treatments. Each of these treatments is similar to the argument for the estate tax in Kuziemko et al. (2015), except that the accompanying visual is always a graph (rather than, e.g., a picture) and the accompanying argument is always framed in terms of increasing social mobility. All treatments are standardized for length (between 159 and 162 words, about the length of a typical abstract).
To ensure that it is the content of each treatment, and not the type of activity (interactive guessing or reading arguments) or the length of time that drives the result, we pair each information treatment (Relative and Absolute) with a placebo argument and each argument treatment with a placebo interactive information task. Following Nickerson’s (2008) use of recycling as a “placebo” (as opposed to an uncontacted control group), we also employ an argument for recycling as a placebo argument for the two information treatments. Similarly, we use an interactive information task about recycling as a placebo for the six argument treatments (specifically, subjects guess what fraction of various products, like lead batteries or newsprint, are recycled). We chose recycling as a relatively “neutral” issue, both on the basis of Nickerson (2008) and of its overwhelming popularity across partisan lines (Pew Research Center 2009), though other survey evidence shows a partisan divide on recycling consistent with differences in the parties toward environmental issues generally (Coffey and Joseph 2013). To ensure consistency in terms of the order of activities, each treatment begins with an interactive task and ends with an argument.