Experimental Design Details
Setup. We conduct several experiments with the same people. We inform participants (Spectators) that they will make choices about allocating various goods to Recipients. We inform them that these goods will be allocated by a nonprofit that works in Memphis.
Our main objective is to test for the existence of “non-welfarist” preferences for health care and/or lawyers. By non-welfarist preferences, we mean preferences in which Spectators value features of the goods that do not directly increase the utility of the Recipients. Most of our elicitations are designed with the goal of comparing the treatment goods (health care or lawyers) to placebo goods (bus passes or YMCA memberships), which are also useful for participants but over which we do not expect Spectators will have non-welfarist preferences.
Randomization. Participants (Spectators) are randomized to observe all elicitations for one good. The exception is that for Elicitation 1, half of the Spectators who are otherwise randomized to observe a placebo good are randomized to see the elicitation with cash as the good.
We sometimes use the notion of “willingness to pay” (WTP) as a shorthand. Participants never pay. In some cases, they are asked to trade off providing cash to an outside option (future programming for the nonprofit or a donation to a food bank) and providing a good, service, or cash to the Recipient. In other cases, they are asked to trade off providing cash to some tenants versus goods to the same or other tenants.
Elicitation 1. This elicitation asks Spectators to choose between allocating the good and money to low-income Recipients, versus allocating the good to more Recipients (who are relatively less low-income) and less money to the low-income Recipients. For instance, we ask Spectators to choose between giving lawyers to a group of 10 tenants or giving just the 5 poorest tenants a lawyer and some amount of cash. We ask these questions iteratively until we identify the number of Recipients at which they are indifferent.
We expect that participants will exhibit more “anti-targeting,” i.e. allocate the good to a larger number of Recipients (even though they are relatively less low-income), when the good is health care or lawyers, versus the placebos. We compare targeting behaviors across the goods.
Elicitation 2. This elicitation informs Spectators that a Recipient has been allocated a good in a lottery. The Spectator can choose to rerun the lottery and earn money for the nonprofit which can be used to allocate more goods in the future to other Recipients. The idea is that Spectators may think that health care or lawyers are “inalienable” rights, and therefore require a large amount of money to take these goods from Recipients once they have been allocated. Spectators are informed that Recipients will not know whether the lottery was rerun are not and are only informed of the final allocation. We test for the amounts of inalienability across goods. We elicit the WTP, which is operationalized as the indifference point for the amount of money the nonprofit would need to save by rerunning the lottery.
Elicitation 3. This elicitation proceeds in three parts. First, we ask Spectators to report incentivized prior beliefs about how likely it is that Recipients would choose the good when offered the choice between $X of cash and the good. Second, we truthfully inform Spectators that choosing the good is rare, based on pilot data we collected. We then ask Spectators to report posterior beliefs after receiving the information. Third, we ask Spectators to report their WTP to give Recipients the choice between $X of cash and the good, versus giving the Recipients $X of cash alone and saving the remaining money for future nonprofit programs.
The idea behind this elicitation is that Spectators with non-welfarist preferences may value giving Recipients the choice in itself, even if they are told that exercising the choice is unlikely. Our primary tests will compare the Spectators’ willingness to pay for choice when the good is health care/a lawyer vs. the placebo goods.
A concern about this elicitation is that Spectators value choice for welfarist reasons, e.g. that Recipients with high value for the good may benefit from the choice even if it is unlikely that they will exercise it. We address this concern in two ways. First, we randomize the $X of cash (between $200 and $300). If WTP for choice does not depend on $X, that implies that Recipients are not “selecting on gains.” Second, we restrict to Spectators with high posterior beliefs that the Recipient will choose cash. In the limit, if the Spectator believes that the Recipient never chooses the good over cash, welfarist preferences would imply that the Spectator never has a positive WTP for choice.
Elicitation 4. This elicitation informs Spectators that Y in {1, 5, 9} Recipients out of 10 have been granted the good via a lottery. We then elicit the WTP to provide an additional Recipient with the good. Y is randomized. The WTP is operationalized slightly differently: we inform the Spectator to choose between providing the good to the Recipient and donating money to a food bank.
The idea behind this elicitation is that Spectators with non-welfarist preferences may value giving all Recipients the good, due to “universalist” preferences. Thus, universalist preferences would predict a higher WTP when Y = 9 than when Y = 1 or Y = 5. Pure consequentialist preferences should have equal WTP for all N.
A key concern about this elicitation is that Spectators may generally have inequality aversion, even over the placebo good. Therefore, we employ a “difference in differences” specification, where we compare Y = 9 to Y = 1 and Y = 5, across both the treatment and placebo goods.
We conduct two secondary elicitations to further probe Elicitation 1.
Elicitation 1a. An important confound to Elicitation 1 is that participants may provide more of some goods if they believe that the implied price set by the total budget is a better deal for that good. For instance, if participants believe lawyers are valuable in expected value, then they may provide lawyers to everyone. To address this concern, we set the implied price of each good in Elicitation 1 to the median WTP in the pilot surveys. We also elicit the WTP for each good directly. We examine whether this price is below the amount that is given to the low-income tenant in Elicitation 1. Note that this elicitation is secondary, as it complements Elicitation 1.
Elicitation 1b. This elicitation proceeds in four parts. We only conduct it among the treatment group (those randomized into observing rights-based goods). First, we ask people prior beliefs about the effectiveness of legal assistance or health care. Second, we provide (truthful) information about different facts related to the effectiveness of legal assistance or health care. We sub-randomize people into a High or Low treatment, where the High treatment suggests that lawyers or health care is effective and the Low treatment does the opposite. Third, we ask Spectators whether they would like to revise their prior beliefs. Fourth, we ask Spectators whether they would like to revise their choice from Elicitation 1, and the direction that they would like to change their choice.
The purpose of Elicitation 1b is to understand the extent to which Spectators are influenced by welfarist concerns about the effectiveness of lawyers/health care. We therefore examine the share who revise and the magnitude of their revision. Secondary analyses may also use the change in beliefs to instrument for the effect of beliefs on targeting. If beliefs updates do not drive behaviors in Elicitation 1, that implies that targeting decisions are more driven by non-welfarist considerations.
Placebo randomization and pooling. The presentation for the health care treatment differs slightly from the lawyers treatment. For instance, we do not tell participants that health care will be incentivized (as it will not be). For this reason, we randomize participants who see placebo goods into seeing exactly what the health care people see, versus exactly what the lawyers see.
We present analyses that disaggregate lawyers and health care and compare them to each other and pooled placebos. We also present analyses that pool lawyers and health care. We typically pool placebos, and only in secondary analyses do we disaggregate them. We disaggregate cash in Elicitation 1 from the other placebos.
We typically pool all placebos (including those who see the health care versus lawyers presentation). In supplemental analyses we will disaggregate them. The difference in outcomes across placebos (holding fixed a specific good, but varying the presentational elements only) serves as a joint test for whether slight differences in presentation together with incentives affect results.
We acknowledge that (some) participants may view people as having the right to transportation (say), which means that treatment versus placebo comparisons serve as a lower bound.
Incentives. We incentivize the provision of lawyers by collaborating with a local nonprofit in Memphis, TN, that provides attorneys to tenants facing eviction. We incentivize the provision of bus passes and/or YMCA memberships similarly, if they are shown the same presentation as the lawyers. We incentivize belief elicitations in Elicitation 1b for lawyers only and Elicitation 3 for lawyers and placebo goods.
We do not incentivize provision of health care. As a secondary test, to examine the extent to which incentives matter, we do not incentivize a portion of the WTP for lawyers (Elicitation 1a), selected at random.