Effects of varying the numbers of choice options and choice questions in discrete choice experiments for valuing public goods

There are four main survey treatments, which vary the voting procedures (i.e., discrete choice experiment design) across experiment participants:
- Single binary choice involving a "small" project. Participants are asked to vote on whether the group will fund a specific project. The option with the most votes is implemented for real.
- Single binary choice involving a "large" project. Participants are asked to vote on whether the group will fund a specific project. The option with the most votes is implemented for real.
- Repeated binary choice. Participants vote on nine voting questions. Each question is a yes or no vote on whether the group will fund a specific project. The projects vary across voting questions. One vote is randomly selected to be binding. The option with the most votes (for the randomly selected question) is implemented for real.
- Repeated trinary choice. Participants vote on nine voting questions. Each question asks participants to vote for one of two projects or no project. The pair of included projects varies across voting questions. One vote is randomly selected to be binding. The option with the most votes (for the randomly selected question) is implemented for real.

2023-03-03

2023-10-31

(1) Individuals' choices (votes) made in the discrete choice experiment. (2) Estimates of willingness to pay (WTP) for the projects.

WTP is identified based on experimental variation in the cost of funding the public good.

Experiment participation is facilitated through an online survey. The survey’s is about the value that people place on providing services to improve the well-being of farmworkers in New York State and about how they form voting choices. Participants are placed in groups of 50 people that are completing the survey experiment at roughly the same time. Votes are consequential in the sense that they determine whether the group will fund a farmwork assistance project, in which case people will forego some or all of their experiment earnings. We have partnered with the Cornell Farmworker Program to implement the projects as described in the survey.

The questionnaire starts with several "warm-up" questions to make people start thinking about the survey’s topic. Then, the survey describes possible assistance projects, an explanation of the voting procedures, and how projects will be carried out if implemented. This is followed by quiz questions to gauge and reinforce participant understanding. Next, participants are asked one or nine voting questions. Each question is presented as a choice set (standard for discrete choice experiments) and framed as a group vote. These votes are over farmworker assistance projects, which vary in the amount of education, clothing, and transportation services they would provide to needy farmworkers living in New York. Several follow-up questions are asked to assess understanding and to gauge possible motives underlying voting choices. The survey ends with socio-demographic questions.

There are four main survey treatments:
- Single binary choice involving a "small" project. Participants are asked to vote on whether the group will fund a specific project. The option with the most votes is implemented for real.
- Single binary choice involving a "large" project. Participants are asked to vote on whether the group will fund a specific project. The option with the most votes is implemented for real.
- Repeated binary choice. Participants vote on nine voting questions. Each question is a yes or no vote on whether the group will fund a specific project. The projects vary across voting questions. One vote is randomly selected to be binding. The option with the most votes (for the randomly selected question) is implemented for real.
- Repeated trinary choice. Participants vote on nine voting questions. Each question asks participants to vote for one of two projects or no project. The pair of included projects varies across voting questions. One vote is randomly selected to be binding. The option with the most votes (for the randomly selected question) is implemented for real.

There are other important sources of variation:
- For all treatments, the cost of a given project is randomly varied across participants. This allows for identification of willingness to pay.
- Half of the repeated binary choice participants vote on the "small" project first. The other half vote on the "large" project first.
- Half of the repeated trinary choice participants first face a choice set (vote) that includes the "small" project, the "large" project, and "no project" as voting options.
- Aside from deliberate placement of "small"/"large" project voting questions, the voting questions in the nine-question sequences are randomly ordered.

Randomization is computer automated (i.e., we use Qualtrics' "randomizer" tool).

Individual

Yes

1200 respondents

An observation is a single vote from a single participant. With 300 participants per treatment, this yields 300 observations for each of the two single binary choice treatments and 2700 observations for each of the repeated choice treatments. Total number of observations is 6000.

300 respondents in each of the four treatments.

Comparison of vote proportions. This includes: (1) single binary choice "small" versus "large"; and (2) comparing the single binary choice ("small" or "large") to the same vote when placed in the binary choice voting sequence. Based on data from a pilot experiment, the MDE is a 10-point difference in the percentage of "yes" (for project) votes. Comparison of willingness to pay. For the repeated choice treatments, WTP can be estimated either by pooling all the data or focusing on a single vote. As the cleanest comparison across treatments involve what happened in a single vote, we focus on this case here. Data from a pilot experiment suggests an MDE in the range of 89 cents to $1.16.