Politics of Food

Information on the intervention is hidden until the end of the trial.

2024-04-15

2024-05-06

1. Demand for orange juice

Demand for orange juice is our most important primary outcome. This outcome is elicited during the obfuscated follow-up survey (see Experimental Design). Specifically, we measure the willingness to pay for an $8 product voucher for Tropicana’s orange juice using a multiple price list (MPL).

2. Policy support index

This outcome is measured during the first survey and is intended to summarize participants’ support for the policy of spraying citrus crops with antibiotics in order to combat citrus greening.

3. USPIRG donation

This outcome is measured during the first survey. We measure the amount donated (out of a $1 bonus payment) to the United States Public Interest Research Group (USPIRG), a nonpartisan, non-profit organization that advocates against streptomycin spraying on citrus crops.

Heterogeneity:

We will look at heterogeneity of the treatment effects by gender and moral type (universalist vs. communal). Moreover, we will consider heterogeneity of the treatment effect with respect to whether EPA’s name was explicitly mentioned or replaced by “regulatory agency” (we randomized this in the survey).

Lastly, we will analyze heterogeneity of the treatment effect with respect to past consumption of orange juice. It is natural to expect that elasticity to information will differ for never consumers, marginal consumers, and habitual consumers. First, for outcomes collected in the first survey we will compare the treatment effect for OJ CONSUMERS and OJ NEVER CONSUMERS (see Experimental Design for the definitions). Second, for all outcomes, we will also perform a more granular heterogeneity analysis by comparing people who indicated that in the past month they (i) never consumed orange juice, (ii) consumed orange juice once/twice a month or once a week, (iii) consumed orange juice a few times a week or everyday.

Re 1: Participants make a series of choices between a cash bonus payment of a particular amount and an $8 Amazon gift card for orange juice (see Experimental Design section for more on how we introduce product-specific gift cards to participants). Specifically, in each question, we tell participants to “click on the choice that they prefer”. We also display a warning “think carefully, if the computer randomly selects this question, you will receive what you choose below.”

We use a dynamic MPL that imposes monotonicity (we never ask about selections that allow inconsistency with previous choices). We start by asking whether participants prefer $4 in cash or an $8 gift card for Tropicana orange juice. If they select the former, we will then ask for a preference between $2 in cash vs. an $8 gift card for orange juice. If they select the latter, we will ask them to choose between $6 and an $8 gift card for orange juice. This process continues until we know the willingness to pay with $0.5 precision. If the process indicates that a person has WTP above $7.5, we ask a text entry question about the value of cash that would make them indifferent between receiving cash and receiving the voucher. Participants cannot enter a value lower than $7.5.

After the MPL concludes, the computer randomizes the value of the cash bonus between $0.5 and $8 with increments of $0.5 (i.e., $0.5, $1, $1.5, ..., $7, $7.5, $8). Each value is equally likely. Randomizing the value of the cash bonus always corresponds to one of the possible questions from the MPL.

Our procedure allows us to know if people’s willingness to pay is between $0 and $0.5, $0.5 and $1, $1 and $1.5 etc. When recording the value of the outcome we use the middle of each band. So for example, for the person whose WTP is between $0.5 and $1, we will record the outcome as $0.75. When testing for hypotheses, we cap the WTP at $8, even though participants can enter higher values using the text entry question.

Lastly, if the randomized value of the cash bonus exceeds the WTP, participants receive the cash bonus. If the opposite happens, they receive an $8 gift card for orange juice.

Re 2: The index is based on participants’ agreement (on a scale from 0 to 100) with four statements, two of which imply agreement with the policy and two indicating disagreement. The statements are as follows:
1) The policy is safe as it is endorsed by the relevant government agency.
2) The policy has negative consequences on people's health.
3) I think that antibiotic spraying of citrus crops should be outlawed.
4) I support the policy as it helps protect the economy with no major risks involved.

To construct the index, we first standardize agreement scores for each statement. Then, the scores for statements that indicate disagreement are multiplied by −1. Lastly, we compute the sum of the sign-adjusted standardized scores.

Re 3: For each individual, we will report the proportion of the bonus payment donated to the USPIRG. For example, if a participant donates $0.3 to the organization and keeps $0.7 for themselves, the value of the outcome will be 0.3.

Re Heterogeneity: Moral universalism is measured using a selection of moral relevance statements from Enke (2020). Participants are asked “When you decide whether something is right or wrong, to what extent are the following considerations relevant to your thinking?” The considerations are as follows:
1) Whether or not someone suffered emotionally.
2) Whether or not some people were treated differently than others.
3) Whether or not someone’s action showed love for his or her country.
4) Whether or not someone showed a lack of respect for authority.
5) Whether or not someone cared for someone weak or vulnerable.
6) Whether or not someone acted unfairly.
7) Whether or not someone did something to betray his or her group.
8) Whether or not someone conformed to the traditions of society.

For each consideration, participants can choose one of the following options: not at all relevant, not very relevant, slightly relevant, somewhat relevant, very relevant, extremely relevant. These choices are assigned values from 0 (not at all relevant) to 5 (extremely relevant). To compute the index of moral universalism, we add scores for considerations consistent with universalist moral type (1, 2, 5, 6) and subtract scores for considerations consistent with communal moral type (3, 4, 7, 8).

Having computed the index, we will analyze heterogeneity of treatment effects by whether someone has an above-median (more universalist) or a below-median score (more communal).

1. Trust index

This outcome is measured during the first survey and is intended to summarize participants’ trust in EPA’s arguments regarding the safety of antibiotic spraying of citrus crops. The outcome is measured only for participants in the TRUMP or BIDEN groups. We cannot elicit it in the CONTROL as participants in that group are not informed about EPA’s arguments.

2. Beliefs about policy support by political affiliation

This outcome is measured during the first survey. We separately elicit participants’ beliefs about the share of Democrats and Republicans who support the policy of spraying citrus crops with antibiotics in order to fight citrus greening. Participants are rewarded for accuracy. They receive $0.25 for each prediction within +/- 3 percentage points of the true value (as taken from a representative survey).

3. Planned orange juice consumption (with rationale)

This outcome is measured only for a randomly selected 15% of participants in the TRUMP and BIDEN groups during the first survey. We ask participants about the likelihood (0-100) of purchasing orange-based products (such as orange juice) or other citrus-based products in the upcoming month.

We follow up on this question by asking to what extent (0-100) the following factors were a consideration in their decision: (1) I am concerned about the health effects of directly consuming citrus products, (2) I am concerned about contributing to antibiotic resistance in bacteria.

Heterogeneity:

We will perform the same heterogeneity analysis for secondary outcomes as for the primary outcomes.

Re 1: The index is based on participants’ agreement (on a scale from 0 to 100) with four statements, two of which imply trust in EPA’s arguments and two indicating lack of trust. The statements are as follows:
1) I trust the [EPA/regulatory agency] which determined that the policy is safe.
2) The arguments provided by the opponents of the policy were convincing.
3) The scientific evaluation conducted by the [EPA/regulatory agency] was thorough and I trust it.
4) The opponents of the policy are scaremongering.

To construct the index, we first standardize agreement scores for each statement. Then, the scores for statements that indicate less trust are multiplied by −1. Lastly, we compute the sum of the sign-adjusted standardized scores.

Information on the experimental design is hidden until the end of the trial.

We recruit participants on Prolific. We restrict the pool of eligible participants to those who answered "Democrat" on the Prolific's US political affiliation question. Participants are required to complete a 15-minute Qualtrics survey about a public policy issue. The survey begins with demographic questions. As a part of this section, we elicit consumption and purchases of three basic products (milk, bread, orange juice) during the past month. We separately ask participants to indicate if, in general, they never consume any of these products. We classify people as OJ NEVER CONSUMERS if they answered that they did not purchase or consume orange juice in the last month and that they generally do not buy orange juice. Everyone else is classified as OJ CONSUMERS. Furthermore, the demographics section contains a short version of the moral foundations questionnaire, as applied by Enke (2020).

Following the demographics section, participants learn about citrus greening disease and the damage that it has caused to citrus crops. They also learn about regulation allowing large quantities of antibiotics to be sprayed on citrus crops to combat the disease.

Subsequently, participants are randomized into one of three treatment groups: TRUMP, BIDEN, or CONTROL. Random assignment of treatment is performed separately for OJ CONSUMERS and OJ NEVER CONSUMERS. In both cases, the probability of being assigned the TRUMP group and the BIDEN group is 40%, whereas the probability of being assigned the CONTROL group is 20%.

Participants in both the TRUMP and BIDEN groups learn about the opposition to the policy, including arguments made by issue advocacy groups that focus on increasing antibiotic resistance and direct effects of exposure.

This is followed by information about the Environmental Protection Agency’s (EPA’s) authorization decision to allow streptomycin spraying. In the TRUMP group, we highlight the Trump administration’s support for the authorization and we state that “the EPA under the administration of President Donald Trump provided arguments and scientific evidence supporting the position that antibiotic spraying of citrus crops poses little risk and meets the regulatory and safety standards." We also provide a photo of Donald Trump (we randomize whether it is a photo of Trump shaking hands with his EPA head appointee or Trump’s portrait). The BIDEN treatment is identical to the TRUMP treatment, except that we replace all mentions of Trump with Biden and provide Biden’s photos instead (either Biden shaking hands with his EPA head appointee or Biden’s portrait).

On the next screen, we provide a list of specific arguments made by EPA’s scientists in favor of the policy. The arguments do NOT vary by treatment. The photo of the relevant president (the same photo as in the previous screen) is shown on the page. In both the BIDEN and TRUMP groups, we additionally randomize whether we explicitly mention the EPA or not. In the latter case, "EPA" is replaced with “regulatory agency” throughout the survey.

Participants in the CONTROL group do not see information on the opposition to the policy and the EPA’s final authorization, including the scientific arguments.

The last screen before outcome elicitation, shown to participants in all groups, contains a map of the US with four states responsible for almost all commercial citrus production highlighted (according to USDA report). Participants are reminded that “the policy allowing streptomycin spraying on citrus crops remained in force in these states as of December 2023.”

We first elicit support for the policy of streptomycin spraying. This is done in two ways: (1) policy support index and (2) USPIRG donation question (see the Outcomes section for details). The order of outcomes (1) and (2) is randomly selected. Furthermore, the order of questions in outcome (1) is randomly assigned.

Subsequently, participants in TRUMP and BIDEN groups complete the trust index, which is a series of questions about their trust in EPA’s arguments about the safety of streptomycin spraying. For a randomly selected 15% of the participants in TRUMP and BIDEN groups, this is followed by a question about planned orange juice consumption. Lastly, in all groups, we elicit incentivized beliefs about the support for the policy of antibiotic spraying of citrus crops among Democrats and Republicans.

Participants classified as OJ CONSUMERS will be invited to take an obfuscated follow-up survey.

After asking many placebo questions about preferred brands of products such as soda, margarine, or shampoo, and recalling advertisements by various brands, we use MPL to elicit willingness to pay for an Amazon gift card for one of the products from the brands that we asked about in the previous section. This product is randomly selected to be either orange juice or shampoo (the former option has a 99% chance). Specifically, the MPL gives us the WTP for an $8 product voucher for Tropicana orange juice. Participants know that it is a product-specific Amazon gift card that cannot be used on ineligible products. We focus on Tropicana orange juice, which is the key available item. However, we highlight that other brands of orange juice and other citrus drinks of similar quality are also eligible. Importantly, the participants expect all eligible products to be citrus-based, meaning that the safety considerations related to antibiotic spraying certainly apply.

Notes on hypothesis testing:

1. Broadly speaking, we test the following hypotheses using comparisons between the TRUMP and BIDEN groups.

We (1) analyze whether the political affiliation of the presidential administration overseeing the relevant federal regulator changes consumers’ (a) support for the antibiotic usage policy, (b) willingness to donate to groups that oppose the policy, (c) trust in scientific validity, (d) beliefs about the policy support by party affiliation. Furthermore, we (2) assess the impacts on consumer demand for orange juice.

The Outcomes section provides more details. It also crystalizes the priority that we attach to each outcome.

2. Separately, we use comparisons between the CONTROL group and the TRUMP/BIDEN groups to test whether knowledge of opponents’ arguments and EPA’s defense of the antibiotic spraying policy affects the outcomes listed above.

3. For all hypothesis testing, we will use a two-sided t-test for difference in means between pairs of treatment groups unless otherwise stated.

4. One exception is the following hypothesis related to the first primary outcome: whether the political affiliation of the presidential administration overseeing the relevant federal regulator changes consumers’ demand for orange juice. Here, we specifically test whether oversight by the ingroup, as opposed to the outgroup, presidential administration INCREASES willingness to pay for the $8 orange juice voucher. As a result, we will use a one-sided t-test for difference in means between TRUMP and BIDEN groups.

There are two main reasons for this choice. First, the theoretical prediction is clear regarding the possible direction of the effect. Oversight by the ingroup presidential administration might increase trust in EPA’s arguments, alleviating safety concerns related to the consumption of orange juice, thus increasing the WTP for the voucher. It is hard to argue that a treatment effect in the opposite direction is a well-founded possibility. Second, a major constraint that we face is the limited number of potential participants on Prolific who are Democrats. Our power calculation suggests 1,400 individuals per treatment arm for this outcome for a one-sided test, which already likely exhausts the subject pool available to us.

5. For the USPIRG donation question and the WTP for an $8 voucher for orange juice, we will additionally perform the Epps–Singleton characteristic function test of equality of distributions of these outcomes by pairs of treatment groups. However, we highlight that these tests will be considered secondary to the tests for differences in means when evaluating the hypotheses.

6. We want to highlight that, ex ante, we do not necessarily expect that a potential negative treatment effect of being in the TRUMP group (vs. the BIDEN group) on the support for antibiotic spraying of citrus crops has to translate to higher donations to the USPIRG, an organization opposing the policy. This may be because in the TRUMP group participants assume that Democratic politicians are against the policy and can be trusted with eventually overturning the policy, thus crowding out the need for donations to NGOs. In the BIDEN group, participants likely believe that Democratic politicians support the policy, which means that a Democratic opponent of the policy considers donating money to relevant NGOs worth it. This mechanism was suggested by Kashner and Stalinski (2024) in the context of donations to a net neutrality charity as an explanation for heterogeneous effects of their treatment by political affiliation. It is interesting to test whether the direct support channel (donation amounts follow support levels) is stronger/weaker than the crowding out effect.

7. We cross-randomize several characteristics with the main treatment groups (TRUMP, BIDEN). For example, we randomize whether we mention EPA’s name explicitly or not. These additional dimensions of randomization allow us to perform extra heterogeneity analysis or serve as robustness checks. However, the changes that they introduce are minor in comparison to the main intervention, which is varying the political affiliation of the administration overseeing the EPA while holding the scientific arguments about the safety of the antibiotic spraying policy constant. Consequently, when testing all hypotheses, we use the entirety of the sample size per treatment arm (TRUMP vs. BIDEN), regardless of the values of the additionally randomized characteristics.

Qualtrics randomization

Individual

No

N/A

We plan to recruit 5,470 individuals to complete the first survey. This is to ensure that we have approximately 3,500 individuals who take up the obfuscated follow up survey, where we elicit one of the primary outcomes (demand for orange juice). Informed by pilot results, we expect 20% of participants to indicate in the first survey that they never consume orange juice (we do not invite such individuals to take up the follow-up study). Furthermore, we assume a 20% attrition rate (not taking up the follow-up survey) among people who consume orange juice at least occasionally. Notes on recruitment: Our intended number of participants in the first survey (5,470) is high relative to the available subject pool on Prolific (conditional on the necessary pre-screeners). If recruitment of the entire sample of this size is unsuccessful on Prolific in a reasonable time span, we may either perform additional recruitment using a different platform, or include pilot observations in the analysis. If either of these happens, we will clearly indicate any differences in results by method of recruitment.

For the TRUMP and BIDEN groups, we aim to recruit 2,188 individuals per treatment arm to complete the first survey. Additionally, we aim to recruit 1,094 individuals for the CONTROL group. Overall, 40% of participants will be assigned the TRUMP group, 40% will be assigned the BIDEN group, and 20% will be assigned the CONTROL group.

Overall, our recruitment should translate into approximately 1,400 individuals per arm in the TRUMP and BIDEN groups who take up the follow-up survey. The corresponding number for the CONTROL group is 700.

For the primary outcome in the follow-up survey (demand for orange juice), the minimum detectable effect is 0.094 s.d (TRUMP vs. BIDEN comparison). This accounts for our attrition assumptions, as explained above. Choosing a sample size consistent with this MDE allows us to detect a treatment effect in line with our pilot results. Furthermore, it is lower than the minimum MDE recommended for information provision experiments (0.15 s.d.) by Haaland et al. (2023), and takes into account that certain types of outcomes, including demand, are more inelastic to information interventions than other outcomes typical for this type of experiment. For outcomes measured in the first survey, the corresponding minimum detectable effect is 0.085 s.d. This takes into account a higher sample size per arm in the first survey (no attrition) and the use of a two-sided test.