Food Swaps: Defaults vs Active Choice Using an experimental online supermarket

Following a baseline shopping task in week 1, the same for all participants and prior to week 2, participants will be (stratified and) randomly assigned to one of four experimental arms:
The four experimental arms, applied in week 2, are:
1. Control: No swap mechanism.
2. Default opt-in: Swap with original grocery preselected.
3. Default opt-out: Swap with alternative grocery preselected.
4. Active choice: Swap with neither original or alternative pre-selected.

In week 1 of recruitment, participants answer some survey questions and complete the baseline shopping task (common to all participants). For week 2, participants will be allocated to our four experimental arms through stratified randomisation based on variables considered relevant for our eco shopping choices (gender, income, education and eco ranking of baseline shop from week 1). Those assigned to the control experimental arm, will have access to the same shopping environment and conditions as in week 1, but with the addition of the swap mechanism previously described, and with one of the 3 possible designs (opt-in, opt-out, active choice). In all cases, and in both weeks the shopping is incentive compatible with participants given a maximum and minimum spend, and with an opportunity to win one of their week’s baskets and any unspent budget in cash. The typical weekly shop on an experimental online grocery shopping platform, Woods (https://woodssupermarket.co.uk). Participants’ sense of consumer green identity and satisfaction will be measured post-shop in weeks 1 and 2. Their willingness to pay for the swap (with and without the eco and nutri information (Clark et al., 2020)) will be an incentive compatible multiple price list, where participants indicate their willingness to pay for these if they are invited to a third week of shopping.

The swaps, in all treatment arms, suggest an alternative grocery item from the same ‘shelf’ category as the participant’s initial choice. The swaps are proposed if and when an alternative priced within 25% of the original chosen product, that is has a better eco score and at least same nutri score, is available. Across the treatments the only difference between the different swap designs is whether and if the original chosen product or the suggested alternative is preselected.

These treatment arms lead to our main hypotheses:
Hypothesis 1: Effect of Swaps. Testing ‘default’ theory in swaps.
H1_a: The inclusion of swaps leads to a greater reduction in the average environmental impact (per 100g and per GBP spent) of shopping baskets in the intervention group, compared to the change in the control group (experimental arm 1) from baseline (week 1) to follow-up (week 2).

H1_b: The default opt-in swap leads to the smallest reduction in the average environmental impact (per 100g and per GBP spent) of shopping baskets in the intervention group, compared to control (experimental arm 1) from baseline (week 1) to follow-up (week 2), relative to experimental arms 3 and 4.

H1_c: Theory suggests that default opt-out swap leads to the biggest reduction in the average environmental impact (per 100g and per GBP spent) of shopping baskets in the intervention group, compared to control (experimental arm 1) from baseline (week 1) to follow-up (week 2), relative to experimental arms 2 and 3. However, reactance to the opt-out design could lead to an ambiguous difference in change in baskets from week 1 to 2 between default opt-out and active choice.

To evaluate the average treatment effect of our manipulations we follow (Panzone et al., 2021, 2024) by using a difference-in-difference (DID) estimator (Bertrand et al. 2004; Imbens and Wooldridge 2009; Wing et al. 2018). We remember that participants shop in two separate weeks, called tasks. Task 1 has no intervention, regardless of the participant. For Task 2, individuals are randomly assigned (through stratified randomisation, as previously described) to one of four experimental groups k = 1, 2, 3 and 4. Here, k=1, here our pure control, will be the reference basket. Within each shopping task t = week 1 and week 2 participants i will make their purchase decisions, choosing food items to fill basket Ci;t which will have a normalised eco and nutri value (per grammes of basket and GBP spent). The average treatment effect (ATE) will then be estimated as the difference between the average changes in basket values observed from week 1 to 2:

Swap Effect: \emptyset_{k;2}=Ek;2-Ek;1]-[E1;2 - E1;1 ∀k∈{2;3;4}

Where \left[\bar{E_{k;t}}\right] is the average eco score value of individuals in experimental group k in task t.
For our analysis, we perform a log-linear panel regression for concerns of non-normality and heteroskedasticity of the dependent variable (Panzone et al.’s research suggests data will not be normal, we will test or data for normality but also expect non-normality):
\ ln(E_{i;t})=\ \alpha_{1;i}\ +\ \sum_{k=1}^{4}{\alpha_{1;i}G_{i;k}\ +}\ \alpha_{1;i}W_t\ +\ \sum_{k=1}^{4}{\Pi_{k;t}W_tG_{i;k}\ +}\ \varepsilon_{i;t}

\alpha_{1;i}: Individual’s fixed effects
G_{1;i}: Treatment dummy variable: 1 if k=1, 2 if k=2, etc
W_t: Time dummy capturing the task number t.
\varepsilon_{i;t}: Error term.

To convert the DID interaction term \Pi_{k;t} back to our desired ATE estimate, we use the Puhani (2012) transformation:
\emptyset_{i;k;t}=\ exp\ (ln(E_{i;t}))\ -\ exp(E_{i;t}\ \ -\ \Pi_{k;t}W_t\ )
NOTES: We also note that Panzone et al. (2024) use a 10% significance level for judgements on significance due to inherent noisiness and overestimation of standard errors (Betrand et al., 2004) in DID. To minimise some of the individual-level heterogeneity in shopping choices, our study design incorporates a prescriptive shopping list.

H1_d: Default opt-out and active choice leads to a higher likelihood of accepting swaps than default opt-in design swaps.
This will be empirically tested with mixed-effect (or multi-level) logistic regression, considering whether consumers accepted or rejected swaps during the shopping task. This is more of a secondary analysis, and the experiment is not primarily powered for this study.

Hypothesis 2: Identity Effect. Testing ‘identity-action’ theory (Benabou et Tirole, 2011)
H2: Accepting green alternatives in ‘active choice’ has a greater green consumer identity increase, from week 1 to week 2, than accepting the green alternative pre-selected in default opt-out.
This is studied with the same Panzone et al. difference-in-difference strategy as above. In this hypothesis, the outcome variable is drawn from the consumer identity question asked in the survey post-shopping task in week 1 and 2. Theory only suggests a hypothesis for active choice vs default opt-out. A comparison of effect between opt-in and active choice will be exploratory, with no priors.
Exploratory Analysis:
We will conduct additional exploratory analysis to investigate the following research questions:

Does default op-out lead to a lower willingness-to-pay for the swaps.
Does consumer search effort negatively predict the likelihood of accepting a swap?
Does an increase in consumer green identity lead to greater search effort and choice of more eco goods (moral consistency, self-image consistency).
Does impulsivity moderate search effort and whether a swap alternative is accepted.

2025-04-21

2025-04-24

Primary Outcomes (end points): This study has two main outcome variables.
First primary variable: Eco-score of the final shopping baskets from Week 1 (pre-intervention baseline) and Week 2 (post-intervention treatment) based on the final product choices made by all participants in both weeks (in a shopping budget of £25-50). Eco-scores are based on calculations by Clark et al. (2022). The final basket eco scores will be re-balanced per 100g of produce bought and per GBP spent.

Second primary variable: Consumer green identity changes from Week 1 (pre-intervention baseline) and Week 2 (post-intervention treatment) based on survey questions from (van der Werff et al., 2014; Guidetti et al., 2023).

First primary outcome variable: Eco basket.
In both weeks of the shopping task, we collect data on the products each participant selects in the experimental online supermarket. We use this data to compute a basket-level eco score, which reflects the average environmental impact of the chosen items, normalized per 100g and per GBP spent.
This outcome allows us to test whether the inclusion of swap, and which swap design leads to meaningfully more eco-friendly purchasing behaviour, as reflected by the grocery basket eco score.

Second primary variable: Eco Identity.
We capture Green Consumer Identity through a series of statements to which participants indicate their degree of agreement (following Guidetti et al, 2023). The identity measures are given after the shopping tasks in week 1 and 2.
This outcome variables allows to test whether active choice swaps can help boost consumer green identity compared to the default designs.

Our experiment includes secondary variables.

Our other secondary variable measured in both week 1 and 2, is search effort in the shopping task (based on time spent going through the shopping platform). Through the experimental online supermarket platform and the survey platform, we will get these two separate measures for each participant each week.

We also collect the data on whether proposed swaps were accepted or not.

In addition to these two key secondary variables, we collect data on socio-demographics (age, income, weekly grocery expenditure, children education, etc). We also collect preference elicitation data for the cognitive reflection test, locus of control, self-control.

The search effort variable should be a strong moderating factor for the effect of the ‘sort by’ ordering. Specifically, we expect consumers with a shorter search effort to be more affected by a ‘sort by’ ordering, in our case, with a more eco basket when allocated to the green ‘sort by’ ordering, regardless of it was desired or not.

Power willing, knowing the when swaps were accepted or not by each participant, we can test for different predictive factors of accepting swaps, including the swap design, the food category, and more widely socio-demographic factors and preference elicitation variables.

All the other variables are for exploratory purposes. These will be used for mechanism and heterogeneity analysis.

Similar to (Panzone et al., 2021a, 2021b; Zizzo, 2010), we will employ a between and within-subject experimental design, where participants are recruited to shop twice on our simulated grocery store. Week 1 serves as a baseline, where all participants are asked to do a weekly shop on the platform. Participants will then be randomised into one of 4 conditions using a stratified randomisation procedure and will be invited to complete a second survey and shopping task in the following week. Importantly, our experimental design will be incentive-compatible and can be classified as a framed field experiment (List, 2011).

Participants
A UK adult (18+) representative (as defined by Prolific) population sample, and pre-screened by Prolific for diet (to avoid participants with strict dietary requirements, such as veganism, Halal or Kosher requirements), online shopper (to select those that interact with online grocery shopping at least a few times a year) and primary household shopper (to get the main grocery shoppers) is initially recruited through the Prolific recruitment platform. We will also track whether individuals have other dietaries (vegetarianism, food allergies and intolerances, whom we might exclude from our main analysis at a later date.

Experimental Procedure
Week 1:
After consenting to the general details of the experiment, participants will be asked a series of socio-demographic and preference elicitation questions, as described in the Secondary Outcome variable section.

Then all participants will be sent to an experimental online supermarket (Woods Supermarket) where they will be asked to do their typical weekly shop within a budget of £25 and £50, which should include: Soup - Breakfast Cereal - Ready Meal (to heat up) - Sandwich - Pizza - Fruit/Veg. The prescribed shopping list is designed to reduce between-subject variability driven by purchase preferences (taste, diet, culture) by providing products that all participants need to add to their baskets in both weeks 1 and 2 (in the treatment arms). The remainder of the budget can be spent on the additional items of their typical weekly shop.
The week 1 shopping platform is set exactly the same for everyone.

For the incentive compatible element of the experiment, each participant has a chance to win their basket of food and unspent portion of their budget in the real world (5 participants will be randomly selected across the full sample). If they are randomly drawn, they will receive a Click and Collect code with their food basket ready made from one of their randomly selected baskets from their week 1 or week 2 choices. This is to collect at a Tesco’s store of their choice. Any unspent budget will be paid to them through Prolific. Both of these aspects maintain the anonymity of the winning participants.

After the shopping task is complete, participants are sent to a post-shop survey where additional preferences and beliefs are elicited, as well as answering questions on their consumer satisfaction with eth shopping experience, their sense of green identity, and where they are shopping.

Week 2:
Participants are stratified and randomly assigned to one of the four experimental arms available in week 2. From the beginning of week 2, participants are sent to the same experimental online shopping task, as week 1. In the shopping task, participants are required to follow the same budget and shopping requirements as week 1. The only difference between the groups is the presence of a swap and its design, as previously described. Swap suggestions are consistent across all three of the swap treatment arms (Product x is always offered if Product y is initially chosen).

Once the shopping task is complete, participants as forwarded to a survey, with follow-up survey questions on consumer satisfaction and green identity (idem week 1), as well some further preference elicitation questions.

To allocate the participants to one of the 4 experimental arms in week 2, we use a stratified randomisation technique, based on gender, eco score of participants’ week 1 shopping task baskets (as eco terciles).

The unit of randomisation is at the individual level.

No

N/A

We collect 2-week data (from 2 pre- and post-shop surveys and two shopping tasks) from c.960 participants for the whole experiment (spread over 4 treatments, 240 per treatment). This sample size is based on previous papers of a similar nature and power calculation simulations for linear and log-linear difference-in-difference (detail below) and for a multilevel logistic and linear regression analysis. This does not cover likely non-completion of the survey and experiments, attrition across the two-week study, comprehension and attention check failures, and budgetary constraints.

The recruitment sample size collected for week 1 is 280 participants per treatment arm. This sample size is based on previous papers of a similar nature and power calculations. This accounts for non-completion and attrition in the range of c.20% across the two-week study.

In the absence of prior data, we simulate required sample sizes using Stata 18, targeting 80% power at a 5% significance level for detecting a standardized effect size (Cohen’s d) in the range of 0.2 (conservative compared to literature reviews that suggest a medium Cohen’s d effect on default designs). For the difference-in-difference simulation, we also estimate a 0.8 correlation between a participant’s basket in week 1 and week 2 (based on Panzone et al., 2021). We also compare minimum sample size requirements for linear and log-linear difference-in-difference (DiD) regressions. For a Cohen’s d of 0.2, the simulations suggest a required sample of approximately 165 participants per treatment arm (for both linear and log-linear DiD specifications). A power calculation for the multilevel logistic regression (likelihood to swap hypothesis), assuming 4 swaps per person (not independent), Cohen’s d of 0.2 and 80% power at a 5% significance level, with 4 experimental arms, we get an estimation of requiring 220 people per treatment. We find that the cross-sectional linear regression analysis (willingness to pay hypothesis) with the same statistical requirements than the logistic regression above, also requires roughly 220 people per treatment arm. All our power calculations are done with 1000 repetitions. The big unknown is the attrition across weeks and dropouts within each week (from survey, to online shopping and back to survey). Also allowing for a buffer on our power calculation results, assume we should add a little over 20% on our results. 220*1.2=264, so we round up to 280.