Evaluation of Interventions in Online Grocery Shopping for Sustainability and Health: An Adaptive Design Randomized Controlled Trial

We will evaluate the following two interventions:

1. ​​​Eco-labelling: Eco-labels provide consumers with information on the environmental impact of their food purchases, graded from A (lowest impact, most sustainable) to G (highest impact, least sustainable) based on an underlying numerical score ranging from 0 (most sustainable) to 500 (least sustainable). These environmental impact scores were calculated based on the ingredients in 1kg of each product, which are then linked to the life-cycle analysis (LCA) database Agribalyse. The scores take into account the environmental impact of a product in 16 categories, including land use, water scarcity, resource use, human health, wildlife damage, and climate change. More details on the methodology are provided in appendix A.

2. Price discounts on more sustainable alternative products: Price discounts will be offered through swaps at the first checkout screen. Participants will see a pop-up window suggesting a more sustainable alternative for a particular product in their basket, which is offered at a randomly varying price discount. The alternative products are selected to not cost £2.00 more than the original product, and price discounts will be offered at three levels: £1.00, £0.50 or no discount. The intention is that this will result in discounts being roughly equally spaced between 0 and 100% of the price difference between the original and the alternative products (explicitly offering percentage discounts is not feasible for technical reasons, because it would involve “live” scraping of the prices). We will use this information to calculate the willingness to pay for more sustainable products. As an example, a participant may be prompted to swap Greek yoghurt (500g) priced at £2.30 for dairy-free coconut yoghurt (600g) priced at £3.60, and the pop-up will notify participants that they can buy the coconut yoghurt for £2.60, £3.10 or the full price of £3.60, depending on the trial arm that the participant was assigned to. If the shopper accepts the swap, they will be reimbursed for the price discount. Each participant will get offered a swap for a (potentially discounted) alternative product on up to 3 products in their shopping basket. The products for which a swap is offered are randomly chosen from a list, composed by the researchers, of not-so-sustainable products with suitable alternatives. To avoid offering alternative products with an inferior macronutrient profile, alternative products on the list will be selected so that they are in the same or a better category for fat, sugar and salt content on the traffic-light label.

2024-04-15

2024-11-30

The average eco-score of the basket of groceries purchased with a lower score meaning more sustainable purchases.2 Using this outcome variable, we will estimate the effect size of introducing eco-labels (intervention 1) and the price-elasticity for sustainable alternative products (intervention 2), allowing us to calculate the willingness to pay for sustainable groceries. We aim to measure both short- and long-term responses to price discounts to see if trying a more sustainable alternative may shift demand persistently beyond a one-time purchase.

Basket eco-score for 16 impact categories of the Product Environmental Footprint (PEF): climate change, water use, resource use: minerals and metals, resource use: fossils, land use, ozone depletion, human toxicity: cancer, human toxicity: non-cancer, ionising radiation and human health, particulate matter, eutrophication: marine, acidification, eutrophication: terrestrial, eutrophication: freshwater, ecotoxicity: freshwater). This outcome will give some insight into what type of improvements can be expected from more sustainable grocery shopping because of our interventions. It is likely that the largest sustainability gains can be realised on the impact category climate change (greenhouse gas emissions).

The nutritional value of the basket of groceries purchased, as measured by the Food Standard Agency’s (FSA) Nutrient Profiling Model. We will use the changes in the nutritional composition of groceries purchased to model the health impact of our interventions if these were delivered at scale in the UK using the PRIMEtime model, a multi-state life table model, to calculate the impact of the interventions on diet-related diseases.

Total cost of the shopping basket: The total cost of the shopping basket per household expressed in £, will be used, to understand the impact on grocery costs.

Total number of products in the basket. This is primarily to check whether shoppers respond to eco-labels by buying more (less sustainable) products elsewhere (e.g. by adding them to their order on the mobile, or by buying them at a different supermarket).

Setting, Recruitment and Study Design

This protocol has been informed by a pilot study, which was run in September and October 2023. Methods and results of the pilot are presented in Bentil et al. (2024).

Our setting is the online grocery shopping environment on the website of a large UK supermarket. We will use a browser extension (plug-in) to implement the interventions and collect purchase data. Underlying the plug-in is a database with information on the environmental impact of over 160 thousand products. The plug-in was developed by Sustained (https://sustained.com) and customised based on our specifications. It can be used to display eco-labels and/or offer price discounts on more sustainable alternative products, depending on the participant’s ID, which is randomly allocated to one of the arms of the trial. The plug-in also records information on products in the shopping basket, any swaps or clicks on the eco-labels, and products that are eventually purchased.

Participants will be recruited from the online research platform, Prolific (https://www.prolific.com/). To be eligible for the study, participants must:
- Be 18 years or older and located in the UK
- Be the primary grocery shopper of their household
- Frequently buy groceries online (at least once per month, self-reported)
- Usually shop at the supermarket(s) included in this study
- Usually use a laptop or desktop with the Google Chrome browser for online grocery shopping or be willing to do so for the duration of the study
- Consent to participate and be willing to download and install the plug-in and use it for the duration of the study
- Not have participated in the pilot study or previous wave(s) of the study.

These eligibility criteria are informed by the pilot study. We found that take-up is much lower among participants who were asked to shop at a different supermarket than where they usually shop, but participants who usually shop on a mobile device and therefore need to change their shopping behaviour to use the browser extension are no less likely to shop using the extension than those who usually use a desktop or laptop (Bentil et al., 2024).

The study will use a cross-sectional Randomised Controlled Trial (RCT) design. For participants in the intervention group, the intervention will be on from the start and there will not be a baseline period. The reason for this is that the results from our pilot study showed that inter-participant variation in the primary outcome was similar to intra-participant variation. Therefore, there is no efficiency gain from controlling for individual-specific fixed effects by comparing changes rather than levels of purchases. In a longitudinal design, some observations are lost, because some participants did not shop either in the baseline or in the intervention period, and our pilot results consistently showed a more precise estimation of the treatment effect in a cross-sectional design.

The trial will run over 8 weeks. If self-reported shopping frequency were accurate, our eligibility criteria would guarantee 2 shops per participant in a one-month trial. However, the pilot study showed that participants were shopping substantially less frequently than they said they would. On the other hand, we found that attrition did not increase over the one-month duration of the pilot, suggesting that a longer trial duration would be preferable. After the end of the trial period, participants are allowed and encouraged to keep using the plug-in and we will continue to collect their purchase data for use in future research. We will explicitly ask for consent for this post-intervention period of data collection at the start of the trial.

We will use an adaptive design for the RCT (Figure 3). An adaptive design, rather than the traditional fixed sample size RCT design, was chosen because uncertainty regarding recruitment rates and attrition makes determining the sample size challenging. The adaptive design will enable a reassessment of the sample size requirements and the potential to stop the trial early should interim analyses indicate success or futility, ensuring a more efficient allocation of resources and time ​(Pallmann et al., 2018)​.

The study will thus be implemented in phases or waves. We will approach Prolific panellists in four or five waves of 10,000 people, with the aim of recruiting around 700 participants in each wave, until we reach the sample size required for a 2-arm trial with a 2-month intervention period or until it becomes clear that it is futile to try and reach this sample size. As illustrated in Figure 4, we will use data from wave 1, anticipated to start in April 2024, to determine the number of people who shop during the trial (recruitment rate). Based on this information, we will recalculate the sample size requirements and continue with wave 2 only if it seems likely that after a maximum of five waves, we will successfully recruit a sufficiently large sample. This exercise is repeated after each wave. Between the second and third waves there is a two-months break to avoid the summer holidays, when shopping patterns may be atypical. We will use this break to consider whether we need to relax our eligibility criteria, add a fifth wave, add a second supermarket, increase compensation, or make other changes to the study design.

We have obtained ethical approval for our study from the University of Warwick’s Humanities & Social Sciences Research Ethics Committee (reference: HSSREC 123/23-24). We will obtain written informed consent from participants through the online survey, which will start with an informed-consent form, approved by the HSSREC.

Data Collection Procedures

Prior to the collection of data, participants will be informed about the study and will need to provide written consent to participate, see appendix B for the PIC forms. They will also be asked to answer survey questions on eligibility, demographics and other background information, see appendix C for questionnaires. Informed consent forms and surveys will be administered using Qualtrics. We will recruit potential participants via the Prolific (https://www.prolific.com/). Participants will be compensated for completing the online surveys at a rate of £12 per hour, and for keeping the browser plug-in enabled while they shop for groceries using the Chrome browser. Compensation for grocery shopping will be a fixed amount of £10 per month (plus any price discounts) for participants who shop at least twice in that month, as recorded by the plug-in. Participants will receive regular reminders via the Prolific platform to prompt them to complete their usual grocery shopping on Chrome with the plug-in enabled throughout the duration of the trial.

Screening/baseline survey: At the start of the intervention period, we will survey potentially eligible participants to collect background characteristics, including age, gender, ethnicity, household income, household size and composition, as well as attitudes towards food shopping and sustainability, see appendix C for the questionnaire. The plug-in will be distributed as part of the baseline survey using a customised download link for each participant that includes their participant ID. The plug-in will remember this ID number and pass it on with all purchase data that are generated, allowing us to match the purchase data to the information collected through the surveys. The personalised download links will allow only a single installation for each ID number, to avoid multiple households using the same participant ID. When there is an attempt to use a download link for the second time, the user will get a message to contact the researcher for a second link if appropriate.

Grocery purchases: The plug-in will collect data on grocery purchases at the point of checkout. These data are collected for initiated, updated, purchased (paid for), and cancelled checkouts. For each product, the plug-in will record a detailed description of the product and pack size, including a retailer-specific ID number, the purchase price, the quantity purchased, as well as information on the product including the environmental impact scores (eco-scores).

Endline survey: This survey will be administered at the end of the intervention period to gather quantitative and qualitative data for process evaluations, see appendix C for the survey questionnaire.

Not available

The randomisation is independent for each of the two interventions. We will randomly assign half of the sample to the eco-labels (intervention) group and the other half to the no eco-labels (control) group. Similarly, we will randomly assign participants to one of the three arms of the price discount trial so that roughly one third of participants will receive £1.00 price discounts on their swaps, one third will receive £0.50 discounts, and one third will not receive a price reduction on the alternative products that they are offered. This randomisation process is illustrated in Figure 5.

Individuals who are the main grocery shopper for their household

No

2760 individuals

2760 individuals

There are two independently randomised interventions (eco-labels and price discounts). We aim to allocate roughly 50% of the sample to the control and treatment groups for each interventions. Therefore, the trial will have 4 arms, each with approximately 690 participants.

The sample size calculation is based on an assumed minimum meaningful effect size. If on average participants swap 2 out of 34 products (the average basket size in the pilot study) for a more sustainable alternative, upgrading the eco-score for that product from D to B (-14.5 points) or from F to D (-35.0 points), then the expected effect size for the average basket score ranges from -0.85 points to -2.1 points. The standard deviation of the basket score across participants in our pilot study was 21 points. Therefore, to achieve a 5% significance level in a simple 2-arm trial, the required sample size ranges from 400 to 2760. We expect that we need to survey about 10,000 potentially eligible Prolific panellists to recruit 2760 participants for this study. This number is based on the results of the pilot study and expected increases in recruitment rates based on improvements in the study design (34% of screened panellists are eligible for the study, and we assume that 50% of those eligible will download the browser extension, and 80% of those who install the extension will complete at least 2 grocery shops during the study period). With an expectation that we will run 4 waves (but with time and resources available to run 5 waves if necessary), we will start by contacting 2,500 participants for the baseline survey in wave 1, with numbers recruited at further waves determined after our sample size, recruitment rate and attrition numbers are updated.