Back to History Current Version

Evaluation of Interventions in Online Grocery Shopping for Sustainability and Health: An Adaptive Design Randomized Controlled Trial

Last registered on April 02, 2024

Pre-Trial

Trial Information

General Information

Title
Evaluation of Interventions in Online Grocery Shopping for Sustainability and Health: An Adaptive Design Randomized Controlled Trial
RCT ID
AEARCTR-0013247
Initial registration date
March 25, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
April 02, 2024, 10:52 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Affiliation
University of Warwick

Other Primary Investigator(s)

PI Affiliation
Queen Mary University London
PI Affiliation
University of Warwick

Additional Trial Information

Status
In development
Start date
2024-04-15
End date
2025-03-31
Secondary IDs
NYA
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Effective interventions are needed to promote more sustainable and healthier food choices for both human and planetary health. This study will evaluate two interventions in two independent adaptive design randomised controlled trials (one 2-arm and one 3-arm trial) within the same study population (a factorial design is not powered). The interventions are (i) eco-labelling, which will provide participants with information on the environmental impact of their food purchases using a score ranging from A (most sustainable) to G (least sustainable); and (ii) price discounts on alternative products with a better sustainability profile (and equal or better nutritional profile) in place of specific products in their basket. To implement these interventions, we will use a browser extension on online shopping websites for one or multiple large UK supermarkets, accessed using the Google Chrome browser on a desktop or laptop computer. We will assess the effect of these interventions on the average eco-score of the basket (primary outcome) as well as its nutritional content (secondary outcome). Health outcomes of the intervention will be modelled using nutrition scores.

What is already known?

- The food system is responsible for more than one third of total global greenhouse emissions, while also contributing to land degradation and biodiversity loss. It will be necessary to reduce emissions from the food system to achieve agreed climate targets including a maximum global temperature increase of 1.5⁰C.

- Poor diets are responsible for 1 in 7 deaths in the UK. More environmentally sustainable diets are likely to be better for human health as well.

- Effective interventions are urgently needed to promote more sustainable and healthier food choices for both human and planetary health.

- Online grocery shopping is on an upward trend and understanding behaviour in this setting is therefore increasingly important. As well as elucidating effective approaches to shift online shopping towards healthy and sustainable purchases, studies of interventions in the online shopping environment also provide an opportunity to examine the behaviour of many consumers, which would be more difficult and more costly in physical supermarkets.

- There are no existing randomised controlled trials examining sustainability outcomes in real (not simulated) online grocery stores. Previous work has focused on healthier choices. Most of these studies found that economic interventions, such as price discounts, show the most promise, and a few found swaps to be effective as well.

- There are no existing randomised controlled trials evaluating the effect of price interventions in real online grocery stores. The setting of all previous studies on price interventions was either a physical supermarket or a simulated online supermarket. The very limited number of studies on swaps (2 studies) and information interventions (3 studies) in real online grocery stores provide mixed evidence, indicating that these interventions may prompt healthier purchasing in some circumstances for some people.

- Studying actual shopping behaviour in real online grocery stores is important because consumers often behave differently in simulated environments. For example, it has previously been noted that nutrition labelling appeared 17 times more effective in laboratory studies than real grocery settings.

- We have shown in a pilot study that it is feasible to run a field trial on an actual online supermarket website using a bespoke browser extension and have learned valuable lessons on how to implement such a trial.

What does this study add?

- This will be the first study to provide causal evidence on interventions to promote sustainable food purchases from a large sample of consumers in a real online grocery shopping environment.

- We will add to the evidence base on how choice architecture influences purchasing behaviours and provide evidence on two interventions: (i) eco-labelling, which will provide participants with information on the environmental impact of their prospective food purchases using a score ranging from A (most sustainable) to G (least sustainable); and (ii) price discounts on alternative products with a better sustainability profile (and equal or better nutritional profile) offered in place of specific products in participants’ baskets.

- Our study results will also allow us to calculate consumers’ willingness to pay for more sustainable groceries.

- As secondary outcomes, we will look at whether either intervention affects total purchases. We will also examine the impact on the nutritional composition of purchases and (modelled) health outcomes.

- This study will provide evidence about the potential for eco-labels, or for food subsidies that could alter the prices of sustainable food relative to less sustainable food, to change consumer behaviour to meet health and sustainability goals.

How will we determine this?

- A large-scale cross-sectional randomised controlled trial with real consumers doing their usual shopping over a two-month period on the website of a large UK supermarket. The trial will be implemented using an adaptive design in up to five waves for cost reasons.

- Members of the public were involved in the design of this study and associated study materials.

- Participants will be recruited from the UK Prolific panel and will be compensated for their participation in this study. The Prolific panel is not representative of the UK population but provides good coverage of different socio-economic and demographic groups. We will only recruit participants who are the primary grocery shoppers for their household and who report buying groceries online at least once a month.

- Participants will be randomly allocated into different arms at the start of the trial. The two interventions will be independently randomised within the same study population so that there are effectively two independent trials. One of these will be a 2-arm trial (eco-labels on or off) and the other one a 3-arm trial (price discount of £1 or £0.50 or no discount). The study is not powered to detect any interaction between the two interventions being tested.

- To implement the interventions, we will use a browser extension (plug-in) that manipulates the supermarket website when participants access it through the Google Chrome browser on a desktop or laptop computer.

- The plug-in will collect data on product selections, substitutions and purchases. Additionally, survey participants will be surveyed for information on demographics and attitudes towards food shopping and sustainability.

- We will assess the effect of these interventions on the average eco-score of the basket (primary outcome) as well as its nutritional content (secondary outcome). Health outcomes of the intervention will be modelled using nutrition scores.

- A separate process evaluation will consider what worked, for whom, and in what circumstances.
External Link(s)

Registration Citation

Citation
Bentil, Helena, Oyinlola Oyebode and Thijs van Rens. 2024. "Evaluation of Interventions in Online Grocery Shopping for Sustainability and Health: An Adaptive Design Randomized Controlled Trial ." AEA RCT Registry. April 02. https://doi.org/10.1257/rct.13247-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
We will evaluate the following two interventions:

1. ​​​Eco-labelling: Eco-labels provide consumers with information on the environmental impact of their food purchases, graded from A (lowest impact, most sustainable) to G (highest impact, least sustainable) based on an underlying numerical score ranging from 0 (most sustainable) to 500 (least sustainable). These environmental impact scores were calculated based on the ingredients in 1kg of each product, which are then linked to the life-cycle analysis (LCA) database Agribalyse. The scores take into account the environmental impact of a product in 16 categories, including land use, water scarcity, resource use, human health, wildlife damage, and climate change. More details on the methodology are provided in appendix A.

2. Price discounts on more sustainable alternative products: Price discounts will be offered through swaps at the first checkout screen. Participants will see a pop-up window suggesting a more sustainable alternative for a particular product in their basket, which is offered at a randomly varying price discount. The alternative products are selected to not cost £2.00 more than the original product, and price discounts will be offered at three levels: £1.00, £0.50 or no discount. The intention is that this will result in discounts being roughly equally spaced between 0 and 100% of the price difference between the original and the alternative products (explicitly offering percentage discounts is not feasible for technical reasons, because it would involve “live” scraping of the prices). We will use this information to calculate the willingness to pay for more sustainable products. As an example, a participant may be prompted to swap Greek yoghurt (500g) priced at £2.30 for dairy-free coconut yoghurt (600g) priced at £3.60, and the pop-up will notify participants that they can buy the coconut yoghurt for £2.60, £3.10 or the full price of £3.60, depending on the trial arm that the participant was assigned to. If the shopper accepts the swap, they will be reimbursed for the price discount. Each participant will get offered a swap for a (potentially discounted) alternative product on up to 3 products in their shopping basket. The products for which a swap is offered are randomly chosen from a list, composed by the researchers, of not-so-sustainable products with suitable alternatives. To avoid offering alternative products with an inferior macronutrient profile, alternative products on the list will be selected so that they are in the same or a better category for fat, sugar and salt content on the traffic-light label.
Intervention Start Date
2024-04-15
Intervention End Date
2024-11-30

Primary Outcomes

Primary Outcomes (end points)
The average eco-score of the basket of groceries purchased with a lower score meaning more sustainable purchases.2 Using this outcome variable, we will estimate the effect size of introducing eco-labels (intervention 1) and the price-elasticity for sustainable alternative products (intervention 2), allowing us to calculate the willingness to pay for sustainable groceries. We aim to measure both short- and long-term responses to price discounts to see if trying a more sustainable alternative may shift demand persistently beyond a one-time purchase.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Basket eco-score for 16 impact categories of the Product Environmental Footprint (PEF): climate change, water use, resource use: minerals and metals, resource use: fossils, land use, ozone depletion, human toxicity: cancer, human toxicity: non-cancer, ionising radiation and human health, particulate matter, eutrophication: marine, acidification, eutrophication: terrestrial, eutrophication: freshwater, ecotoxicity: freshwater). This outcome will give some insight into what type of improvements can be expected from more sustainable grocery shopping because of our interventions. It is likely that the largest sustainability gains can be realised on the impact category climate change (greenhouse gas emissions).

The nutritional value of the basket of groceries purchased, as measured by the Food Standard Agency’s (FSA) Nutrient Profiling Model. We will use the changes in the nutritional composition of groceries purchased to model the health impact of our interventions if these were delivered at scale in the UK using the PRIMEtime model, a multi-state life table model, to calculate the impact of the interventions on diet-related diseases.

Total cost of the shopping basket: The total cost of the shopping basket per household expressed in £, will be used, to understand the impact on grocery costs.

Total number of products in the basket. This is primarily to check whether shoppers respond to eco-labels by buying more (less sustainable) products elsewhere (e.g. by adding them to their order on the mobile, or by buying them at a different supermarket).
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Setting, Recruitment and Study Design

This protocol has been informed by a pilot study, which was run in September and October 2023. Methods and results of the pilot are presented in Bentil et al. (2024).

Our setting is the online grocery shopping environment on the website of a large UK supermarket. We will use a browser extension (plug-in) to implement the interventions and collect purchase data. Underlying the plug-in is a database with information on the environmental impact of over 160 thousand products. The plug-in was developed by Sustained (https://sustained.com) and customised based on our specifications. It can be used to display eco-labels and/or offer price discounts on more sustainable alternative products, depending on the participant’s ID, which is randomly allocated to one of the arms of the trial. The plug-in also records information on products in the shopping basket, any swaps or clicks on the eco-labels, and products that are eventually purchased.

Participants will be recruited from the online research platform, Prolific (https://www.prolific.com/). To be eligible for the study, participants must:
- Be 18 years or older and located in the UK
- Be the primary grocery shopper of their household
- Frequently buy groceries online (at least once per month, self-reported)
- Usually shop at the supermarket(s) included in this study
- Usually use a laptop or desktop with the Google Chrome browser for online grocery shopping or be willing to do so for the duration of the study
- Consent to participate and be willing to download and install the plug-in and use it for the duration of the study
- Not have participated in the pilot study or previous wave(s) of the study.

These eligibility criteria are informed by the pilot study. We found that take-up is much lower among participants who were asked to shop at a different supermarket than where they usually shop, but participants who usually shop on a mobile device and therefore need to change their shopping behaviour to use the browser extension are no less likely to shop using the extension than those who usually use a desktop or laptop (Bentil et al., 2024).

The study will use a cross-sectional Randomised Controlled Trial (RCT) design. For participants in the intervention group, the intervention will be on from the start and there will not be a baseline period. The reason for this is that the results from our pilot study showed that inter-participant variation in the primary outcome was similar to intra-participant variation. Therefore, there is no efficiency gain from controlling for individual-specific fixed effects by comparing changes rather than levels of purchases. In a longitudinal design, some observations are lost, because some participants did not shop either in the baseline or in the intervention period, and our pilot results consistently showed a more precise estimation of the treatment effect in a cross-sectional design.

The trial will run over 8 weeks. If self-reported shopping frequency were accurate, our eligibility criteria would guarantee 2 shops per participant in a one-month trial. However, the pilot study showed that participants were shopping substantially less frequently than they said they would. On the other hand, we found that attrition did not increase over the one-month duration of the pilot, suggesting that a longer trial duration would be preferable. After the end of the trial period, participants are allowed and encouraged to keep using the plug-in and we will continue to collect their purchase data for use in future research. We will explicitly ask for consent for this post-intervention period of data collection at the start of the trial.

We will use an adaptive design for the RCT (Figure 3). An adaptive design, rather than the traditional fixed sample size RCT design, was chosen because uncertainty regarding recruitment rates and attrition makes determining the sample size challenging. The adaptive design will enable a reassessment of the sample size requirements and the potential to stop the trial early should interim analyses indicate success or futility, ensuring a more efficient allocation of resources and time ​(Pallmann et al., 2018)​.

The study will thus be implemented in phases or waves. We will approach Prolific panellists in four or five waves of 10,000 people, with the aim of recruiting around 700 participants in each wave, until we reach the sample size required for a 2-arm trial with a 2-month intervention period or until it becomes clear that it is futile to try and reach this sample size. As illustrated in Figure 4, we will use data from wave 1, anticipated to start in April 2024, to determine the number of people who shop during the trial (recruitment rate). Based on this information, we will recalculate the sample size requirements and continue with wave 2 only if it seems likely that after a maximum of five waves, we will successfully recruit a sufficiently large sample. This exercise is repeated after each wave. Between the second and third waves there is a two-months break to avoid the summer holidays, when shopping patterns may be atypical. We will use this break to consider whether we need to relax our eligibility criteria, add a fifth wave, add a second supermarket, increase compensation, or make other changes to the study design.

We have obtained ethical approval for our study from the University of Warwick’s Humanities & Social Sciences Research Ethics Committee (reference: HSSREC 123/23-24). We will obtain written informed consent from participants through the online survey, which will start with an informed-consent form, approved by the HSSREC.

Data Collection Procedures

Prior to the collection of data, participants will be informed about the study and will need to provide written consent to participate, see appendix B for the PIC forms. They will also be asked to answer survey questions on eligibility, demographics and other background information, see appendix C for questionnaires. Informed consent forms and surveys will be administered using Qualtrics. We will recruit potential participants via the Prolific (https://www.prolific.com/). Participants will be compensated for completing the online surveys at a rate of £12 per hour, and for keeping the browser plug-in enabled while they shop for groceries using the Chrome browser. Compensation for grocery shopping will be a fixed amount of £10 per month (plus any price discounts) for participants who shop at least twice in that month, as recorded by the plug-in. Participants will receive regular reminders via the Prolific platform to prompt them to complete their usual grocery shopping on Chrome with the plug-in enabled throughout the duration of the trial.

Screening/baseline survey: At the start of the intervention period, we will survey potentially eligible participants to collect background characteristics, including age, gender, ethnicity, household income, household size and composition, as well as attitudes towards food shopping and sustainability, see appendix C for the questionnaire. The plug-in will be distributed as part of the baseline survey using a customised download link for each participant that includes their participant ID. The plug-in will remember this ID number and pass it on with all purchase data that are generated, allowing us to match the purchase data to the information collected through the surveys. The personalised download links will allow only a single installation for each ID number, to avoid multiple households using the same participant ID. When there is an attempt to use a download link for the second time, the user will get a message to contact the researcher for a second link if appropriate.

Grocery purchases: The plug-in will collect data on grocery purchases at the point of checkout. These data are collected for initiated, updated, purchased (paid for), and cancelled checkouts. For each product, the plug-in will record a detailed description of the product and pack size, including a retailer-specific ID number, the purchase price, the quantity purchased, as well as information on the product including the environmental impact scores (eco-scores).

Endline survey: This survey will be administered at the end of the intervention period to gather quantitative and qualitative data for process evaluations, see appendix C for the survey questionnaire.
Experimental Design Details
Not available
Randomization Method
The randomisation is independent for each of the two interventions. We will randomly assign half of the sample to the eco-labels (intervention) group and the other half to the no eco-labels (control) group. Similarly, we will randomly assign participants to one of the three arms of the price discount trial so that roughly one third of participants will receive £1.00 price discounts on their swaps, one third will receive £0.50 discounts, and one third will not receive a price reduction on the alternative products that they are offered. This randomisation process is illustrated in Figure 5.
Randomization Unit
Individuals who are the main grocery shopper for their household
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
2760 individuals
Sample size: planned number of observations
2760 individuals
Sample size (or number of clusters) by treatment arms
There are two independently randomised interventions (eco-labels and price discounts). We aim to allocate roughly 50% of the sample to the control and treatment groups for each interventions. Therefore, the trial will have 4 arms, each with approximately 690 participants.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
The sample size calculation is based on an assumed minimum meaningful effect size. If on average participants swap 2 out of 34 products (the average basket size in the pilot study) for a more sustainable alternative, upgrading the eco-score for that product from D to B (-14.5 points) or from F to D (-35.0 points), then the expected effect size for the average basket score ranges from -0.85 points to -2.1 points. The standard deviation of the basket score across participants in our pilot study was 21 points. Therefore, to achieve a 5% significance level in a simple 2-arm trial, the required sample size ranges from 400 to 2760. We expect that we need to survey about 10,000 potentially eligible Prolific panellists to recruit 2760 participants for this study. This number is based on the results of the pilot study and expected increases in recruitment rates based on improvements in the study design (34% of screened panellists are eligible for the study, and we assume that 50% of those eligible will download the browser extension, and 80% of those who install the extension will complete at least 2 grocery shops during the study period). With an expectation that we will run 4 waves (but with time and resources available to run 5 waves if necessary), we will start by contacting 2,500 participants for the baseline survey in wave 1, with numbers recruited at further waves determined after our sample size, recruitment rate and attrition numbers are updated.
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
Humanities and Social Sciences Research Ethics Committee, University of Warwick
IRB Approval Date
2024-02-26
IRB Approval Number
HSSREC 123/23-24