Ranking effects in recipe choices : a randomized controlled trial in a food app

Last registered on February 09, 2024

Pre-Trial

Trial Information

General Information

Title
Ranking effects in recipe choices : a randomized controlled trial in a food app
RCT ID
AEARCTR-0012705
Initial registration date
January 31, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
February 02, 2024, 4:08 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
February 09, 2024, 11:41 AM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
Univ Rennes and CREM-CNRS, UMR 6211

Other Primary Investigator(s)

PI Affiliation
Rennes School of Business
PI Affiliation
Rennes School of Business
PI Affiliation
Rennes School of Business

Additional Trial Information

Status
In development
Start date
2024-01-01
End date
2024-02-29
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Food consumption has two main impacts: (1) on the human health through nutrition, and (2) on the whole environment through the life cycle of products. From a policy perspective, favoring the consumption of healthy and environmental-friendly products is both a major public health issue and one important lever to address climate change. However, consumers only have limited information per se about the true qualities of a given food product, which may hamper their ability to decide on what they can safely eat or stop eating. A primary way of informing consumers about their food purchases is the use of labels, and especially 5-colors labels (from A to E) whose effectiveness has been scientifically documented. In Europe, and particularly in France, the NutriScore has been adopted as the main tool to inform consumers about the nutritional quality of products or meals. As for the environmental impact of goods, the Ecoscore is emerging as a potentially effective candidate.
There exists a large literature on the impact of nutritional labels, and in particular the NutriScore, on consumers' food quality intake as well as a growing literature focusing on the effects of environmental labelling (such as the EcoScore). However, studying the combination of both types of labels is relatively new. At the same time, that labels may enable to achieve the reduction in consumption of both unhealthy and environmentally-damaging products is debatable. Moreover, product information is only one dimension of the food decision, which is also heavily dependant on the broad choice architecture, and particularly on the way products are presented to consumers. In physical food purchases (online shop, supermarket, recipe book, etc.), products are organised by categories, price families, or other rankings that are decided by sellers. In web or mobile apps, which are increasingly used in daily food decisions, products are often ranked by popularity or by grades provided by past customers (e.g: restaurants). Selective ranking of products, which is a type of “nudge”, has indeed been documented as a powerful lever for behavioral change. If it was be possible to rank the products, meals or recipes by their NutriScore or their EcoScore (from A to E, from green to red), what would be the impact on the nutritional and environmental quality of food choices ?
To answer this question, we build a field experiment based on naturally-occuring food decisions using a mobile phone application which offers users the possibility to search for recipes that fit their needs and habits. We partner with the private firm that owns the application to exogenously manipulate the ranking of proposed recipes given users' inputs. Through randomization into treatments and observation of all users in a pre-intervention phase (with ranking by recipes' grades as baseline), we are able to assess the ceteris paribus effect of ranking recipes based on either the NutriScore or the EcoScore on both the nutritional and environmental quality of selected recipes.
External Link(s)

Registration Citation

Citation
Ackermann, Claire-Lise et al. 2024. "Ranking effects in recipe choices : a randomized controlled trial in a food app." AEA RCT Registry. February 09. https://doi.org/10.1257/rct.12705-1.1
Experimental Details

Interventions

Intervention(s)
The experiment consists in modifying an existing food application that is maintained by a private-sector firm with which we partner. The application is explicitly designed to reduce food waste and offers its users access to a search engine of recipes that is tailored to the user’s food habits. First, users build a profile based on their food habits (diet, daily food supplies) and, second, they can search for recipes that contains an ingredient that they want to consume. The search engine then provides users with a list of recipes that fits the user’s profile. Upon clicking on a recipe, a user can access the details of the recipe (including, e.g., the ingredients, the steps of the recipe, estimations of the cooking time and budget, etc.). In the baseline version of the application, the recipes are ranked according to the grades given by users to the recipes. In addition to the grades, the search engine also displays the NutriScore and the EcoScore of recipes. With the agreement and support of our private partner, our intervention consists in modifying the ranking of recipes based on such scores: the first (resp. second) treatment ranks recipes based on the underlying values of the NutriScore (resp. EcoScore) from A to E. Our intervention is organized in two steps: first, a purely observational phase where the application is not modified, and, second, an intervention phase where the treatments are implemented (alongside a control condition).
Intervention Start Date
2024-02-01
Intervention End Date
2024-02-29

Primary Outcomes

Primary Outcomes (end points)
We have two primary outcomes which we construct as follows. First, we define a “selected” recipe as a recipe that is accessed by a user for more than 30 seconds. Second, for each “selected” recipe we extract the numerical values of both the NutriScore and EcoScore. Third, we compute the average value of each score per user per phase. Ultimately, we obtain two primary outcomes: the individual averages of both the NutriScore and the EcoScore.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
We also have two secondary outcomes we intend to use to run robustness checks on our main analyses: the individual averages of the ratio between the value of a score (NutriScore or EcoScore) of the selected recipe and the average score of the recipes that have been “viewed” by the user. “Viewed recipes” are defined as all the recipes above the lowest point in the list from the search engine that has been reached by the user.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The experiment is a Randomized Controlled Trial within a food application: all users of the food application will be followed over time and will be (uniform-)randomly assigned to one of three groups. One group serves as control and is not assigned to a treatment: the ranking of recipes in the search engine will still rely on recipes’ grades. The two remaining groups are assigned to a treatment that modifies the ranking of recipes: it will be based on the underlying value of either the NutriScore or the EcoScore. All rankings will appear in descending order: the recipe with the highest grade/value of NutriScore/value of EcoScore is always ranked first. Before the intervention phase where the treatments are implemented, all three groups will be observed in the no-modification environment (observational phase). Both phases last for a month: the observational phase started on 01/01/2024 and the intervention phase will begin on 01/02/2024. During the whole process the experimenters will not have access to the data nor to the application: it is only after all the data is collected that our private partner will give us access to the data (which is regularly saved on the partner’s servers). After the intervention, the application will resume to its pre-intervention status (i.e. recipes ranked by grades). Notably, upon registering a profile on the application, all users must agree on the principle that they may participate in scientific studies.
Experimental Design Details
Randomization Method
Computer-based randomization issued by the private-sector partner.
Randomization Unit
Individual.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
Based on observations from past years, we expect from approximately 900 (2022) to 1200 (2023) active users in the application in both January and February 2024. Given that these numbers are compatible with our power analysis, and because the actual total number of users cannot be decided in advance, we decided to run the experiment on the the whole database rather than to randomize a power-compatible pre-defined number of users into our experiment.
Sample size: planned number of observations
Since the design is not clustered, the number of observations is identical to the number of clusters: from 900 to 1200.
Sample size (or number of clusters) by treatment arms
About 300 to 400 for each treatment.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials