Annoyance costs of reminders

Last registered on December 21, 2022

View Trial History

Pre-Trial

Trial Information

General Information

Title

Annoyance costs of reminders

RCT ID

AEARCTR-0009608

Initial registration date

July 06, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

July 08, 2022, 12:40 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated

December 21, 2022, 6:58 AM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Country

United States of America

Region

All

Primary Investigator

Name

Christina Gravert

Affiliation

University of Copenhagen

Contact Primary Investigator

Other Primary Investigator(s)

PI Name

Janna Ter Meer

PI Affiliation

Scripps Research Translational Institute

Contact Investigator

Additional Trial Information

Status

In development

Start date

2022-07-30

End date

2023-07-31

Keywords

Behavior, Health

Additional Keywords

JEL code(s)

M31, C93, I19

Secondary IDs

Prior work

This trial does not extend or rely on any prior RCTs.

Abstract

We study the impact of reminders on unsubscriptions from and engagement with The All of Us Research Program (AoURP). While reminders are generally seen as effective in encouraging individuals to complete the requested task, they may also impose an ‘annoyance cost’ on the user (Damgaard and Gravert, 2018 (D&G2018)). This in turn can increase the likelihood that users may unsubscribe from the communication or otherwise disengage from future communication efforts. In this study, we set out to replicate the finding by D&G2018 in a different setting and further analyze the mechanisms through which beliefs about future annoyance costs arise.

External Link(s)

Registration Citation

Citation

Gravert, Christina and Janna Ter Meer. 2022. "Annoyance costs of reminders." AEA RCT Registry. December 21. https://doi.org/10.1257/rct.9608-1.1

Sponsors & Partners

Experimental Details

Interventions

Intervention(s)

After an individual registers for the All of Us Research Program (AoURP), but does not complete consent, they are defaulted into an e-mail sequence of a total of 10 reminders to complete the consent. The first email comes 5 hours after registration and the remaining emails come every 5 days (e.g., after 5 days, after 10 days, after 15 days). We vary the information given in the first reminder concerning the cadence of the reminders they will receive to investigate the effect of beliefs on unsubscriptions.

Note that one of the platforms on which this intervention will be implemented, will feature a total of 3 reminders. We currently expect that the platform will update the experiment to the 10 reminders one month after data collection begins.

Intervention Start Date

2022-07-30

Intervention End Date

2023-02-28

Primary Outcomes

Primary Outcomes (end points)

The unsubscribe rate after receiving the first reminder (before the second reminder) and the cumulative unsubscribe rate over the series of reminders.
The primary consent rate after receiving the first reminder (before the second reminder)
and the cumulative consent rate for the full series of reminders.

Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)

The average number of data donations after primary consent within a 30 day period of completing primary consent.
The open rate for each reminder email (as far as possible per platform)
The click rate for each reminder

Secondary Outcomes (explanation)

Experimental Design

We will vary the information people will receive about the cadence of the reminders to consent to taking part in the research study they will receive. The experiment is a natural field experiment embedded in the AoURP reminder sequence.

Experimental Design Details

For more details, see the attached pre-analysis plan.

Randomization Method

Users will be randomized to experimental conditions when creating an account. This randomization will be set up by Scripps using RKStudio and by PTSC using Iterable.

Randomization Unit

The level of randomization are individuals.

Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters

Based on platform traffic between March and May 2022 and outreach forecasts, we expect that 1,180 individuals become eligible for the evaluation each week across both CareEvolution (CE) and PTSC platforms (about ~5,100 per month). We have policy support to run the evaluation for three months. This leads to 15,300 individuals in total.

Sample size: planned number of observations

15,300

Sample size (or number of clusters) by treatment arms

5100 per arm.

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

After three months of data collection, we are able to detect a 1.1-1.2 pp difference in the unsubscribe rate over a baseline of 5%.

Supporting Documents and Materials

IRB