Annoyance costs of reminders

Last registered on December 21, 2022

Pre-Trial

Trial Information

General Information

Title
Annoyance costs of reminders
RCT ID
AEARCTR-0009608
Initial registration date
July 06, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
July 08, 2022, 12:40 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
December 21, 2022, 6:58 AM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
University of Copenhagen

Other Primary Investigator(s)

PI Affiliation
Scripps Research Translational Institute

Additional Trial Information

Status
In development
Start date
2022-07-30
End date
2023-07-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
We study the impact of reminders on unsubscriptions from and engagement with The All of Us Research Program (AoURP). While reminders are generally seen as effective in encouraging individuals to complete the requested task, they may also impose an ‘annoyance cost’ on the user (Damgaard and Gravert, 2018 (D&G2018)). This in turn can increase the likelihood that users may unsubscribe from the communication or otherwise disengage from future communication efforts. In this study, we set out to replicate the finding by D&G2018 in a different setting and further analyze the mechanisms through which beliefs about future annoyance costs arise.
External Link(s)

Registration Citation

Citation
Gravert, Christina and Janna Ter Meer. 2022. "Annoyance costs of reminders." AEA RCT Registry. December 21. https://doi.org/10.1257/rct.9608-1.1
Experimental Details

Interventions

Intervention(s)
After an individual registers for the All of Us Research Program (AoURP), but does not complete consent, they are defaulted into an e-mail sequence of a total of 10 reminders to complete the consent. The first email comes 5 hours after registration and the remaining emails come every 5 days (e.g., after 5 days, after 10 days, after 15 days). We vary the information given in the first reminder concerning the cadence of the reminders they will receive to investigate the effect of beliefs on unsubscriptions.

Note that one of the platforms on which this intervention will be implemented, will feature a total of 3 reminders. We currently expect that the platform will update the experiment to the 10 reminders one month after data collection begins.
Intervention Start Date
2022-07-30
Intervention End Date
2023-02-28

Primary Outcomes

Primary Outcomes (end points)
The unsubscribe rate after receiving the first reminder (before the second reminder) and the cumulative unsubscribe rate over the series of reminders.
The primary consent rate after receiving the first reminder (before the second reminder)
and the cumulative consent rate for the full series of reminders.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
The average number of data donations after primary consent within a 30 day period of completing primary consent.
The open rate for each reminder email (as far as possible per platform)
The click rate for each reminder
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We will vary the information people will receive about the cadence of the reminders to consent to taking part in the research study they will receive. The experiment is a natural field experiment embedded in the AoURP reminder sequence.
Experimental Design Details
For more details, see the attached pre-analysis plan.
Randomization Method
Users will be randomized to experimental conditions when creating an account. This randomization will be set up by Scripps using RKStudio and by PTSC using Iterable.
Randomization Unit
The level of randomization are individuals.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
Based on platform traffic between March and May 2022 and outreach forecasts, we expect that 1,180 individuals become eligible for the evaluation each week across both CareEvolution (CE) and PTSC platforms (about ~5,100 per month). We have policy support to run the evaluation for three months. This leads to 15,300 individuals in total.
Sample size: planned number of observations
15,300
Sample size (or number of clusters) by treatment arms
5100 per arm.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
After three months of data collection, we are able to detect a 1.1-1.2 pp difference in the unsubscribe rate over a baseline of 5%.
IRB

Institutional Review Boards (IRBs)

IRB Name
All of Us Institutional Review Board
IRB Approval Date
2018-11-05
IRB Approval Number
2016-05
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials