Competing Behavioral Mechanisms for Notification Timing in a Digital Health Experiment

The study evaluates a simple question: does the timing of a daily reminder shape how regularly people engage with a health-monitoring app? To answer this, we vary only when a single neutral reminder is delivered, holding the content and frequency constant.

After a brief quiet period with no notifications, participants are randomly assigned to one of five groups:

No-notification control, to benchmark natural usage.

Morning, afternoon, or evening fixed-time reminders, where the reminder arrives at the same time every day.

A variable-time group, where the reminder time shifts daily and is randomly drawn from the three fixed-time options.

Each reminder contains the same short, neutrally phrased message encouraging daily monitoring—nothing tailored or behaviorally charged. Every participant in the reminder conditions receives exactly one message per day for four weeks.

We then return to a period without notifications to see how engagement patterns persist once the reminders stop. This structure allows us to isolate whether consistency or variability in timing plays a meaningful role in sustaining healthy app habits.

The study is designed to isolate the causal effect of timing consistency in daily push notifications on sustained engagement with a health-monitoring app. The intervention proceeds in three phases, with all events passively logged throughout.

Phase 0: Pilot
Before the full experiment, we run a small pilot: 2 days of baseline observation, 4 days of timed notifications using the same structure planned for the main study, and 2 days of follow-up. Daily data dumps during this period allow us to identify any operational issues before scaling to the full sample.

Phase 1: Pre-experimental quiet period
All out-of-app push notifications are suspended for the entire user base. This establishes each user’s natural usage pattern free from recent notification exposure. The app continues to record all usage events.

Phase 2: Randomized timing intervention
At the end of the quiet period, users are randomly assigned at the individual level to one of five arms:

Control (No nudges) – Receives no notifications during the full 4-week window.

Fixed Morning – Receives one push notification daily at the same morning time (e.g., 10:00 AM IST).

Fixed Afternoon – Receives one notification daily at the same afternoon time (e.g., 2:00 PM IST).

Fixed Night – Receives one notification daily at the same evening time (e.g., 9:00 PM IST).

Variable Timing – Each night at midnight (or shortly before the morning send), the system draws a random assignment for the next day from the three fixed-timing slots. The user then receives the notification at whichever time slot is assigned for that day.

Across all treatment conditions (2–5), nudge load is strictly held constant: every treated user receives exactly 28 notifications, one per day, without exception. Message content is also held constant and is intentionally neutral—for example: “Take a reading to improve your daily control” or “Take a reading to spot spikes and keep your sugar in check.” These messages avoid any temporal cues (e.g., references to meals) to prevent confounding.

Phase 3: Post-experimental washout
After the 4-week intervention, all out-of-app notifications are again paused. This window allows us to observe whether behavior established during the experimental period persists once reminders stop. We continue monitoring engagement in real time and may extend the washout up to six weeks, though a minimum of two weeks is required.

This structure ensures that only the consistency of timing varies, not content, frequency, or total exposure. The design allows us to test whether predictable timing supports habit formation more effectively than unpredictable timing—an important question for digital health applications seeking to build durable engagement patterns.

2025-12-11

2026-01-08

The main outcome of interest is daily engagement with the health-monitoring app, measured through objective behavioral traces recorded by the app. Specifically, we focus on:

Probability of taking at least one reading on a given day.
This is the core behavioral metric used in the prior experiment (e.g., daily counts of readings shown in the charts on pp. 1–2 of the descriptives report) and provides a clear, interpretable measure of engagement

Number of readings taken per day.
This captures intensity of engagement, complementing the extensive-margin measure above. The earlier analysis similarly relied on daily reading counts to detect lift during interventions and decay afterward

Timing consistency of user engagement.
For users exposed to fixed-time versus variable-time nudges, we track whether readings cluster more tightly around the assigned nudge time, relative to baseline patterns.

All outcomes are observed continuously throughout the pre-intervention quiet period, the 4-week randomized timing intervention, and the post-intervention washout window. This structure allows us to test whether predictable versus unpredictable reminder timing differentially supports habitual, sustainable app use.

This study examines whether delivering a single daily reminder at a consistent time of day, versus a time that varies from day to day, influences how regularly people use a health-monitoring app.

After a short period with no notifications, users are randomly assigned to one of several groups. One group receives no reminders, while the others receive one neutral, non-personalized push notification each day, either at a fixed time (morning, afternoon, or evening) or at a time that changes daily. All users in reminder groups receive the same number of notifications over the intervention period, and the message content remains identical across groups.

The experiment runs for several weeks and includes observation windows before and after the reminder period. This structure allows us to measure baseline behavior, immediate responses to reminder timing, and whether any effects persist once reminders stop. Because the intervention varies only the timing of a simple daily reminder, the design cleanly isolates the role of timing consistency in shaping digital-health engagement.

Users are randomly assigned to experimental groups using a computerized, reproducible randomization procedure. The allocation is automated within the platform’s backend, ensuring that each eligible user has an equal chance of being placed into any of the study arms. No manual assignment is involved.

The randomization is conducted at the individual user level. Each user is independently assigned to one of the experimental arms. There are no group- or cluster-level assignments in this study.

Yes

The study randomizes at the individual user level. We therefore expect approximately 50,000 users to serve as individual clusters.

Because randomization occurs at the individual user level, the planned number of observations corresponds to the number of participating users. We anticipate approximately 50,000–150,000 users contributing daily engagement data throughout the study period.

Randomization occurs at the individual-user level. We plan to divide the sample evenly across the five arms:

Control (no notifications): ~20% of users

Fixed Morning Reminder: ~20% of users

Fixed Afternoon Reminder: ~20% of users

Fixed Evening Reminder: ~20% of users

Variable-Timing Reminder: ~20% of users

Exact counts will depend on the final pool of eligible active users at the start of the experiment, but the allocation will be balanced across the five groups.