Using Text Messages to Increase Hospital Attendance

Last registered on September 30, 2014

View Trial History

Pre-Trial

Trial Information

General Information

Title

Using Text Messages to Increase Hospital Attendance

RCT ID

AEARCTR-0000516

Initial registration date

September 30, 2014

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

September 30, 2014, 11:09 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country

Australia

Region

Sydney

Primary Investigator

Name

Michael Sanders

Affiliation

Behavioural Insights Team

Contact Primary Investigator

Other Primary Investigator(s)

PI Name

Alex Gyani

PI Affiliation

Behavioural Insights Team

Contact Investigator

Additional Trial Information

Status

In development

Start date

2014-10-02

End date

2015-03-31

Keywords

Health

Additional Keywords

JEL code(s)

Secondary IDs

Abstract

Systematic reviews of telephone and SMS reminders find that they significantly improve attendance to health care providers, with SMS reminders being as effective as phone call reminders and postal reminders (Car et al. 2012; Hasvold, 2011). More broadly, the evidence indicates that SMS messages can change behaviour when aimed at short-term behavioural outcomes (Fjeldsoe et al. 2009). However, the UK Behavioural Insights Team (BIT) has found that the content of the message is also very important.

Early trials have found that adding a person’s name to the a text message increases the likelihood that someone will pay a court fine (Behavioural Insights Team, 2012) and more recent trials have found that emphasizing the costs to the healthcare system can increase attendance rates (Hallsworth et al., in submission). Similarly, personable and personal messages increased attendance substantially (Kirkman et al., in submission). The trials run by the Behavioural Insights Team have also found that bland and impersonal text messages can have no impact on attendance whatsoever.

Text messages are currently used in some New South Wales (NSW) hospitals to encourage people to attend their outpatient appointments. However, these messages are not sent out as routine practice.

In this trial, a NSW hospital will trial the whether 8 text messages will increase the likelihood of people attending their outpatient appointments. Testing various text messages will allow us to further the science on persuasive messages and lead to cost savings for the hospital.

The content of these messages shall focus on: lost revenue to the hospital, lost revenue to other patients, revenue gains to the hospital, revenue gains to patients, aggregated losses to the hospital, a notice that people are free not to attend but they should cancel in advance or that their failure to turn up shall be recorded.

These messages were chosen as they are based on prior work, namely Prospect theory (Kahneman & Tversky, 1979), meta analyses into the use of the phrase "But you are free" (Carpenter, 2013) and some unpublished work by the Behavioural Insights Team. These phrases all have theoretical underpinnings, but this study shall test whether or not these theoretical underpinnings translate into an applied policy setting.

References:

Behavioural Insights Team (2012). Applying behavioural insights to reduce fraud, error and debt. Crown Copyright

Car, N. J., Christen, E. W., Hornbuckle, J. W., & Moore, G. A. (2012). Using a mobile phone Short Messaging Service (SMS) for irrigation scheduling in Australia–Farmers’ participation and utility evaluation. Computers and Electronics in Agriculture, 84, 132-143

Carpenter, C. J. (2013). A meta-analysis of the effectiveness of the “But you are free” compliance-gaining technique. Communication Studies, 64(1), 6-17.

Fjeldsoe, B. S., Marshall, A. L., & Miller, Y. D. (2009). Behavior change interventions delivered by mobile telephone short-message service. American journal of preventive medicine, 36(2), 165-173.

Hasvold, P. E., & Wootton, R. (2011). Use of telephone and SMS reminders to improve attendance at hospital appointments: a systematic review. Journal of telemedicine and telecare, 17(7), 358-364.

Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica: Journal of the Econometric Society, 263-291.

External Link(s)

Registration Citation

Citation

Gyani, Alex and Michael Sanders. 2014. "Using Text Messages to Increase Hospital Attendance." AEA RCT Registry. September 30. https://doi.org/10.1257/rct.516-1.0

Former Citation

Gyani, Alex and Michael Sanders. 2014. "Using Text Messages to Increase Hospital Attendance." AEA RCT Registry. September 30. https://www.socialscienceregistry.org/trials/516/history/2777

Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Experimental Details

Interventions

Intervention(s)

In this trial 8 messages shall be sent out through the standard appointment management system. The control message shall be the message that is currently used by the hospital.

The content of these messages shall focus on:

(1) Current template message
(2) lost revenue to the hospital,
(3) lost revenue to other patients,
(4) revenue gains to the hospital,
(5) revenue gains to patients,
(6) aggregated losses to the hospital,
(7) a notice that people are free not to attend but they should cancel in advance or
(8) that their failure to turn up shall be recorded.

Intervention Start Date

2014-10-02

Intervention End Date

2015-03-31

Primary Outcomes

Primary Outcomes (end points)

The outcome measure of interest will be whether or not a person turned up to their appointment. This will be measured using the hospital's appointment management system.

Primary Outcomes (explanation)

This measure is a routinely collected behavioural measure. It reflects revealed behaviour, rather than self reported behaviour.

Secondary Outcomes

Secondary Outcomes (end points)

Secondary Outcomes (explanation)

Experimental Design

The study shall take a pseudo randomised experiment design in which participants are sent a particular message based on a routinely assigned number. This 7 digit number has been allocated sequentially when the person first registered with the hospital. In order to overcome the issues of a number being allocated sequentially, the last 3 digits of the number shall be reversed and then used to allocate participants to a particular message. This message shall be sent the day before the person's appointment.

The same system shall then be used to record whether or not someone turned up to their appointment. This system is updated in real time.

Experimental Design Details

Randomization Method

The allocation shall be pseudo randomised. As the hospital system cannot be amended to include a random number generator it shall use a number that is currently assigned to people sequentially. This number shall be reversed and then people will be allocated to receive certain messages.

Prior to the start of the trial the hospital's previous 6 months of data (n=27488) were analysed to understand whether or not the messages that people would have received was associated with their likelihood to turn up to their appointment. A simple chi squared test indicated that there was no significant impact [x2(7)=10.3, p=.175]. Similarly, when 7 logistic regression models were run to test whether or not any groups would have performed better than the control group no significant differences were found.

The same tests shall be used to analyse whether or not the interventions led to a significant change in attendance rates after the trial has concluded.

Randomization Unit

Individual

Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters

There is no clustering

Sample size: planned number of observations

8000 appointment

Sample size (or number of clusters) by treatment arms

There are 8 messages being sent out:

Each message shall be sent out roughly 1000 times.

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

The minimum effect size of interest in this study is 3%. The current systems indicate that 14% of people fail to turn up to their appointments. G*power has been used to calculate minimum required effect sizes. To test whether or not the message received has had an impact on attendance, a chi squared text shall be used with 7 degrees of freedom. This will be able to find a very small effect (w>0.52). If this test is significant, 7 logistic regression shall be used. This method shall be used instead of a bonferroni correction, which is deemed too conservative. This shall test whether or not any of the message variants perform better than the control message. These tests will each include roughly 2000 observations (1000 from treatment variant n and 1000 from the control) and will be able to find an odds ratio of 1.24, based on analysis from g*power with alpha=0.05 and beta= 0.05. If beta is increased to 0.2, then the study will be powered to find an odds ratio of 1.17. These effect sizes are smaller than the minimum effect size of interest. References Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G* Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior research methods, 39(2), 175-191.

Supporting Documents and Materials

IRB