Enhancing Organ Donor Registration Rates through Strengthening ServiceOntario Customer Representatives’ Motivations

Last registered on May 21, 2018

Pre-Trial

Trial Information

General Information

Title
Enhancing Organ Donor Registration Rates through Strengthening ServiceOntario Customer Representatives’ Motivations
RCT ID
AEARCTR-0001974
Initial registration date
May 24, 2017

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
May 24, 2017, 4:21 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
May 21, 2018, 11:25 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
University of Toronto

Other Primary Investigator(s)

PI Affiliation
University of Toronto
PI Affiliation
University of Toronto
PI Affiliation
University of Toronto
PI Affiliation
Johns Hopkins Univeraity

Additional Trial Information

Status
In development
Start date
2016-11-01
End date
2018-12-31
Secondary IDs
Abstract
In virtually every country, the need for organs for transplants far exceeds supply, leaving many patients to spend years waiting and even die before receiving a transplant. In the U.S. in 2009, for instance, where there are about 26 donors per million people, among candidates newly wait-listed for either a first or repeat kidney transplant the median wait time was 3.6 years (about one year for a liver transplant), and only slightly over 60% of individuals wait-listed ever received an organ. Approximately twenty individuals die each day because they cannot find a matching donor. In addition to the implications for transplant candidates, a kidney transplant, for example, also saves at least $200,000 over the life of the individual relative to on-going dialysis treatment.

Donation rates are even lower in Canada, with about 15 deceased donors per million people. In 2013, for example, there were 1,419 kidneys transplanted — 588 of which from living donors — but that left more than 3,000 Canadians on the waiting list for a new organ. Almost 42,000 Canadians were living with failing kidneys, creating an unprecedented demand for dialysis and transplants. During 2010, 229 patients died while waiting for organs. The end of that year saw 501 patients waiting for a liver, 135 for a heart, 310 for a lung and 98 for a pancreas.

There are many opportunities for individuals to express their intention to become an organ donor. In particular, when visiting public offices such as ServiceOntario for a variety of services (e.g., obtaining or renewing a driver’s license or health card), individuals have the opportunity to consent to be added to the organ donor registry when asked by a customer service representative (CSR). Similar procedures occur in the United States at the offices of the Department of Motor Vehicles (DMV). Yet, registration rates, although increased over the past few years, remain surprisingly low in Canada, at about 20-30%. These low donor registration rates are especially frustrating and surprising in light of the fact that an overwhelming majority of Canadians support organ donation – in Ontario in particular, 85% of citizens are in favor of organ donation.

With an aging population, advances in medical knowledge and technologies that make transplants an increasingly applicable option for many patients, and a growing ethnic heterogeneity that requires a more diverse composition of the organ supply, the imbalance between demand and supply is bound to increase, thus exacerbating individual, social and economic costs that could be avoided if more people behaved consistently with their beliefs and donated their organs.

The objective of this research is to apply concepts and methods from social and behavioural science to understand how registration rates can be increased, thus providing a larger base of potential donors to address the organ supply shortage and improve health and living prospects of thousands of Canadians and their families.

Previously, ServiceOntario and the Trillium Gift of Life Network partnership with academics explored whether low registration rates may be due in part to the length and complexity of registration forms or the timing when those forms are handed out. Moreover, these partners also tested interventions leveraging emotional and perspective-taking appeals, and discovered that these minor changes to the organ donor registration forms can significantly increase the rate with which Ontarians sign-up as organ donors.

(see in particular:
Nicole Robitaille, Nina Mazar, and Claire I. Tsai (2015) ,"Nudging to Increase Organ and Tissue Donor Registrations", in NA - Advances in Consumer Research Volume 43, eds. Kristin Diehl and Carolyn Yoon, Duluth, MN : Association for Consumer Research, Pages: 176-180.
http://www.acrwebsite.org/volumes/1019975/volumes/v43/NA-43)

In this proposal, we take a new but complementary perspective, focused on the customer service representatives who help Ontarians register using the forms. We ask whether and how the performance of customer service representatives at ServiceOntario offices, who are instructed to ask each customer whether they would like to register their consent to become organ donors (a “prompted choice” approach), can be improved.

Our interventions will consist of providing information to each CSR about his or her organ donor registration rate, with and without a comparison with that of the rest of ServiceOntario CSRs' conversion rates, using historical data. The idea would be that learning that, for example, the majority of other CSRs more effectively register organ donations might encourage a CSR to be more consistent in asking customers about their willingness to register and thereby improve their personal effectiveness. The information provided will not affect any actual evaluation of the CSRs; their relative effectiveness in terms of “conversion rates” of customers into donors will not be used for any formal performance evaluation, promotion or salary determination. Also, CSRs’ relative effectiveness is disclosed only to them individually but not to their peers.

The proposed interventions are based on the hypothesis that simply providing information about relative effectiveness, even when it has no impact on economic outcomes for an individual (i.e., compensation or promotion), will still have an effect by appealing to other, non-monetary forms of motivation (e.g., one’s self-image). If effective, this and similar interventions may represent simple, cost-effective ways to motivate CSRs, and are likely to be applicable to other contexts. Performance benchmark could also be demotivating, however. For example, if an agent is far below average, having this information may reduce incentives instead of stimulating effort. Or, being far to the right of the distribution (being a high performer) may, too, lead to mitigate effort.

Scholars performed similar behavioral investigations to study, for example, whether the (over)prescribing behavior or physicians is affected by providing aggregate statistics of the behavior of peers (http://jamanetwork.com/journals/jama/fullarticle/2488307); whether information about one student's as well as other students' absence rate helps reducing excess absenteeism at school (http://scholar.harvard.edu/files/todd_rogers/files/influential_third_parties.pdf); and whether information on the energy consumption of others affect one's individual consumption (http://www.sciencedirect.com/science/article/pii/S0047272711000478; http://www.econ.ucla.edu/costa/nudge23withtablessubmitted.pdf).

External Link(s)

Registration Citation

Citation
Lacetera, Nicola et al. 2018. "Enhancing Organ Donor Registration Rates through Strengthening ServiceOntario Customer Representatives’ Motivations." AEA RCT Registry. May 21. https://doi.org/10.1257/rct.1974-5.0
Former Citation
Lacetera, Nicola et al. 2018. "Enhancing Organ Donor Registration Rates through Strengthening ServiceOntario Customer Representatives’ Motivations." AEA RCT Registry. May 21. https://www.socialscienceregistry.org/trials/1974/history/29775
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
Details of the interventions are reported in the description of the experimental design below.
Intervention Start Date
2017-06-19
Intervention End Date
2018-06-30

Primary Outcomes

Primary Outcomes (end points)
The main outcome of interest will be the in individual organ donor registration rates before and after the intervention. The registration rates of each CSRs will be the ratio: number of new individuals added to the organ donor registry while served by the CSR / total number of clients served for transactions for which a CSRs is expected to ask about the desire to join the registry. We will calculate these values, before and after the intervention, for periods up to six months before and six months after, and of course for the period between the first and the second intervention wave..

ServiceOntario records some of the activities of a CSR (those related to health services such as the issue or renewal of a health card) at the transaction levels -- one different entry for each customer served. For other activities (those that are not health related, such as issuing or renewing a driver's license), recording is in a separate data system and at the day level, i.e. how many transactions, and of what type, a CSR did in a given date. To have reliable measures of the overall activity of a CSR, per day and over longer periods of time, we had to combine for the pre-intervention information, and we will have to combine in order to calculate the post-intervention performance, the two datasets. However, because the recording is at a different unit of analysis in each of the datasets, we need to perform some adjustments which, in turn, require making some assumptions, as described below.

For the transaction-level (health services) data, for each day we sum the number of entries (rows) in the dataset to obtain the number of clients served who needed any kind of health related service, including organ donor registrations. The organ donor registrations are reported in the transaction-level data because this is considered a health-related transaction.

Because, from the transaction-level (health-related) data, there are about 1.3 services (excluding organ donor registrations) per client, when aggregating the different interactions to determine how many customers a CSR served we first divide the entries from the dataset at the day level (for non-health related services) by 1.3; thus if a CSR in a given day provided 13 non-health related services, the assumption is that she served 10 clients who needed non health related services. We then sum this amount to the total number of customers served for health related products. In the day-service level data, one of the entries for the various services offered is for "joint" health and non-health services offered. These would be services to clients who needed assistance for both health and non-health transactions (e.g. renewing both a driver's license and a health card). To avoid duplication when summing to the health services reported in the transaction dataset, we do not include the "joint" transaction amounts to the total of non-health based transactions. In this way, we are more confident that we are considering non-health services offered to customers who did not also receive health-related services in the same transaction.

Each CSR who ever registered one client will therefore appear in the transaction level data, but of course she may also appear in the day-services data if she provided also non-health services. Again, summing the daily health services (excluding organ donor registration) and daily non-health transactions divided by 1.3 (excluding the joint health / non-health transactions) gives us a plausible proxy of how many "unique" or different clients a CSR served in a given day. There may still be some noise and potentially overstating the number of different clients, but we do not expect any bias to be systematically related to certain CSRs or locations.

Note also that there are multiple reasons for a "failed" donor registration (which we code with value of zero in the columns that reports organ donor registrations at the transaction or client level): the client may have declined the offer, or the client may be already registered. Therefore a CSR does not actually have an opportunity to sign someone up at every transaction, if someone has already signed up to the registry. Again, we expect this bias to not be systematic. A further reason for a failed registration, and an important one for the purposes of this study, is if the CSR did not even ask the client.

We also keep track of the overall intensity/amount of activity of each CSR other than signing up new organ donors.
Primary Outcomes (explanation)
Please see above for some details on variable construction.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The intervention will involve 79 sites directly managed by Service Ontario (it excludes private ServiceOntario centers).
We will randomly assign each site to one of three conditions. Each site will be in the same condition in the two times in which it will be treated: June 19 2017, and one day in November (or early December) 2017 to be defined. These two days in June and November/December 2017 are the therefore intervention periods. The trial, including the collection of pre-intervention performance data and post intervention performance data, goes from November 2016 to June 2018.

Here is a description of the three conditions.

Group 1 – CSRs in Group 1 offices will receive general information about organ donation, as well as some tips for asking customers to sign up.

Group 2 – CSRs in Group 2 offices will receive general information about organ donation, some tips for asking customers to sign up, as well as quantitative information about their individual registration rates, in numeric as well as graphical form (bar chart). The registration rates that we will report is the average over six months (November 2016-April 2017 for the first round, and May 2017-October 2017 for the second round). We had to limit the historical performance data to the end of April because these were the most updated data that were available when defining the intervention.

Group 3 – CSRs in Group 3 offices will receive general information about organ donation, some tips for asking customers to sign up, as well as quantitative information about their individual registration rates, and the average and 80th percentile registration rates in the region to which the center belongs (there are four regions: Central, Western, Eastern and Northern Ontario). This information, again, will be both in numeric and graphical form (bar chart -- see below). The regional distributions (from which the mean and 80th are derived) are therefore distributions of individual average registration rates over the period of interest.
The individual registration rate that we will report is the average over six months (November 2016-April 2017 for the first round, and May 2017-October 2017 for the second round). We had to limit the historical performance data to the end of April because these were the most updated data that were available when defining the intervention.

In group 2 and 3, the communication will include a note specifying that the registration rate information is exclusively for informational purposes and will not affect any formal evaluation.

CSRs will be assigned to the condition of the office where they are working as of the date of the first intervention. Some CSRs worked at more than one location between November 2016 and the time of the intervention; their personal registration rate that they will see if assigned to Group 2 or 3 is the average over the full six month period from November 1 2016 to April 30 2017 (included) -- we sum all of their registrations and divide them by the total of the customers they served.

To calculate the average and 80th registration rate for a region, however, we will calculate, for CSRs who worked in multiple regions (a minority of the total), separate average registration rate limited to the interactions in a given region. So for example, if in the period Nov 1 2016-Apr 30 2017 a CSR called Y served 200 customers in region A and had 10 registrations, and served 100 customer in region B with 8 registrations (and she is currently working in site X of region B), she will:
1) Be assigned to the treatment condition randomly determined for site X in region B
2) Be informed, if in group 2 or 3, that her registration rate between Nov 1 2016 and Apr 30 2017 was (10+8)/(100+200)=18/300=6%.

In determining the average and 80th signup rate for region A and B to show to CSRs in sites assigned to Group 3 in those region, however, we include a registration rate of 10/200=5% by CSR Y in region A, and a registration rate of 8/100=8% by CSR Y in region B.
Because CSR Y currently worked in an office of region B, if her current office is in Group 3 she will be shown her overall registration rate of 6%, and will be shown the average and 80th registration rate for region B. The reason for these calculation choices is that we want to give a CSR a sense of her own overall performance, but we want to give, as reference in group 3, statistics for the current region, because of potential regional differences. This is also the reason for separating, in calculating overall distributions, the performances of one given CSRs across regions.
Moreover, 3 of the 79 offices were administratively relocated to a different region in early April 2017. Because for the most part of the period for which we collected CSR performance data they were assigned to a region different from the current one, we kept them in the "old" region for the purposes of assigning reference stats (for Group 3) and for the stratified randomization (see below).

The communications just described will occur via email, sent at the dates specified above. An email from the Assistant Deputy Minister-Customer Care at ServiceOntario will inform about further communication to follow about organ donation registrations.

The emails for group 2 and 3 will include a brief explanation that the individual registration rates include the full activity of a CSR between Nov 1 2016 and Apr 30 2017, and will clarify (in Group 3) from what region the distributional info on registration rates comes.

Information on signup rate performance will be in numerical and graphical form.
In Group 2, the graph will be a horizontal bar with the signup rate reported next to it. The horizontal bars will be of the same length for all CSRs, regardless of the actual signup rate of a given CSR. The reason for this choice is to focus on the "absolute" value of the rate rather than the relative one.

In group 3, there will be three bars: the actual performance, and the average and 80th signup rate (or, better, average individual signup rates) for the relevant region. The scale of the bars will be such that the longest of the three bars (i.e., either the 80th percentile bar or the bar representing the individual performance) will be of a defined length, the same for all CSRs, regardless of their actual absolute value. The other two bars will be scaled accordingly. Doing so maximizes the relative difference between bars, especially for low performers.

In order to protect the privacy of the CSRs and to not exert undue influence, we will not inform the CSRs of the intervention under way. The researchers will not have access to the identity of the CSRs or other identifying information. We will elaborate a procedure such that it will be possible to assign CSRs to different treatments using numerical IDs, and to use these IDs to follow CSRs over time.

UPDATES AS OF JAN 22 2018: a few administrative delays led to running the second round of the trial in late January. Moreover, Service Ontario approved running a third round, organized as the other two, in june 2018. We will then extend data collection until the end of 2018.
Experimental Design Details
Randomization Method
Randomization will come from a computer-based random number generator.
Randomization Unit
Randomization will be at the office or centre level -- all CSRs in one office will receive the same treatment condition. As specified above, for the first intervention wave on June 19 the office for each CSR will be the one at which they are working at that same.

An agreement with ServiceOntario is that we would send an email also to those CSRs who joined after April 30th 2017, for whom we do not have any performance information. These few CSRs, regardless of the office where they are on June 19 2017, will be assigned to Group 1, in June 2017 and again in November/December 2017.

For CSRs who will join after the first intervention date, we will be able to collect performance data but they would not have been treated on June 19th. They will only be part (if still working at ServiceOntario) of the second intervention wave, and will be assigned to the condition of the office where they will be working at the date of the second intervention. At the office level and with the exception of CSRs who joined between April 30 and June 19th, the experimental condition will be the same in both intervention waves.

UPDATES AS OF JAN 22 2018: a few administrative delays led to running the second round of the trial in late January. Moreover, Service Ontario approved running a third round, organized as the other two, in june 2018. We will then extend data collection until the end of 2018.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
79 centres, approximately 560 CSRs. We expect some variation in the number due to some turnover in the offices. Please see above (section on Randomization Unit) for a description of how we will treat CSRs joining ServiceOntario in different periods.
Sample size: planned number of observations
We will treat and follow approximately 560 CSRs. We expect some variation in the number due to some turnover in the offices. Information on donor registration will be at the center-CSR-day level, and we will then aggregate to compute averages.
Sample size (or number of clusters) by treatment arms
Each treatment arm as the same number of observations in expectation. Because the number of centers is not divisible by 3, there will be a slight imbalance. Within each region, we will randomly assign M/3 centers to each conditions, where M is the greatest number that is less then or N, i.e. the number of centers in a region. Each of the remaining N-M sites, if any (one or two) will be randomly assigned, with equal probabilities, to one of the three conditions.
Regarding the three offices that were assigned to a different region in early April 2017, we will assign them to the region as of before early April, because this is where they were assigned during most of the period for which we use pre-intervention performance data
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Homewood Institutional Review Board --Johns Hopkins University
IRB Approval Date
2017-05-01
IRB Approval Number
HIRB00005769
IRB Name
University of Toronto Social Sciences, Humanities, and Education Research Ethics Board
IRB Approval Date
2016-03-21
IRB Approval Number
32650

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials