Nudges in IT Support Case Resolution: A Randomized Control Trial

Last registered on July 01, 2021

Pre-Trial

Trial Information

General Information

Title
Nudges in IT Support Case Resolution: A Randomized Control Trial
RCT ID
AEARCTR-0006896
Initial registration date
December 15, 2020
Last updated
July 01, 2021, 8:39 AM EDT

Locations

Region
Region
Region
Region
Region
Region
Region
Region
Region
Region
Region
Region
Region

Primary Investigator

Affiliation
Karlsruhe Institute of Technology

Other Primary Investigator(s)

PI Affiliation
SAP SE
PI Affiliation
Karlsruhe Institute of Technology

Additional Trial Information

Status
Completed
Start date
2021-02-09
End date
2021-06-18
Secondary IDs
Abstract
This project uses a randomized controlled trial to evaluate the efficacy of applying nudges in the context of ticket resolution within a multinational software corporation. Several tools have been introduced to automate key steps in the process of solving customer cases, however, many suffer from low adoption. The present study aims to investigate the potential for nudges (in the form of notifications about the availability of a diagnostic report) to increase the adoption of a specific diagnostic tool designed to process customer log files. We analyze both the immediate effects and persistence in behavior following the deactivation of the nudge. Approximately 900 employees will be randomly assigned to one of three groups, comprising one control and two treatment groups (N1 receives notifications for the entire 12-week duration of the study, N2 ceases to receive the nudge after 6 weeks of the study to assess long-term effects. A control group B receives no notifications).
External Link(s)

Registration Citation

Citation
Gorny, Paul M., Sean MacNiven and Petra Nieken. 2021. "Nudges in IT Support Case Resolution: A Randomized Control Trial." AEA RCT Registry. July 01. https://doi.org/10.1257/rct.6896-3.0
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details

Interventions

Intervention(s)
Our intervention encompasses three treatments: In the baseline B we record business-as-usual, in N1 we nudge employees with a message for the 12-week trial period. In N2 we discontinue the nudge of N1 after six weeks. The nudge is the information about the availability of a diagnostic report. This report can be accessed by all groups.
Intervention Start Date
2021-02-09
Intervention End Date
2021-05-04

Primary Outcomes

Primary Outcomes (end points)
Usage of diagnostic report (yes/no). Frequency of usage per ticket.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We have three treatment groups (baseline B, treatments N1 and N2 where in the latter the treatment manipulation is discontinued after 6 weeks). Employees will be randomly allocated to one of the groups in equal proportions. All employees working in the respective Service and Support Centers of the cooperating company and using the central support platform will be included in the study.
Experimental Design Details
Randomization Method
Randomization is donbe by the software in the company.
The log-in system is global and tracks both whether an employee has been assigned as well as the treatment the employee has been assigned to. The order is running through a repeating (1,2,3) only for those employees whose treatment assignment variable is yet equal to zero. Once employees have been assigned, the treatment assignment variable changes to one and their assigned treatment stays constant over the course of the intervention.
Randomization Unit
The randomization is done on the employee level and is based on log-in sequence.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
n.a.
Sample size: planned number of observations
900 employees
Sample size (or number of clusters) by treatment arms
We will allocate approximately 300 participants randomly to each treatment. Employees are located across multiple sites of the company, where all treatments are run in parallel across sites (no clustering).
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
With a current mean of Usage of diagnostic report by employee at 10%, we get a minimum detectable effect size of 0.07911 (alpha=.05 Power=.8) and 0..09331 (alpha=.05 Power=.9) respectively.
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number

Post-Trial

Post Trial Information

Study Withdrawal

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
June 18, 2021, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
June 18, 2021, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
3
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
967 employees
Final Sample Size (or Number of Clusters) by Treatment Arms
314 employees in N1, 327 employees in N2, 326 employees in N3
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials