Nudges in IT Support Case Resolution: A Randomized Control Trial

This project uses a randomized controlled trial to evaluate the efficacy of applying nudges in the context of ticket resolution within a multinational software corporation. Several tools have been introduced to automate key steps in the process of solving customer cases, however, many suffer from low adoption. The present study aims to investigate the potential for nudges (in the form of notifications about the availability of a diagnostic report) to increase the adoption of a specific diagnostic tool designed to process customer log files. We analyze both the immediate effects and persistence in behavior following the deactivation of the nudge. Approximately 900 employees will be randomly assigned to one of three groups, comprising one control and two treatment groups (N1 receives notifications for the entire 12-week duration of the study, N2 ceases to receive the nudge after 6 weeks of the study to assess long-term effects. A control group B receives no notifications).

External Link(s)

Registration Citation

Citation

Gorny, Paul M., Sean MacNiven and Petra Nieken. 2021. "Nudges in IT Support Case Resolution: A Randomized Control Trial." AEA RCT Registry. February 11. https://doi.org/10.1257/rct.6896-2.0

Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Experimental Details

Interventions

Intervention(s)

Our intervention encompasses three treatments: In the baseline B we record business-as-usual, in N1 we nudge employees with a message for the 12-week trial period. In N2 we discontinue the nudge of N1 after six weeks. The nudge is the information about the availability of a diagnostic report. This report can be accessed by all groups.

Intervention Start Date

2021-02-09

Intervention End Date

2021-05-04

Primary Outcomes

Primary Outcomes (end points)

Usage of diagnostic report (yes/no). Frequency of usage per ticket.

Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)

Secondary Outcomes (explanation)

Experimental Design

We have three treatment groups (baseline B, treatments N1 and N2 where in the latter the treatment manipulation is discontinued after 6 weeks). Employees will be randomly allocated to one of the groups in equal proportions. All employees working in the respective Service and Support Centers of the cooperating company and using the central support platform will be included in the study.

Experimental Design Details

Randomization Method

Randomization is donbe by the software in the company.
The log-in system is global and tracks both whether an employee has been assigned as well as the treatment the employee has been assigned to. The order is running through a repeating (1,2,3) only for those employees whose treatment assignment variable is yet equal to zero. Once employees have been assigned, the treatment assignment variable changes to one and their assigned treatment stays constant over the course of the intervention.

Randomization Unit

The randomization is done on the employee level and is based on log-in sequence.

Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters

n.a.

Sample size: planned number of observations

900 employees

Sample size (or number of clusters) by treatment arms

We will allocate approximately 300 participants randomly to each treatment. Employees are located across multiple sites of the company, where all treatments are run in parallel across sites (no clustering).

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

With a current mean of Usage of diagnostic report by employee at 10%, we get a minimum detectable effect size of 0.07911 (alpha=.05 Power=.8) and 0..09331 (alpha=.05 Power=.9) respectively.

Supporting Documents and Materials

IRB