Improving Worker Productivity Through Tailored Performance Feedback: Field Experimental Evidence from Bus Drivers

Drivers are randomly assigned an experimental condition, stratified along the dimensions of base location, gender, and years of service at the company. We construct reference groups in which driver performance on each comfort dimension is compared to colleagues with the same base location and treatment status. The comfort dimensions are disaggregated measures of driving behavior over which drivers have a strong direct influence, thereby making the feedback as concrete and useful as possible to the recipients.
At the start of each month, the company shares with us a summary of each driver’s performance during the previous month. We use this information to assess how a driver performed compared to his/her peers and to assign peer-comparison messages. Dependent on treatment assignment, a number of negative (positive) messages are provided if a driver belongs to the bottom 50% (top 25%) of the reference group.
Treatment T1 [0n0p] is the control condition with no peer-comparison messages. In treatment T2 [1n0p], one negative message is provided if drivers underperform on a particular dimension. That is, they are explicitly informed that they rank poorly compared to peers and are encouraged to improve. In T3 [1n1p], drivers additionally have a chance of receiving one positive message. In this case, they are made aware of their good ranking and are encouraged to keep up the good work. If a driver performs poor (or well) on multiple dimensions, one will be randomly chosen. Finally, in T4 [3n0p], drivers run the risk of receiving corrective feedback on all comfort dimensions.

Using T3 [1n1p] as an example, the precise (translated) text of the messages reads as follows:
Dear colleague,
In terms of taking corners, you belong to the top 25 percent of the bus drivers in your location.
You are doing excellent on this dimension!
In terms of braking, you belong to the bottom 50 percent of the bus drivers in your location.
You can improve on this dimension!

A printed version of the report is delivered around the 15th day of each feedback month via the team manager or pigeonhole. Drivers in the control condition receive the same feedback report but without the targeted messages, so as to account for general feedback effects. The report contains general feedback in the form of a letter score, ranging from A (highest score) to D (lowest score) on the comfort dimensions and fuel economy. Furthermore, it contrasts the overall score of the individual driver with the score of his or her base location.

In parallel, the company initiated a coaching program. Six experienced drivers (one for each base location) were recruited as coaches based on their track record of driving behavior. All coaches participated in a training on how to approach drivers and how to communicate feedback. Since coaches are bus drivers themselves, there is only limited time available for coaching activities (about one day every two weeks). Furthermore, because of the hop-on hop-off approach to on-the-road coaching, a coach’s previous session determines the choice set for the next. This makes random allocation of coaching sessions at the driver-trip level impossible. At the same time, it is next to infeasible for coaches to target specific drivers, also because coaches have no access to the individual feedback reports and hence cannot target drivers with poor scores. We will provide empirical support for the view that the assignment of drivers to coaching is the outcome of a quasi-random process.
In a coaching session, a coach rides along with a bus driver for a portion of the driver’s shift. This allows the coach to personalize the feedback and to direct attention to the driver-specific issues at hand. A session is not announced to the driver beforehand. The
coach writes down examples of what goes well and wrong and identifies obstacles that may hinder driver performance, such as sharp corners. Due to the presence of passengers, there is no or limited interaction between the driver and the coach during the ride. The coach
provides feedback once the trip is completed and passengers have left the bus. The trip is reconstructed using the written-down examples. Both personal and general advice are offered that focus on fuel consumption, punctuality and the ABC dimensions. Drivers are treated as equals and feedback is delivered in a constructive and positive manner.

2015-11-09

2017-01-31

Driver scores on fuel economy (l/100km) and the comfort dimensions: Acceleration, Braking and Cornering (no. event/100km).

We run a field experiment at a large public transport company that is in the process of installing electronic on-board recorders (EOBRs) in its entire bus fleet. EOBRs enable the high-frequency measurement of a range of productivity outcomes, such as fuel efficiency and the number of Acceleration, Braking and Cornering events, the so-called ABC comfort dimensions.

Following the launch of the company’s EcoManager campaign to promote efficient and comfortable driving, all drivers receive a monthly written feedback report on their driving performance in the preceding month. This part of the campaign is not subjected to experimental variation: the launch date and timing of the monthly feedback are the same for all drivers. To this general report, we add a text box in which we experimentally vary the number of ABC dimensions on which drivers receive information on their relative ranking. This text box is empty for drivers in the control group. Drivers in the first treatment condition receive information on their poor relative performance (if any) on
only one of the ABC dimensions, even when performance is relatively poor on multiple dimensions. The second treatment condition is similar, except that negative feedback is supplemented with positive feedback in case a driver who performs poorly on some
dimensions scores well on others. In the final condition, all relative positions on driving behaviors are communicated whenever the driver performs poorly compared to a reference group of peers. Together, these interventions enable us to explore the potential of on-board monitoring technologies in customizing written relative performance feedback such that it enhances worker motivation.

In addition to the written peer-comparison feedback, we evaluate the effects of a parallel in-person coaching program with a quasi-experimental design. In this program, designated experienced drivers engage in coaching their colleagues by riding along with a
bus driver for a portion of the driver’s shift. At the end of the ride, the coach evaluates the trip in detail and gives tailored tips for improvement. Due to the hop-on hop-off approach to coaching and regulations that disallow coaches access to the driver’s perfor-
mance, the timing of the coaching sessions can be considered the outcome of quasi-random assignment: coaches select the drivers they will coach on a given day in a way that is unrelated to a driver’s past performance.

Randomization done in office by computer (code in STATA do-file)

Unit of randomization: Inidividual bus drivers.

No

409 bus drivers

409 bus drivers

103 drivers control [T1:0n0p]
102 drivers treatment 1 [T2:1n0p]
102 drivers treatment 2 [T3:1n1p]
102 drivers treatment 3 [T4:3n0p]

How should performance feedback be tailored to improve worker productivity? In a natural field experiment with 409 bus drivers and over 500,000 trip-level observations, we test the potential of two forms of individual feedback on improving worker productivity: written peer-comparison feedback and in-person coaching.

We find that a) the announcement of the written feedback program has a substantial and significant effect on fuel economy and outcomes pertaining to passenger comfort; b) targeted peer-comparison feedback is generally ineffective; c) in-person coaching generates significant improvements on all dimensions for drivers in the bottom half of the performance distribution. These effects last for about eight weeks; d) in-person coaching reduces the impact of written peer-comparison feedback but not vice versa.

https://drive.google.com/file/d/1dpVXLgspCYSS0YEF2-bUDk6AxT2qhLIh/view