x

We are happy to announce that all trial registrations will now be issued DOIs (digital object identifiers). For more information, see here.
On Rating Scales in Subjective Performance Evaluations - Performance Effects of Kindness
Last registered on October 18, 2019

Pre-Trial

Trial Information
General Information
Title
On Rating Scales in Subjective Performance Evaluations - Performance Effects of Kindness
RCT ID
AEARCTR-0004385
Initial registration date
October 18, 2019
Last updated
October 18, 2019 10:42 AM EDT
Location(s)

This section is unavailable to the public. Use the button below to request access to this information.

Request Information
Primary Investigator
Affiliation
University of Cologne
Other Primary Investigator(s)
PI Affiliation
University of Cologne
PI Affiliation
University of Cologne
PI Affiliation
University of Cologne
Additional Trial Information
Status
In development
Start date
2019-10-21
End date
2020-06-30
Secondary IDs
Abstract
To incentivize employee performance, many companies employ performance appraisals that are tied to compensation. In such ratings, employees are typically evaluated on a specific predetermined scale. It is often observed that the lowest rating categories are unused resulting in compressed appraisals. So far, research on subjective evaluations often associated this behavior with rating bias. In this project however, we study the question whether compressed appraisals may be designed by companies intentionally adding unused lower categories in order to frame rating scales to avoid negatively reciprocal reactions of employees.
External Link(s)
Registration Citation
Citation
Sliwka, Dirk et al. 2019. "On Rating Scales in Subjective Performance Evaluations - Performance Effects of Kindness." AEA RCT Registry. October 18. https://doi.org/10.1257/rct.4385-1.0.
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details
Interventions
Intervention(s)
Intervention Start Date
2019-10-21
Intervention End Date
2019-12-31
Primary Outcomes
Primary Outcomes (end points)
Points for counting the number “7” in blocks of randomly generated numbers on the individual level (individual performance).
Points for counting the number “7” in blocks of randomly generated numbers on the group level (group performance).
The number of times the time-out button was used, i.e. leisure time was taken.
Questionnaire data (post)
Primary Outcomes (explanation)
Points for counting the number “7”:
For each correct answer, a subject receives two points, and for each wrong answer, 0.5 points are substracted.
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
We conduct a laboratory experiment that involves the role of “employers” and “employees” resembling a standard work setting. In all treatments, subjects in the role of employees are randomly matched into groups with one subject in the role of employers. Employees work on a real effort task and receive a performance dependent bonus. Employers benefit from higher work effort as their payment (piece rate) depends on their employees’ performance. Employers determine the rating scale shown to employees when their performance is evaluated by the computer. The experiment consists of two stages. In the first stage, employers determine the rating scales shown to their groups of employees. Subsequently, employees learn the rating scale chosen by their employer and work on a real effort task afterwards.

We assess individuals’ ability in a pre-round, in which all subjects work on the same real-effort task. Subjects need to count the number “7” in blocks of randomly generated numbers (Berger et al. 2013). Based on this ability test, best performing subjects become employers. The remaining subjects become employees and are randomly assigned to an employer. Matching is anonymous and participants never receive information on the identity of other subjects. Using stratified sampling, we ensure equal ex ante performance across employee groups and equal number of employees per employer. To assure understanding of the experimental instructions, subjects must pass a comprehension quiz. Before the main task, subjects learn their role (employer or employee). The main task consists of two consecutive stages. In the first stage, employers determine the rating scales of the performance appraisal shown to their employees. Subsequently, employees learn the rating scale chosen by their employer. In the second stage, employees work on the real effort task of the pre-round for six rounds, each lasting 2.5 minutes. After the main task follows a questionnaire section.
Experimental Design Details
Not available
Randomization Method
Stratification method
Randomization Unit
Individual
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
none
Sample size: planned number of observations
480 subjects total sample size
Sample size (or number of clusters) by treatment arms
160 subjects in each treatment:
160 subjects NoDummy (control) treatment,
160 subjects Dummy treatment,
160 subjects DummyTransparent treatment
Note: due to possible no-shows of subjects the final number of subjects per treatment might be less
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
MDES= 2,5 [Points for counting the number “7”], standard deviation=6,99 [Points for counting the number “7”], 10%
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
Research Ethics Review, Faculty of Management, Economics and Social Sciences, University of Cologne
IRB Approval Date
2019-07-06
IRB Approval Number
19017TV