Spoken or written: the differences in evaluation formats in a picture-naming task

Last registered on August 18, 2023

Pre-Trial

Trial Information

General Information

Title
Spoken or written: the differences in evaluation formats in a picture-naming task
RCT ID
AEARCTR-0009413
Initial registration date
May 18, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
May 18, 2022, 5:16 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
August 18, 2023, 6:55 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Affiliation
Paderborn University

Other Primary Investigator(s)

Additional Trial Information

Status
In development
Start date
2022-05-30
End date
2024-06-30
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Previous research has shown that employee evaluations tend to be biased. When evaluating job performance, appraisers tend to judge inappropriately between employees. In our experiment, we let subjects perform a group task that is observed and evaluated by two appraisers. When the task is completed, one appraiser records her evaluation verbally while the other appraiser records her evaluation in writing. Therefore, we investigate whether there is a difference between written and oral evaluations and which format reflects subject performance most accurately.
External Link(s)

Registration Citation

Citation
Mehic, Miro . 2023. "Spoken or written: the differences in evaluation formats in a picture-naming task." AEA RCT Registry. August 18. https://doi.org/10.1257/rct.9413-1.2
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2022-05-30
Intervention End Date
2024-06-30

Primary Outcomes

Primary Outcomes (end points)
The key outcome variable is subject performance.
Primary Outcomes (explanation)
Subject performance is measured by completion time and correct order of the pictures. Later, this objective performance is compared to the subjective performance evaluations of the appraisers.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
First, we measure the participants' individual competences by using self-report questionnaires. Second, the participants are randomly assigned to groups of two/three and solve a picture-naming task. The participants play twenty rounds and the completion time is limited to 2:30 minutes for each round. When solving the task, the participants are observed by two appraisers. One appraiser evaluates the subjects' performance in written and the other in spoken format.
Experimental Design Details
Not available
Randomization Method
Participants draw a random lot.
Randomization Unit
Individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
150 participants
Sample size: planned number of observations
150 participants
Sample size (or number of clusters) by treatment arms
Our treatment is continuous.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Ethics Committee of Paderborn University
IRB Approval Date
2022-04-28
IRB Approval Number
N/A