Perceived Anonymity and Socially Desirable Responding in Course Evaluations: Evidence from a Die-Assisted Response Design

Last registered on April 01, 2026
View Trial History

Pre-Trial

Trial Information

General Information

Title
Perceived Anonymity and Socially Desirable Responding in Course Evaluations: Evidence from a Die-Assisted Response Design
RCT ID
AEARCTR-0018221
Initial registration date
March 26, 2026

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
April 01, 2026, 9:53 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
United Kingdom of Great Britain and Northern Ireland
Region
London

Primary Investigator

Name
Bernd Irlenbusch
Affiliation
University of Cologne
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Yona Rubinstein
PI Affiliation
London School of Economics
Contact Investigator

Additional Trial Information

Status
Completed
Start date
2026-03-27
End date
2026-03-28
Keywords
Behavior
Additional Keywords
JEL code(s)
C91
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This study tests whether a die-assisted response option reduces socially desirable responding in student evaluations. Students answer two evaluation items on a 1-to-6 scale: one about the quality of their study program and one about the teaching quality of the course professor. For one item, participants answer directly; for the other, they may either answer directly or roll a die and report the resulting number. The assignment of the die option and the order of the two questions are counterbalanced across four questionnaire versions. The main hypothesis is that responses to questions for which the die option is available will be less socially desirable, and therefore more honest on average, because participants can attribute their response to chance. The main analysis compares ratings in die versus non-die conditions while controlling for question type and order, with heterogeneity analyses by question and robustness checks using ordinal and binary-response models.

External Link(s)

Registration Citation

Citation
Irlenbusch, Bernd and Yona Rubinstein. 2026. "Perceived Anonymity and Socially Desirable Responding in Course Evaluations: Evidence from a Die-Assisted Response Design." AEA RCT Registry. April 01. https://doi.org/10.1257/rct.18221-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
For the question assigned to the die condition, respondents are told that they may either answer truthfully or roll a six-sided die and report the resulting number. For the direct-response question, participants answer truthfully. Participation is voluntary and anonymous, and no names are recorded.
Intervention (Hidden)
Intervention Start Date
2026-03-27
Intervention End Date
2026-03-28

Primary Outcomes

Primary Outcomes (end points)
The primary outcomes are the numeric ratings for:
• study program quality
• professor teaching quality
Each outcome ranges from 1 to 6.


Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
A useful secondary outcome is an indicator for a relatively critical response, for example:
• rating of 1–3 versus 4–6, or
• rating of 1–2 versus 3–6
This can help quantify whether the die option increases the probability of a negative or critical evaluation.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The study uses a within-subject two-question design combined with between-subject variation in question order and die assignment. Each participant answers the following two questions on a 1-to-6 scale:
• Question A: “Overall, how would you rate the quality of your study program?”
• Question B: “Overall, how would you rate the professor’s teaching in this course?”

The response scale is:
1 = very poor
2 = poor
3 = fair
4 = good
5 = very good
6 = excellent
Each participant receives one of four questionnaire versions:
• AdB: Question A first with die option; Question B second without die option
• ABd: Question A first without die option; Question B second with die option
• BdA: Question B first with die option; Question A second without die option
• BAd: Question B first without die option; Question A second with die option
Thus, order and die assignment are both counterbalanced across participants.
Experimental Design Details
Randomization Method
Reshuffling of instruction sheets (4 x 25) by hand.
Randomization Unit
Each individual answers two question. Thus, we cluster for individuals.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
100 class participants.
Sample size: planned number of observations
100 answers without die and 100 answers with die.
Sample size (or number of clusters) by treatment arms
100 class participants. Each one answers two questions - one with a die and one without.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials