The Effect of Interview Difficulty on Job Seeker Perceptions and Behavior

Last registered on July 03, 2025
View Trial History

Pre-Trial

Trial Information

General Information

Title
The Effect of Interview Difficulty on Job Seeker Perceptions and Behavior
RCT ID
AEARCTR-0016332
Initial registration date
July 03, 2025

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
July 03, 2025, 3:59 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
Switzerland
Region

Primary Investigator

Name
Soumitra Shukla
Affiliation
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Elliott Ash
PI Affiliation
ETH Zurich
Contact Investigator
PI Name
Jason Sockin
PI Affiliation
Cornell University
Contact Investigator

Additional Trial Information

Status
In development
Start date
2025-07-07
End date
2025-10-08
Keywords
Labor
Additional Keywords
JEL code(s)
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This experiment tests how the difficulty of job interview questions affects candidates’ perceptions of and interest in a job ad.
External Link(s)

Registration Citation

Citation
Ash, Elliott, Soumitra Shukla and Jason Sockin. 2025. "The Effect of Interview Difficulty on Job Seeker Perceptions and Behavior." AEA RCT Registry. July 03. https://doi.org/10.1257/rct.16332-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
Intervention (Hidden)
This experiment directly manipulates the actual interview complexity to see how it shapes perceived difficulty, self-reported interest, and behavioral engagement. Specifically, we assess whether "difficult" interviews might act as a positive signal of employer selectivity or prestige, in line with our prior observational evidence.
Intervention Start Date
2025-07-07
Intervention End Date
2025-10-08

Primary Outcomes

Primary Outcomes (end points)
We measure several primary outcomes, including application completion and engagement.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We intervene on interview characteristics to see its effects on candidate engagement.
Experimental Design Details
We recruit applicants for a data scientist position. To be eligible:

1. Applicants must submit a complete CV/resume.
2. They must complete the first-round technical interview questions online.
3. They must provide valid contact information for follow-up.

We have two arms, assigned with equal probability (1:1):

1. Treatment E (Easy): Simpler technical tasks (e.g., working with 10MB files, interpreting linear regression, basic probabilities, basic regular expressions, and writing a simple function in Python/R.)

2. Treatment D (Difficult): More advanced technical tasks (e.g., working with 10GB files, explaining the limitations of regression analysis, advanced probabilities, more involved regular expressions, and writing a more advanced function in Python/R.)

To avoid bias in actual hiring outcomes, the hiring manager (Elliott) will not see applicants' responses to the randomized questions. Instead, actual hiring decisions will be based solely on resumes/CVs. This design preserves the integrity of the random assignment and ensures that participants in the "easy" arm are not penalized or favored relative to those in the "difficult" arm.
Randomization Method
Computer-generated random assignment (no stratification).
Randomization Unit
Individual applicant.

Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
200 or more applicants (units).
Sample size: planned number of observations
200 or more applicants (units).
Sample size (or number of clusters) by treatment arms
100 or more observations in each treatment arm
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
Institutional Review Board for Human Participants, Cornell University
IRB Approval Date
2025-06-27
IRB Approval Number
0149540
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials