AI Integration in Hiring: A Randomized Experiment

This study employs a 2×2 factorial randomized controlled trial to examine the effects of AI skills training and AI-assisted CV evaluation on recruitment outcomes and subsequent employee performance. Applicants are randomly assigned to one of four groups that vary whether they receive AI training and whether their applications are screened using AI-assisted CV evaluation or conventional human review. Applicants assigned to the training arms complete an AI skills module prior to the recruitment process, while applicants in the AI-screening arms have their CVs initially assessed by an AI tool before final review by recruiters.

This design enables us to separately estimate the causal effects of AI training, AI-assisted recruitment, and their interaction. Specifically, it allows us to assess whether equipping applicants with AI skills improves their recruitment and workplace performance, whether incorporating AI into candidate screening changes selection decisions and employee quality, and whether combining applicant training with AI-assisted recruitment generates complementary or substitutive effects.

2026-07-15

2027-05-31

Applicant and Application Metrics
These metrics focus on performance and outcomes during the recruitment process itself. They include both objective data points and subjective evaluations.
Test Outcomes: Scores and performance on the initial remote tests.
Resume Scores: Scores assigned to CVs by both human and AI evaluators.
Interview Ranking: The final ranking of candidates eligible for interviews.
Interview Outcomes: Performance scores from the interview stage for finalist candidates.

Post-Hire Performance Metrics
These metrics are collected after a candidate is hired to assess long-term impact and the quality of the hiring decision.
Hire Quality: Measured via performance reviews conducted at 3, 6, and 12 months, as well as manager satisfaction surveys collected quarterly.
Retention Rates: Tracking the tenure of employees hired through the different experimental paths.
Productivity Measures: Quantitative and qualitative assessments of employee output and efficiency.
Diversity Metrics: Analysis of the demographic composition of hires across the different treatment arms.

### Experimental Design

Randomization is conducted at the **individual applicant level**. In each hiring cycle, all eligible applicants are randomly assigned by the recruitment system to one of the four treatment arms immediately after submitting their application. Assignment probabilities are equal across treatment arms, ensuring that applicants have an equal likelihood of being allocated to any of the four experimental conditions.

Treatment assignment determines both whether the applicant is offered AI skills training and whether their application undergoes conventional human review or AI-assisted CV evaluation. Once assigned, applicants remain in their allocated treatment arm throughout the recruitment process. Randomization occurs independently for each hiring cycle, ensuring that every applicant within a cycle is exposed to only one treatment condition and allowing causal comparisons both within and across hiring rounds.

Not available

Randomisation is done by a computer.

Applicant level using course stream as a block.

No

N/A

1304

326

0.40 standard deviation