Pre-Registration: Understanding the impact of AI on the appeal of factory jobs

1. Mentions of AI in Job Postings
The intervention involves systematically varying whether artificial intelligence (AI) is referenced in job advertisements for factory roles. Each job posting will be alternately published in one of two formats:

1.1.AI Condition: The job description includes references to AI in both the title and the main body
- Title: XYZ in the factory with AI-powered machines
- Body: Here you will work in a modern, dynamic factory environment, where we increasingly rely on artificial intelligence (AI) to optimize production processes and drive innovation

1.2.Non-AI Condition: The job description omits any mention of AI in both the title and the main body
- Title: XYZ
- Body: Here you will work in a modern, dynamic factory environment.

2. Randomization and Rotation Schedule
The assignment of AI mentions is randomized at the job-post level. For each role, the job advertisement will be published in one condition (AI-framed or non-AI) for a period of one week after which the alternative version will be posted. This within-job, cross-time randomization strategy ensures that candidates are exposed to only one version of the job ad at any given time, minimizing cross-condition contamination. The rotation schedule is designed to allow within-role comparisons over time while maintaining ecological validity in a live recruitment environment.

Labor split: Siemens is responsible for designing the experiment and will handle the randomization, scheduling, and execution. The research team (Jorge Tamayo, Raffaella Sadun, Florian Englmaier) provided input on the experimental design (e.g., rotation logic), but they will not oversee the experiment's design or implementation. The research team will analyze the resulting data to evaluate the intervention's impact.

2025-06-01

2025-12-31

Primary Outcomes (end points)
• Application volume: Total number of applications received per job posting per condition.
• Gender ratio: Proportion of male and female applicants per condition.
• Hiring speed: Time from job posting to candidate hire

Primary Outcomes (explanation)
These outcomes capture the immediate behavioral response to AI mentions in job advertisements. Application volume and response speed reflect candidate interest and urgency, while hiring speed and gender ratio provide insight into how AI mentions may influence the diversity and efficiency of the recruitment funnel.

Secondary Outcomes (end points)
• AI familiarity: Indicators of prior AI-related knowledge or interest (e.g., keywords in CVs and self-reported skills).
• Education level: Highest degree or qualification listed by applicants.
• Shortlisting likelihood: Probability that an applicant is invited for an interview.
• Hiring likelihood: Probability that an applicant is hired.

Secondary Outcomes (explanation)
These outcomes assess the quality and characteristics of applicants attracted under each condition. They help determine whether mentioning of AI influences not just the quantity but also the profile and suitability of candidates, including their potential alignment with AI-integrated work environments.

A field experiment will be conducted to measure the impact of referencing artificial intelligence (AI) in job postings on candidate behavior. The intervention will be implemented across multiple Siemens factories in different countries (e.g., the U.S. and China), using live recruitment postings for both blue-collar (e.g., machine operators) and white-collar (e.g., industrial engineers) factory roles.

Each eligible job posting will be randomly assigned to start in either the AI or non-AI condition, and will alternate weekly between the two formats. Each post is expected to undergo at least one full rotation cycle—i.e., it will be live for one week in the AI condition and one week in the non-AI condition—with some postings possibly undergoing multiple cycles depending on the length of the recruitment period.

Not available

Randomization Method
Random assignment of AI mention conditions is implemented using a structured rotation protocol at the job-post level. Within each location, every second eligible job posting is randomly assigned to start in the AI condition, while the others begin in the non-AI condition.

Following this initial assignment, each job post alternates weekly between the two conditions (AI vs. non-AI) for the duration of the posting. This allows for within-role comparisons over time while preserving ecological validity.

The assignment sequence is automated and tracked using a centralized system maintained by Siemens' recruitment operations team. Treatment status is not visible to recruiters or applicants, minimizing implementation bias.

Random assignment of AI mentions conditions is implemented using a structured alternation protocol. Every second job posting within each location is initially published with AI mentions, while the others begin in the non-AI condition. The assignment sequence is generated and tracked using a computerized system and is concealed from recruiters and candidates to prevent bias.

The unit of randomization is the job-post level, stratified by geography and role type. Randomization occurs independently within each geography (e.g., United States, China) and role category (blue-collar vs. white-collar), resulting in four clusters. Within each cluster, the probability of assignment to either condition (AI-framed or non-AI) is equal.

Yes

We are randomizing four clusters of job postings.

We are randomizing between 30-35 job posts, with an expected distribution of approximately 80% blue-collar and 20% white-collar roles – depending on the recruitment needs per a specific location during the study period (June-December 2025)

We are randomizing between 30-35 job posts, with 50/50 allocation between starting with AI vs non-AI mentioning wording.

In our power calculations, we use the effects of previous job postings that mentioned AI for both white- and blue-collar roles as the baseline. To achieve 80% statistical power, we estimate the following requirements: with 150 applicants, at least 35 job postings are needed for each group (blue- and white-collar). With 100 applicants, the required postings increase to 45 for blue-collar roles and 35 for white-collar roles.