Experimental Design
This experiment runs in two waves. In the first wave, a pool of job “workers” complete test questions and a background questionnaire; their performance and attributes generate the profiles shown to “employers.” This wave tests no hypothesis. In the second wave, employers decide how much to pay to hire these workers.
Worker task. Workers answer questions in two domains, math and literature (5 warm-up and 20 official questions each), and report gender, age, highest education, and preferred school subject; country is recorded by Prolific. A profile displays a worker’s gender, country, age, education, preferred subject, and answers to three fixed warm-up questions in the relevant domain (chosen from the five warm-ups using the realized worker pool, before the employer wave).
Employer task. Employers are told their randomized setup (real or simulated) and domain (math or literature) in the hiring-task instructions. For each worker, an employer sets the maximum wage (0–100 cents) they would pay to hire that worker. The worker is hired if this wage is at least a price drawn at random between 0 and 100 cents; if hired, the employer earns 5 cents per correct answer the worker gave on the 20 official questions in the employer’s domain, minus the price. Each employer evaluates 18 distinct workers across three sessions of six (drawn at random from the pool, gender-balanced with three male and three female per session): Session 1 without AI; Session 2 with AI; Session 3 again without AI. In Session 2, the employer additionally sees, for each worker, a recommended wage from a commercial AI chatbot (GPT-6.1 Sol, default parameters) that observes the same profile as the employer.
We vary three treatments, two between employers and one within. The between-employer treatments are:
- Setup: in the “real” setup, employers are told “the hiring is real, so the worker you hire actually receives the money you pay”; in the “simulated” setup, “the hiring is simulated, so the worker you hire never actually receives the money you pay.”
- Domain: each employer’s task and payment concern either math or literature; the profile shows only that domain’s warm-up performance.
The within-employer treatment is explanation: whether the AI’s recommendation shows a brief explanation of its reasoning or the wage number alone. For every employer, two of the three workers per gender show the explanation, and one shows the number alone (four of the six AI recommendations carry an explanation); which workers is randomized within gender.
The AI recommendations are generated offline in advance with default parameters. For every profile, we additionally generate offline the recommendation under the opposite gender label, holding all else fixed, and we regenerate recommendations under additional frontier LLM models; these support exploratory analyses.
After the 18 decisions, employers complete a prediction task: they predict the average number of correct answers (out of 20) for each of nine worker groups (male; female; education above and below a fixed threshold; younger and older than a fixed age threshold; and favorite subject math, science, or English; both thresholds are set from the realized worker pool before the employer wave). Finally, employers answer a background questionnaire (gender, age, education; hiring experience; AI-usage frequency; trust in AI generally and in the task recommendations; and whether the AI’s recommendations reflected worker performance). We give employers two attention checks and drop from analysis any employer who fails either.
Payments. Employers receive a show-up payment; in addition, one of each employer's 27 decisions (18 hiring, 9 prediction) is selected at random for that employer's bonus. If a hiring decision is selected, a price p is drawn at random on [0, 100]: if w >= p, the worker is hired at price p and the employer earns 5 cents per correct answer the worker gave on the 20 official questions, minus p; otherwise, no hiring. A 100-cent buffer is added whenever a hiring decision is selected (so the realized bonus for employers is never negative; it does not affect the decision-relevant surplus). If a prediction is selected, we use a binarized quadratic scoring rule: the employer receives a 200-cent bonus with probability max(0, 1 - (p - t)^2 / 100), where p is the prediction and t is the true group average (both out of 20). Workers receive a show-up payment for the first-wave task and, in the real setup, an additional bonus equal to the sum of the prices p from every selected-for-payment hiring decision in which a real-setup employer hires them (w >= p); in the simulated setup, they receive no such bonus.
Outcome measures. Write w in [0, 100] for the wage offer (cents) and V = 5s for the worker's true value (cents), where s is the number of correct answers (out of 20) in the employer's domain. Per decision and in expectation over the BDM price (buffer excluded), the worker surplus is w^2/200 cents, the employer surplus is Vw/100 - w^2/200 cents, and the expected bonus lost (inverse accuracy; lower is more accurate) is (V - w)^2/200 cents.