Experimental Design
Our experiment consists of a multi-stage only survey with experimental components and demographic and work background questions.
The demographic and work background information asked are split into two sets, one at the very beginning of the survey and another one after the experimental part, to avoid fatigue effects on answers to main stages.
At the beginning, participants are asked:
- age, gender (key moderator variable of the analyses) and ethnicity
- education level
- employment status
- industry they work in / want to work in and respective occupation
After all the experimental stages, participants are asked:
- income level
- political views
- caring responsibilities and number of children (if any)
- current work seniority (if employed)
- current gender composition of colleagues (if employed)
- gender of current supervisor/manager (if employed)
- any managing experience
Stage 1: Discrete Choice Experiment (DCE)
After the first set of demographics, each participant is administered ten discrete choice (or stated-preference) experiments. They have to choose the preferred job between two hypothetical job offers each time, presented in a table format. An additional attention check job comparison brings the total to eleven choices, as explained below.
Each job offer varies in terms of the following attributes: wage relative to their current wage (if employed) or to what they consider a fair wage for a role in their desired industry (if not employed) – same, +5%, -5%, +10%; gender of the direct supervisor – female, male; schedule flexibility arrangements (relative to a standard 40-hour working week) – no flexibility, 1 hour flexibility at beginning and end of workday, entirely flexible schedule; gender composition of immediate colleagues- similar share of females and males, majority females, majority males; skills development opportunities – low, moderate, high.
We instruct respondents to assume that any job attributes not mentioned in the table are identical across the two offers.
These attributes and respective levels yield us a factorial design size of 216 job profiles. Based on these 216 possible job profiles, we obtained an initial full factorial design made up of 23,220 (=216 x 215/2) possible job pairs. From this initial full factorial design we excluded the pairs containing strictly dominated jobs. To define dominance, the two jobs are compared in terms of wage, flexibility arrangements and skills development opportunities, keeping the other two gender attributes constant across the two. One job is hence dominated if it presents lower levels on one, two or all three attributes, keeping all other attributes constant. In addition, we also excluded the pairs where either the supervisor gender or the colleagues gender composition alone was the only attribute varying between the jobs, to avoid forcing participants when they may be indifferent on those characteristics. To account for this decision, we complement the survey asking direct stated preferences on these attributes, as explained in detail below.
In this way, the final clean factorial design containing all possible job pairs contains 20,952 possible pairs. Based on this, we create a row random design for our discrete choice experiment, where the 10 choice tasks are randomly selected without replacement from the clean factorial design, so that all 20,952 eligible pairs are shown at least once across the sample. We ensure this by subsequently excluding the 10 choices randomly chosen for each participant from the pool of remaining pairs from which to choose those for the following participant. This implies the existence of a “boundary” participant, for whom there will be 2 pairs left from the initial set, whereas the other 8 pairs will be randomly drawn once again from the whole clean factorial design, in a second cycle of random draws.
With this design all 20,952 possible job pairs will be presented once during the experiment, while an additional set of 4,048 twice.
In addition to the ten choices explained above, participants are shown one comparison serving as attention check, where the jobs differ only in terms of wage and flexibility. One of these jobs is clearly dominant (current wage and no flexibility vs wage +10% and fully flexible schedule). This pair is randomly shown between position 6 and 10, increasing the number of choices to eleven. The choice from this comparison will be excluded from the willingness to pay estimation.
After completing the eleven choices, participants are shown again the last job offers and their selected choice, and they are asked to provide a short motivation for their choice in an open-ended question.
Stage 2: Perceptions about Managerial Characteristics
The second stage of the experiment begins with participants rating a list of seven characteristics in terms of how important they consider them in a direct supervisor at work. The attributes are: effective leadership; fairness; empathy; work-life balance support; development opportunities; autonomy granted at work; and work culture. Each characteristic is rated on a 1 (not important at all) to 4 (very important) scale.
Next, for each characteristic respondents indicate whether they think a female or male supervisor would score higher on that characteristic. The attributes are presented using both a positive framing (e.g. “more objective and fair” for fairness) and negative framing (i.e. “less supportive of work-life balance” for work-life balance support), so that systematically choosing the same gender does not always imply a positive attribution to that gender. The positively framed characteristics are fairness, development opportunities and work culture. The negatively framed ones are effective leadership, empathy, work-life balance support and autonomy.
In presenting the question, we randomise: the order of female or male supervisor in the choices presentation of the question, to reduce framing effects by presenting one specific gender as leftmost option; the order of the characteristics.
Stage 3: Actual Job Preference
In the last stage participants are told that a real anonymous big company has an above-average share of either females or males in the top earning quartile (which proxies as likely manager gender). They have to hence state whether they would be interested in finding out more about job opportunities at such company. If their answer is affirmative, they are shown a link to the real company’s webpage of job vacancies. We track if they actually click on the link or not, to compare if stated preferences align to actual behaviour.
Additional parts of the survey
Finally, as additional section, we ask participants to state their preferences regarding the gender of their direct supervisor (female, male or no preference) and of their immediate colleagues (majority females, majority males, similar share of females and males, no preference). For each answer we ask them an open-ended question to motivate their choice as well.
This last section is complementary to the previously indicated decision to exclude from the discrete choice experiment any job pair where only one of the gender attributes varied. Indeed, in these stated preference question respondents have the possibility to indicate their indifference as well.
This section is administered after all experimental stages to avoid anchoring respondents’ choices in various stages.
We will check for inattention in 3 ways.
First of all, we use a traditional attention check question, in which we indicate specifically in the question text how participants should answer. Next, as already outlined above, we include one job comparison as attention check in the DCE, with one job strictly dominating the other. This will be positioned randomly between comparison 6 and the second to last one (10). Finally, a third attention check consists of an image with two arrows (a modified Muller-Lyer illusion) and a question asking which one is larger.
In addition to inattention, a second source of bias could be the use of AI for the completion of the survey. To flag AI usage, we employ three strategies as well. First, the third attention check represents also a so-called AI cognitive trap, as AI models will fail to answer correctly and indicate the traditional answer from the Muller-Lyer illusion (Affonso 2026). Next, we exploit the open-ended questions and transform them into honeypot questions, where a JavaScript code requires AI models and bots to answer in very specific ways, completely unrelated to the survey. However, human participants will be free to answer in the way they consider appropriate as the honeypot indications will be hidden to them. Finally, we will also employ the Prolific AI check. Any answer coming from respondents flagged as AI will be excluded.