Experimental Design
Within the BIBB Cost Benefit Survey 2022/23, we implement a module with survey questions and an experimental design. The BIBB Cost Benefit Survey establishment panel has ample information on the firm, on training and on industry and work characteristics. To this established survey, we add questions on the decision maker’s characteristics and preferences as well as a conjoint survey experiment. The respondents are decision makers in a representative sample of German establishments.
Respondents are first asked for their consent to link their answers to social security data from the German Institute for Labor Market Research (IAB). Second, they are asked some background questions regarding their own position and tenure within the organization, whether they have conducted an apprenticeship at the same company, their age, gender, education, and decision-making power. Then, they see an introductory text that briefly introduces the questions on preferences as well as shortly explains the relevance of preferences in this field and states that it is has become a customary practice to collect data on preferences in such settings. Fourth, they answer questions regarding preferences, in particular their risk aversion, time preferences, reciprocity, and trust in others.
Fifth, the respondents receive the instructions regarding the survey experiment where we ask them to make hypothetical hiring decision on two hypothetical applicants with different characteristics.
Finally, they see five screens with a pair of applicants on each. All applicants are characterized by six features (where the values are drawn randomly from the indicated set):
- In the firm since {0, 2, 7, 18} years
- Experience in leadership positions {0, 1, 3, 6} years
- Expected performance after the assessment center {average, good, excellent}
- Gender {male, female}
- General knowledge and skills {average, good, excellent}
- Experience in the occupation {0, 4, 10, 19} years
The possible (positive) values for the experience attributes are computed as the 25th, 50th, and 75th percentile of the respective distributions in a representative sample of German establishments, the LIAB (longitudinal model for the years 2012-2019). All values are drawn uniformly at random.
For each pair, the respondents are asked to make two decisions: Decide for one of the two applicants and enter a proposed salary for each of the applicants. Respondents can tick the box for their preferred candidate and move a slider to indicate the wage that they deem appropriate for each candidate (in percentage relation to an average wage for a leadership position of the respondent’s own organization). We randomize the order of the items as well as the order of the two tasks (wage setting and forced choice) at the respondent level.
We aim to analyze the data from the trial by estimating average marginal component effects (AMCE, see Bansak et al., 2021 and Hainmueller et al., 2014). Using our fully randomized conjoint design, we estimate linear regressions of our two primary outcomes on dummies for the different item values.
We further aim to estimate conditional AMCEs for different subgroups of respondents defined based on respondent characteristics (see our secondary outcomes). In particular, we plan to distinguish AMCEs conditional on respondents’ gender, industry, tenure at the firm, age, and educational degree. In addition, we estimate conditional AMCEs based on the respondents’ answers to the preference / personality trait questions. While parts of these heterogeneity analyses are explorative, we also aim to test a range of specific hypotheses:
- Respondents with higher levels of reciprocity are more likely to prefer internal candidates.
- Respondents with higher levels of trust are more likely to prefer external candidates.
- Respondents with lower risk tolerance are more likely to prefer internal candidates.
- Respondents in industries with higher average education levels are more likely to prefer internal candidates.
- Respondents with longer firm tenure are more likely to prefer internal candidates.
- Respondents who have conducted an apprenticeship in the same firm are more likely to prefer internal candidates.
We aim to operationalize “higher levels of …” by median splits of the sample. We measure the preference for internal vs. external candidates by the coefficients on the dummies for different values of establishment tenure.
Literature
Bansak, K., Hainmueller, J., Hopkins, D., Yamamoto, T. (2021). Conjoint Survey Experiments. In J. Druckman & D. Green (Eds.), Advances in Experimental Political Science (pp. 19-41). Cambridge: Cambridge University Press. doi:10.1017/9781108777919.004
Hainmueller, J., Hopkins, D.J., Yamamoto, T. (2014). Causal Inference in Conjoint Analysis: Understanding Multidimensional Choices via Stated Preference Experiments. Political Analysis 22(1):1-30. doi:10.1093/pan/mpt024