Experimental Design
Sample: A nationally representative (based on gender, age, geography and education) sample of 500 adult Danish residents. Participants are invited to a YouGov survey on decision making.
Experimental manipulation. We elicit participants’ beliefs about the flyers advertising house cleaning operators. We orthogonally manipulate flyer design, name of the worker to signal migration or native background, the hourly price requested, and the rating awarded (using stars) to the operator.
Measures. We elicit probabilistic beliefs about advertised worker characteristics on the domains of trustworthiness (in general and in working alone at clients’ homes), professionalism, flexibility, diligence, education level, work experience, and language fluency. See attached survey material. Each participant evaluates 2 flyers, with different designs and at least one of the three key characteristics manipulated and randomly ordered. This leaves us with 1000 observations in total.
Attention checks. We want to make sure our sample pays attention to the instructions. We use a simple attention check hidden among a set of belief questions that asks respondents to assign a specific probability to a specific response on a likert scale. A similar attention check is implemented also just before the belief elicitation and we alert respondents to an incorrect response, to increase the chance of attention paid in subsequent tasks. Our main analysis will use the attentive sample, while in secondary analyses we study the entire sample.
Comprehension check. After reading the instructions and before answering the questions, respondents get prompted with a comprehension check, asking three questions about probability distributions. Those who fail will see the correct answers highlighted in red. Our main analysis will use the sample of individuals with perfect comprehension (answering all three questions correctly), while in secondary analyses we study the entire sample.
Outcomes.
We generate a standardized index across the following belief measures: trustworthiness, professionality, flexibility, trustworthiness in working alone, timeliness, experience, fluency. We construct the index as follows: First, we assume that higher values are considered better (e.g., more flexibility is better than less flexibility). Second, we calculate weights with a likert scale centered around 0 at the neutral option (for experience, we take the 3-5 years option as equal to 0) according to elicited beliefs for each option for a given variable. Finally, we combine all the variables in a single standardized index. Higher values imply more positive beliefs.
Hypotheses.
Primary hypotheses (standardized beliefs index as outcome):
(1) Given price and quality, a native operator is more positively evaluated than a migrant operator.
1a) As a secondary hypothesis, we also test hypothesis (1) separately by different prices.
(2) Given quality, the size of the gap in positive beliefs between native and migrant operators increases with price (difference-in-differences).
Multiple hypothesis testing:
To address multiple hypothesis testing, we correct p-values using the method developed by Barsbai et al. (2020), which extends the procedure of List et al. (2019) by allowing for corrections in multivariate regression models.
Secondary hypotheses:
We examine the beliefs for each of the eight measures separately by answering the same questions as in the primary hypotheses.
We also study full distributions to understand if it is only means that change by treatment or if it is also variance that changes.
Heterogeneity by Socio-Demographic Characteristics:
We test whether the results vary based on socio-demographic characteristics such as gender, education, age, and income. For continuous variables (e.g., age, income), the sample is split at the median to facilitate subgroup analysis.
Sanity check:
Given name and price, a higher quality operator is more positively evaluated than lower quality operator.
Controls and Robustness Checks:
In all regressions, we control for demographic characteristics. Robustness checks include sensitivity analyses for including or excluding covariates. We also implement a two-step LASSO procedure to select control variables.
Standard Errors:
Standard errors are clustered at the individual level.
Detailed instructions: see attached script