|
Field
Last Published
|
Before
March 13, 2023 08:47 AM
|
After
June 22, 2023 04:41 PM
|
|
Field
Intervention (Public)
|
Before
We designed a three-stage experiment. In stage 1, surveyors will perform a door-to-door survey with favela residents and ask questions about their socioeconomic characteristics and previous work experiences. During the survey, participants will be asked if they want to share their information with an HR firm (composed of the research team members), which will send them real job ads from our partner. The participants will also perform a skill test and share their beliefs and attitudes about address-based discrimination.
In stage 2, one to eight days after a jobseeker answers the survey from stage 1, the HR firm contacts that person via WhatsApp with an invitation to apply for a set of real full-time sales jobs. Applicants can then fill out an application form. We randomize whether applicants need to disclose their home address to apply, manipulating whether they can expect to suffer address-based discrimination or not.
In stage 3, we invite eligible jobseekers for an interview. Before the interview, with objectivity as the pretext, the receptionist either tells candidates that the interviewer knows their name only or their name and address.
|
After
We designed a three-stage experiment. In stage 1, surveyors will perform a door-to-door survey with favela residents and ask questions about their socioeconomic characteristics and previous work experiences. During the survey, participants will be asked if they want to share their information with an HR firm, which will send them real job ads from our partner. The participants will also perform a skill test and share their beliefs and attitudes about address-based discrimination.
In stage 2, one to eight days after a jobseeker answers the survey from stage 1, the HR firm contacts that person via WhatsApp with an invitation to apply for a set of real full-time sales jobs. Applicants can then fill out an application form. We randomize whether applicants need to disclose their home address to apply, manipulating whether they can expect to suffer address-based discrimination or not.
In stage 3, we invite eligible jobseekers for an interview. Before the interview, with objectivity as the pretext, the receptionist either tells candidates that the interviewer knows their name only or their name and address.
AMENDMENT (2023-06-21): We have included an information treatment in stage 1. We randomly reveal to some participants the actual callback rates in an audit study in which we sent fictitious resumés from the favela and non-favela neighborhoods for sales positions in Rio. Participants are randomly assigned to receive either i) no information, ii) the estimated callback rate for the favela neighborhood, or iii) the estimated callback rate for both neighborhoods.
|
|
Field
Intervention End Date
|
Before
July 30, 2023
|
After
August 16, 2023
|
|
Field
Primary Outcomes (End Points)
|
Before
In stage 2:
Progress-related:
- Completing the job application form
- Showing up for the Interview
Effort-related:
- Effort in application form: Length and quality of the text the jobseeker wrote about themselves when applying for the job.
Strategy-related:
- Strategic information disclosure: declaring or omitting information on jobs and courses taken in the favela, inflating the duration of job experiences.
- Share of individuals misrepresenting their addresses when applying to the job (not an outcome per se, but an important descriptive statistic).
Outcomes in stage 3:
- Overall interview performance
- Nervousness as assessed by the interviewer and self-assessed
- Formality/professionalism, as assessed by the interviewer and self-assessed.
|
After
In stage 2:
Progress-related:
- Completing the job application form
- Showing up for the Interview
Effort-related:
- Effort in application form: Length and quality of the text the jobseeker wrote about themselves when applying for the job.
Strategy-related:
- Strategic information disclosure: declaring or omitting information on jobs and courses taken in the favela, inflating the duration of job experiences.
- Share of individuals misrepresenting their addresses when applying to the job (not an outcome per se, but an important descriptive statistic).
Outcomes in stage 3:
- Overall interview performance
- Nervousness as assessed by the interviewer and self-assessed
- Formality/professionalism, as assessed by the interviewer and self-assessed.
AMENDMENT (2023-06-22):
Outcomes for the information treatment:
First-stage belief change:
- Beliefs about our partner HR firm's callback rates for favela and non-favela neighborhoods (measured in stage 1)
Job application outcomes:
- Completing the job application form
- Show up for job interview
- Number of jobs applied for in the last two weeks (measured in endline survey)
Effort-related:
- Effort in application form: Length and quality of the text the jobseeker wrote about themselves when applying for the job.
Strategy-related:
- Share of individuals misrepresenting their addresses when applying to the job
|
|
Field
Experimental Design (Public)
|
Before
The design is mainly explained together with the intervention description.
For stage-2 treatments, stage 1 respondents are randomized into one of the three treatment groups with equal probability. We will stratify the randomization by predicted discrimination.
For stage 3, those who completed the job application form are randomized into the two treatments with equal probability. We will stratify the randomization by predicted discrimination and stage-2 treatment status.
|
After
The design is mainly explained together with the intervention description.
For stage-2 treatments, stage 1 respondents are randomized into one of the three treatment groups with equal probability. We will stratify the randomization by predicted discrimination.
For stage 3, those who completed the job application form are randomized into the two treatments with equal probability. We will stratify the randomization by predicted discrimination and stage-2 treatment status.
AMENDMENT (2023-06-22): The information treatment is randomized with equal probability, without stratification.
|
|
Field
Randomization Method
|
Before
Randomization done by a computer. Randomization will proceed in batches every few days, since all stages overlap.
|
After
Randomization done by a computer. Randomization will proceed in batches every few days, since all stages overlap.
AMENDMENT (2023-06-22): The information treatment is randomized directly in the offline Qualtrics form, on the spot.
|
|
Field
Planned Number of Clusters
|
Before
2500 participants
|
After
2500 participants
AMENDMENT (2023-06-22): given the current budget and timeline, we project to have up to N=2100 by August. That means up to about N=700 for the information treatment experiment.
|
|
Field
Planned Number of Observations
|
Before
2500 participants
|
After
2500 participants
AMENDMENT (2023-06-22): given the current budget and timeline, we project to have up to N=2100 by August. That means up to about N=700 for the information treatment experiment.
|
|
Field
Sample size (or number of clusters) by treatment arms
|
Before
Stage 2: 833 participants in each arm.
Stage 3: 350 interviewees in each arm, assuming a 35% application rate and 85% interview show-up rate.
Sample sizes are target values and are subject to budget constraints.
|
After
Stage 2: 833 participants in each arm.
Stage 3: 350 interviewees in each arm, assuming a 35% application rate and 85% interview show-up rate.
Sample sizes are target values and are subject to budget constraints.
AMENDMENT (2023-06-22): Information treatment should have up to about 233 (700/3) observations per arm.
|
|
Field
Intervention (Hidden)
|
Before
We will conduct a three-stage intervention, as outlined below.
Stage 1: collect data on a sample of jobseekers.
Stage 2: offer an opportunity to apply for interested people from stage 1
Stage 3: job interview with stage 2 applicants
Stage 1 - Academic Survey
In this step, a team of surveyors will conduct door-to-door surveys with favela residents in the city of Rio de Janeiro. The survey includes:
- Socioeconomic characteristics.
- Labor market experiences.
- Interest in sharing their information with an HR firm, which will send them job ads.
- Eliciting beliefs and attitudes about discrimination
- Expectations regarding the job search.
Respondents must be 18-40, with either complete high school or completing it in 2023, and be looking for a full-time formal job.
Stage 2 - Job Application
A few days after a jobseeker answers the survey from stage 1, a (fictitious) HR
firm contacts that person via WhatsApp with an invitation to apply for a set of real full-time jobs with our partner -- a large firm in the perfumes and cosmetics retail sector. These jobs are mainly sales-related, including Sales Consultants (working in stores) and Direct Sales Representatives, who mainly recruit new direct resellers. These positions require completed high school and some sales and computer skills.
In stage 2 participants are randomly assigned to one of three conditions:
i) Address Omission
ii) Known Address
iii) Control (status quo)
Applicants in the Address Omission condition receive a WhatsApp message saying a home address is not needed for applying. Those in conditions ii) and iii) receive a message saying an address is needed. The application form tells those in the Known Address condition that the HR firm already knows their addresses. Control participants declare their home addresses as they wish (the most common practice in our context), which also allows us to observe how applicants use strategies to obfuscate their real addresses.
We are interested both in pairwise comparisons of the three treatment arms and also comparing Address Omission to the other two conditions. The latter provides us with a higher-powered comparison for the effect of having to disclose home address.
- Comparing Address Omission to Control gives us the effect of removing address from the early stages of the application process, which can be policy-relevant for firms, for instance.
- Comparing Control to Known Address mimics the shift to a case where stigma can not be hidden. This comparison also measures how much difference address obfuscation strategies make when address is required.
- Comparing Known Address to Address Omission maximizes the shift in stigma visibility.
The application form has the usual fields for education, job experiences, and other courses and skills. It also has an open text box where applicants can introduce themselves and say why they are a good fit for the job – our main effort outcome.
Stage 3 - Interviews
In this step, the HR company contacts participants who completed the application form and conducts job interviews.
We plan to invite all applicants who finished the application in stage 2 to an interview. However, if facing budget restrictions, we might stop interviews before all applicants get an interview invite.
Interviews will take place in rented office space in Rio de Janeiro’s downtown, staffed with a receptionist and an interviewer. At the reception of the interview office, interviewees will be randomly told, by the receptionist, what the interviewer knows about them. Half will be told that to keep the process objective the interviewer will know only their name, whereas the other half will be told that the interviewer will know only their name and address. At the moment of the interview, the interviewer will only know the person’s name – addresses will only be revealed to interviewers later.
Finally, participants fill out a form with feedback for the interviewer, the HR firm, and self-assessment questions.
We are also interested in understanding how stigma visibility and anticipated discrimination drive selection into the labor market. Hence we conduct four main types of heterogeneity analysis
First, to understand whether our treatments affect applicants by shifting their perceived probabilities of suffering discrimination, we will explore treatment effect heterogeneity by expected discrimination level. To understand whether it is the level of perceived discrimination or attitudes related to it, we also do this exercise for attitudes such as “how bothered one is by discrimination” and “whether the possibility of being discriminated against in the hiring process is motivating or discouraging”.
Second, to understand the effects on the skill mix available to employers, we explore heterogeneity by skill level as measured by education and our math test.
Third, we explore heterogeneity by race since it is usually visible and one of the key reasons for address-based discrimination (as perceived by our participants).
Fourth, we explore heterogeneity by gender, since it is especially policy relevant and we have some evidence that address-based discrimination is perceived as more bothersome by females. The latter also applies to Black respondents.
|
After
We will conduct a three-stage intervention, as outlined below.
Stage 1: collect data on a sample of jobseekers.
Stage 2: offer an opportunity to apply for interested people from stage 1
Stage 3: job interview with stage 2 applicants
Stage 1 - Academic Survey
In this step, a team of surveyors will conduct door-to-door surveys with favela residents in the city of Rio de Janeiro. The survey includes:
- Socioeconomic characteristics.
- Labor market experiences.
- Interest in sharing their information with an HR firm, which will send them job ads.
- Eliciting beliefs and attitudes about discrimination
- Expectations regarding the job search.
Respondents must be 18-40, with either complete high school or completing it in 2023, and be looking for a full-time formal job.
Stage 2 - Job Application
A few days after a jobseeker answers the survey from stage 1, an HR firm contacts that person via WhatsApp with an invitation to apply for a set of real full-time jobs with our partner -- a large firm in the perfumes and cosmetics retail sector. These jobs are mainly sales-related, including Sales Consultants (working in stores) and Direct Sales Representatives, who mainly recruit new direct resellers. These positions require completed high school and some sales and computer skills.
In stage 2 participants are randomly assigned to one of three conditions:
i) Address Omission
ii) Known Address
iii) Control (status quo)
Applicants in the Address Omission condition receive a WhatsApp message saying a home address is not needed for applying. Those in conditions ii) and iii) receive a message saying an address is needed. The application form tells those in the Known Address condition that the HR firm already knows their addresses. Control participants declare their home addresses as they wish (the most common practice in our context), which also allows us to observe how applicants use strategies to obfuscate their real addresses.
We are interested both in pairwise comparisons of the three treatment arms and also comparing Address Omission to the other two conditions. The latter provides us with a higher-powered comparison for the effect of having to disclose home address.
- Comparing Address Omission to Control gives us the effect of removing address from the early stages of the application process, which can be policy-relevant for firms, for instance.
- Comparing Control to Known Address mimics the shift to a case where stigma can not be hidden. This comparison also measures how much difference address obfuscation strategies make when address is required.
- Comparing Known Address to Address Omission maximizes the shift in stigma visibility.
The application form has the usual fields for education, job experiences, and other courses and skills. It also has an open text box where applicants can introduce themselves and say why they are a good fit for the job – our main effort outcome.
Stage 3 - Interviews
In this step, the HR company contacts participants who completed the application form and conducts job interviews.
We plan to invite all applicants who finished the application in stage 2 to an interview. However, if facing budget restrictions, we might stop interviews before all applicants get an interview invite.
Interviews will take place in rented office space in Rio de Janeiro’s downtown, staffed with a receptionist and an interviewer. At the reception of the interview office, interviewees will be randomly told, by the receptionist, what the interviewer knows about them. Half will be told that to keep the process objective the interviewer will know only their name, whereas the other half will be told that the interviewer will know only their name and address. At the moment of the interview, the interviewer will only know the person’s name – addresses will only be revealed to interviewers later.
Finally, participants fill out a form with feedback for the interviewer, the HR firm, and self-assessment questions.
We are also interested in understanding how stigma visibility and anticipated discrimination drive selection into the labor market. Hence we conduct four main types of heterogeneity analysis
First, to understand whether our treatments affect applicants by shifting their perceived probabilities of suffering discrimination, we will explore treatment effect heterogeneity by expected discrimination level. To understand whether it is the level of perceived discrimination or attitudes related to it, we also do this exercise for attitudes such as “how bothered one is by discrimination” and “whether the possibility of being discriminated against in the hiring process is motivating or discouraging”.
Second, to understand the effects on the skill mix available to employers, we explore heterogeneity by skill level as measured by education and our math test.
Third, we explore heterogeneity by race since it is usually visible and one of the key reasons for address-based discrimination (as perceived by our participants).
Fourth, we explore heterogeneity by gender, since it is especially policy relevant and we have some evidence that address-based discrimination is perceived as more bothersome by females. The latter also applies to Black respondents.
AMENDMENT (2023-06-22): At this point, we have invited around N=1500 jobseekers to apply to one of the sales positions, and we have no decisive evidence that the Known Address or Address Omission treatments have changed how much discrimination jobseekers could expect from the HR firm. Since the design does not allow for manipulation checks immediately after the intervention, we added questions to our endline phone survey (initially described in the secondary outcomes section, but implemented with a 4-week delay, instead of 10 days as described there) to assess whether expectations of discrimination changed in response to the Known Address or Address Omission treatments. We find no treatment effect on whether jobseekers believe the HR firm might avoid or give preference to favela residents (N=366 at the moment). This and average null effects on application rates suggest that the stage-2 manipulation is not strong enough to produce a shift in anticipated discrimination. An alternative hypothesis is that the home address visibility manipulations did change anticipated discrimination, but that was not marginal for application decisions and such shifts in perceptions vanished before the endline survey.
To ensure we have an intervention that verifiably shifts anticipated discrimination, we are introducing an information treatment into stage-1. In this intervention, we use information from an audit study in which we applied for sales jobs with resumés with addresses either from a favela or the neighborhood right beside it. We have three treatment arms:
1) No information.
2) Information about the favela's resumés callback rates.
3) Information about favela and non-favela callback rates (revealing that find no discrimination in callback rates).
The advantage of this intervention is that we can measure whether it shifts beliefs about discrimination immediately after treatment by asking jobseekers to predict callback rates for our partner HR firm.
We have introduced the information treatment two weeks ago, and we are discontinuing the Address Omission and Known Address treatments (all job applicants will be invited with the same message and procedure from now on).
In this registry, all edits are appended to the old text, beginning where it reads "AMENDMENT (2023-06-22)". The one exception is that we clarify the passages referring to the HR firm used in the experiment (which emulates a private HR firm but was controlled by the study members).
|
|
Field
Secondary Outcomes (End Points)
|
Before
Stage 2:
- Clicking the job application form.
- Perceptions about the job-search process and interactions with firms after the stage-2 treatment (collected in phone survey, see below)
Stage 3:
- Self-assessed interview performance
- Specific behaviors during interview, that could signal stress or discomfort
- Information disclosure during interview
- Question-wise performance in interview
- Interviewee’s perceptions about the interviewer (professionalism, preparedness) and HR firm (inclusiveness).
|
After
Stage 2:
- Clicking the job application form.
- Perceptions about the job-search process and interactions with firms after the stage-2 treatment (collected in phone survey, see below)
Stage 3:
- Self-assessed interview performance
- Specific behaviors during interview, that could signal stress or discomfort
- Information disclosure during interview
- Question-wise performance in interview
- Interviewee’s perceptions about the interviewer (professionalism, preparedness) and HR firm (inclusiveness).
AMENDMENT (2023-06-22):
Outcomes for the information treatment:
- Beliefs about the odds of facing discrimination on the job in the future (measured in stage 1)
- Predicted employment and wage gap for someone like the respondent (stage 1)
- Callback probability for own job applications (measured in stage 1)
- Beliefs about job market prospects and address based discrimination (measured in endline survey, Likert scale)
- Sentiments regarding job search (e.g., how excited the jobseeker is, how fair they think the job market is, how much they feel like one has to ignore concerns regarding discrimination in the job market, etc, measured in stage 1)
- Intentions to search more intensively (measured in stage 1)
|