Can Feedback on Racial Bias Change the Behavior of Labor Market Counselors: A RCT in Public Employment Agencies in Colombia

The intervention has two stages. In the first stage, we will invite all job-counselors working in the public job-centers network in Colombia to take an IAT about race. The test has been adjusted for the context of Colombia. After the IAT we will collect information about explicit racial bias.
In the second stage, we will randomly select a group of job-counselors (clustering at job-center level) to give them feedback about their own IAT result. Our treatment variable will be to be working in a job center selected for receiving IAT feedback. We will collect data and analyze short- and medium-run effects. After the analysis is complete, job counselors in the control job centers will receive feedback.

2021-10-01

2021-10-18

Job counselors’ referral rate to jobs of Afro-descendant clients.
Employment of Afro-descendant clients.
Job counselors’ referral of Afro-descendants to a job posting in a simulated task.
Job counselors’ willingness to participate in inclusion training.

The main idea is to measure the effect of the IAT feedback on job-counselors behavior and related labor market outcomes of Afro-descendant individuals. We will use administrative records and a follow-up survey to create the relevant indicators. The first source is the placement data in the job-centers centralized by the UAESPE. In this case, it would be necessary to identify for each resume in the system, the job-counselor (or at least in the employment center) who handles each case. From this data, we can measure both the referral rate and labor market participation of Afro-descendant individuals.

The second alternative is to search for job applicants in the Integrated Contribution Settlement Worksheet (PILA). As has been done in the past, if a person appears in the PILA it indicates that they have a formal job since they are making their contributions to social security (see for example Attanasio et.al, 2007 and Attanasio et.al, 2017).
At this point, it is necessary to clarify that the final effect on employability does not only depend on the intermediation of the counselor, and is limited by the additional frictions in the supply and demand of work in Colombia. The short time frame for the labor market intermediation of a small sample size of Afro-descendant clients presents challenges for measuring impacts on job placement. That is why it is especially important to incorporate a simulation exercise that can provide information on behavior related to the responsibilities of job counselors and linked to long-run employment outcomes.

For the latter, we will collect data through an online work session with the job-counselors that seeks to closely replicate the activities that they carry out at the time of receiving a new job seeker (we closely follow works such as Van Borm et.al, 2021). In this case, in a controlled manner, each counselor participating in the study will be assigned a group of resumes containing the most relevant information from the hypothetical job seekers. Each job-counselor will receive 7 random CVs where we control for variables such as gender, race, education, and experience. The race of the candidate is revealed through the photo on the CV. The photos are composites of publicly available images rather than photos of individuals. After briefly reviewing the 7 CVs, the counselors will receive 3 random different job posts and will have to refer 3 from their 7 CVs. In the last activity, they will receive 3 posts for training courses and will have to refer 3 CVs to each course.

References:
Alesina, A., Carlana, M., La Ferrara, E., & Pinotti, P. (2018). Revealing Stereotypes: Evidence from Immigrants in Schools. NBER Working Paper Series(25333).
Attanasio, O., Guarín, A., Medina, C., & Meghir, C. (2017). Vocational training for disadvantaged youth in Colombia: A long-term follow-up. American Economic Journal: Applied Economics, 131-143.
Attanasio, O., Kugler, A., & Meghir, C. (2011). Subsidizing vocational training for disadvantaged youth in Colombia: Evidence from a randomized trial. American Economic Journal: Applied Economics, 188–220.
Van Borm, H., Burn, I., & Baert, S. (2021). What Does a Job Candidate's Age Signal to Employers? Labour Economics.

Experimental Design
In this study we will replicate the work of Alesina et.al (2018), in Italian schools, adapted for the context of racial bias among job counselors in public employment centers in Colombia. This design seeks to answer the following questions:

1. Is there an unconscious bias against the Afro-descendant population in job-counselors in Colombia?
2. Do job-counselors change their behavior related to job referrals when they know their bias?
3. Are job counselors with low levels of explicit bias more responsive to the feedback than their peers who have reported high levels explicit bias?
4. Could this low-cost intervention have relevant impacts on Afro-descendant's employment?

To answer these questions, the study has 3 stages: (i) list exercise of counselors in participating agencies and baseline measurements, (ii) random assignment and feedback, and (iii) collection of information on results and estimation of impacts. In the first stage, an invitation was made to all job-agencies in Colombia inviting them to participate in the study. According to the information provided by Asocajas so far, the study universe would be composed of:

31 job-agencies (cajas de compensación familiar)
129 job-centers
29 departments
103 municipalities
548 job-counselors (including business managers and intermediaries who have contact with clients)

After the invitation, we aim to work with approximately 400 job-counselors in approximately 110 job-centers. At this stage, all job-counselors participating in the study will take the IAT test and a survey including questions about racial biases.
In the second stage, job-centers will be randomly divided in two groups, treatment and control:

Treatment group: Job-counselors in this group will receive feedback on their IAT test result just after phase 1 ends.
Control group: The counselors in this group will receive their results until the end of the study.

Rigorous randomization will be carried out, taking into account the size of the job-center, the share of the Afro-descendant population in each region, and other characteristics of local labor markets. To reduce the effects of contamination (changes in the control group due to its interaction with the treatment group) and to reduce possible frustration effects (changes in the control group due to not receiving its result), the random assignment will be carried out at the job-center level.

In the third stage, the counselors of both groups will be followed and we will collect information on both their activities and the results of those job seekers who interact with them in order to estimate the impact of the feedback as explained in the next section. In addition, we will collect data about job-counselors activities using an online activity where we will simulate the activities they carry out at the time of receiving a new job seeker. In this case we will randomly assign simulated CVs, job posts, and training courses. We first create all possible combinations of key characteristics in a CV or a job post (for example, education, gender, race for the CVs; and sector, wage, experience required for the job-posts). Then we will randomly create assign groups of 7CVs and 3 job-posts following the strategy of CITA).

The information about job centers will be collected at the moment of the study registration. In case this information is not fully reliable (high proportion of missing values, confusion with online services, or the use of different names for the same job center), we will randomize at the job-agency-city level (caja de compensación-ciudad). For large cities, such as Bogotá, we may randomize at the job-agency-city-locality level.

Randomization will be done in the office by a computer.

The unit of randomization is the job-center (or job-agency-city/locality).
We will create strata for the job-centers by region (to control for local labor markets and proportion of Afro-descendant population) and size of the job-centers.

Yes

We aim to have 129 job centers. If we use job-agency-city clustering we aim to have approximately 80 clusters.

We aim to have approximately 400 job-counselors. The total number of observations from administrative data depends on the final number of job-applications and posts that the system will receive for the period of the study. For the simulated exercise we will have 7 times 3 times N observations. Where N is the final number of job-counselors that take the exercise.

200 job-counselors by group.
50 job-centers by group.

Taking into account that we expect to have 50 job-centers in each arm (treatment and control), and about 400 job-counselors in total. For the simulated exercise, we will have 400 (individuals) times 7 (CVs) times 3 (job posts) observations (about 8,400). According to Peña & Wills (2010), 45.9% of Afro-descendant individuals had a formal job. Thus, for a power of 80% and a confidence of 95%, we estimate an MDE of 3.2 percentage points. This means that we could observe changes in the formal labor market participation of Afro-descendant individuals from 49.1%. The final MDE for the exercise with administrative information will depend on the total number of job applications and posts that we will be able to observe for the period of the study. Reference. Peña, X., & Wills, D. (2010). Ethnic earnings gap in Colombia. Banco de la República’s Seminario Semanal de Economía, Bogotá, June, 16.