Finding the best employees: A field experiment in hiring

We will have the current employees of a microfinance company in Kyrgyzstan answer a questionnaire, measuring their risk and time preferences, trust and trustworthiness, altruism, Big 5 personality traits, performance in the Cognitive Reflection Test (CRT) and Wonderlic Test, performance in Reading the Mind in the Eyes, and confidence. Some of the measures (risk and time preferences, for instance) are incentivized, while others are not. We will match the responses to the personnel data of the firm using firm's data (anonymized matching). This data includes age, gender, education, family status, tenure, portfolio of loans, quality of the portfolio, and bonus received in the last 12 months whih is a measure of productivity. We will train an AI algorithm to predict which employees perform best according to the rankings based on productivity, portfolio size, and the risk of the portfolio.

The next stage will consist in applying the algorithm to study its usefulness for hiring decisions. All candidates who apply to the firm will participate in the survey, excluding the incentivized measures . As is the case now, without our experimental treatment, all candidates will undergo an interview with the central office and one of the regional managers. At this point, we will be able to see the extent to which the pool of employees recommended by the managers and by the algorithm overlap. Then half of the candidates will be hired based on the existing procedure in the company, that is according to the managers' recommendation. The other half of the candidates will be hired based on the AI prediction of the similarity of the candidates to the best performing employees. We will evaluate the results based on the firm’s record of the performance of the new employees, including their sales, the risk of their portfolio, and whether the employee was fired for underperformance or left the company. The field experiment will run for six months, with the possibility of prolonging it, in case that the number of new hires is too small to draw any conclusions. We aim for at least 100 new hires in each treatment.

2021-12-01

2022-05-31

The primary outcome of the experiment is the performance of the new employees. We are interested in whether the performances of the new employees are different whether the new employees were hired using the existing hiring procedure of the firm or the AI algorithm recommendation.

For the first stage the main outcomes are performances of the algorithm given personally data only, personnel data and non-incentivized survey measures, and personnel data , non-incentivized and incentivized survey measures.


Portfolio size
Portfolio at risk 30 days or more
Portfolio free of risk
Sales

We are interested in outcomes in 6 and 12 months after starting the job.

We train algorithm on employees with tenure of 12 months or more. This is because the categorization by management is more informative, as well as their objective performance.

We will test the algorithm predictive power first of the employees with lower than 12 month of tenure in the moment. We will predict the category for them, and evaluate based on the performance after 6 and 12 month from start of the job.

We are primarily interested in the efficiency of hiring decisions using either the hiring procedure of the firm, or the recommendations of the AI algorithm that was trained to predict the quality of employees with the personnel data and the answers to the non-incentivized questions. We will judge the quality of the hiring decisions with respect to the employees’ portfolio, the portfolio at risk, the portfolio without delayed payments, the number of new loans, whether the employee qualified for a bonus, and whether the employee left the firm. We will take these measures six months and 12 months after the employees were hired.
We will also compare the recommendations of the algorithms with those of the managers with respect to the quality of their predictions regarding the employees when the algorithms use either (i) only personnel data, (ii) personnel data and the answers to the non-incentivized questions, and (iii) personnel data, answers to the non-incentivized questions, and the answers to the incentivized questions.
Finally, we plan to study the predictive power of the algorithms for the sample of employees whose current period of employment is less than 12 months. Their data are not used for training the algorithm. Instead, we will use this sample to predict the employees’ performance in six and 12 months after the start of their employment. We will also compare each of the outcomes of interest between two groups of current employees—those who would have been hired by the algorithms and those who would not have been hired.


To do

All candidates for a salesman position in the firm will, as is the case now (before our intervention), undergo an interview with the central office and one of the regional managers. All candidates will also participate in our survey.
Then, half of the candidates will be hired based on the existing procedure in the company, that is according to the managers' recommendation. The other half of the candidates will be hired based on the AI prediction of the similarity of the candidates to the best performing employees.

Randomization will be done by a computer, with every second candidate being in treatment.

Individual.

No

No clusters are planned.

At least 200 new employees. In order to train the AI algorithm, we will use the answer to the survey of the current employees of the firm (about 1000 employees).

At least 100 new employees in each treatment (hired via the existing procedure/ hired using the AI algorithm)