Testing new technologies for skill assessment and matching on expectations, upskilling and reallocation (pilot)

Last registered on October 15, 2021
View Trial History

Pre-Trial

Trial Information

General Information

Title
Testing new technologies for skill assessment and matching on expectations, upskilling and reallocation (pilot)
RCT ID
AEARCTR-0008363
Initial registration date
October 11, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
October 15, 2021, 11:00 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
Italy
Region
Lombardy

Primary Investigator

Name
Alexia Delfino
Affiliation
Bocconi University
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Raffaella Sadun
PI Affiliation
Harvard Business School
Contact Investigator

Additional Trial Information

Status
In development
Start date
2021-10-12
End date
2022-03-01
Keywords
Behavior, Firms & Productivity, Labor
Additional Keywords
active labor market policies, job centres, case workers' efficiency
JEL code(s)
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
We study the effects of an innovative tool for workers’ skills self-assessment and skills matching on individuals’ retraining choices, job search strategies and employment outcomes. Unemployed individuals who contact job centres to start their paperwork qualify as potential participants. Invited unemployed who agree to take part into the study will be randomised into receiving an invitation for our treatment, that is registering to an online platform that provides them with highly individualised information on their skills and skill-match within different occupations of their interest. The remaining individuals will be invited to register to a placebo website that only supplies aggregate information on a set of occupations. We expect those in the treatment group to increase their awareness on their current skill level, as well as align their expectations with the current labor market trends. Furthermore, we expect to find a positive effect on the likelihood that individuals retrain and search for occupations suited to their skills, even if in a sector/occupational category different from what they did in the past. This registration is for a pilot study, which we plan to then scale up. The pilot will happen in two job centres. In one of them, we will also randomise participants into having a phone conversation with a case worker, to measure the extent to which the actual interaction with caseworkers has an effect beyond that of the technology. By embedding our experiment within the organisational processes of the employment center, we will also test the effect of the technology on the job center organisational efficiency. On average, we expect treated individuals randomly selected for the call to achieve (weakly) better employment outcomes than control individuals selected for the call: during the phone conversation, case workers who can take advantage of the individualised information produced by the treatment platform on their users should provide more effective help and be a complement of the technology. In the other job centre, all the participants will have a conversation with a case worker.
External Link(s)

Registration Citation

Citation
Delfino, Alexia and Raffaella Sadun. 2021. "Testing new technologies for skill assessment and matching on expectations, upskilling and reallocation (pilot)." AEA RCT Registry. October 15. https://doi.org/10.1257/rct.8363-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
Our first treatment is an online platform that provides participants with highly individualised information on their skills and skill-match with different occupations. In one of the two job centres, individuals across treatment and control groups are also randomised into receiving a second intervention, which consists in having a phone conversation with their assigned case worker.
Intervention Start Date
2021-10-12
Intervention End Date
2021-12-12

Primary Outcomes

Primary Outcomes (end points)
User level (i.e. the unemployed): job finding rate, job search breadth, job search intensity (time + applications), beliefs updating on occupability and skill fit in different occupations, confidence in own skills, willingness to change occupation/sector compared to past employment, actual reallocation across occupations, training and type of training

Case worker level: job satisfaction, share of users remembered, perceived social impact and social worth

Case worker-unemployed level: beliefs on user's employability, opinion on effectiveness of meeting, topics discussed
Primary Outcomes (explanation)
TBC before analysis starts

Secondary Outcomes

Secondary Outcomes (end points)
Users: flexibiliy in commuting, aspirations in terms of salary/level of responsibility, overall life satisfaction, concerns on job finding
Case workers: meetings take up
Secondary Outcomes (explanation)
TBC before analysis starts

Experimental Design

Experimental Design
Our main hypotheses are that (1) providing unemployed individuals with highly individualised information on their skills and skill-match with different occupations should make them more likely to retrain and look for jobs that are more aligned to their skills, and that (2) providing the same information to job center case workers should allow them to be more effective in helping their users, thereby improving their employment outcomes.
This registration is for a pilot of the main study. The pilot will be conducted in two job centres. Unemployed individuals who contact the two job centres to start their paperwork will be sent an invitation to participate in our project. If they give their consent, they will be randomised into either receiving an invitation for the treatment website or the placebo website.
In one of the partner job centres, individuals across treatment and control groups will be further randomised into either receiving a phone call by their case worker to discuss their case or not. In the second job centre, all the individuals will be sent to have a meeting (online or in presence) with a case worker.
Baseline data on participants will be partly provided by admin records of each job centre and also collected via an online survey participants will fill in when they accept to participate in the project. A follow up survey will be sent to participants 2 to 3 weeks after their registration to the project. Depending on the number of case workers involved in the project, we will also send them a baseline survey before the beginning of the pilot. Each case worker will also complete an online survey after each call or meeting with the users. A follow up survey will be sent to all case workers.
Experimental Design Details
Randomization Method
Randomization into the treatment or placebo website is carried out on a rolling basis by the digital system that handles the project subscriptions, and it is stratified by gender, age, and job center. In the job center where calls are randomised, the additional randomisation into the call with the case worker is done by the researchers on a weekly basis using Stata, and stratifying for the same variables and for treatment status.
Randomization Unit
Individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
N/A
Sample size: planned number of observations
Employment center 1 (based on 2021 monthly data): - expected number of people invited in a month is 350 individuals - expected participation rate is 30%, hence 105 individuals will be randomised into treatment or control website Employment center 2 (based on 2021 monthly data): - expected number of people invited in a month is 320 individuals - expected participation rate is 30%, hence 96 individuals will be randomised into treatment or control website However, one of the purposes of the pilot is to gather information on the take-up rate (which we can only imprecisely guess right now), in order to do power analysis for the main study.
Sample size (or number of clusters) by treatment arms
- Employment center 1: according to an expected flow of 105 participants in one month, we will assign 50% of them to the placebo platform and 50% of them to the treatment platform (thus approximately 52 in each treatment). Within each treatment, we will assign 50% to the call with a case worker and 50% to no call (thus approximately 25 in the cell call+treatment, 25 in the cell call+ placebo, 25 in the cell no call+treatment, 25 in the cell no call+ placebo).
- Employment center 2: according to an expected flow of 96 participants in one month, we will assign 50% of them to the placebo platform and 50% of them to the treatment platform (thus approximately 48 in each treatment).
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
MDE=0.25 SD with full-compliance, significance level of 0.05, a sample size of 200 people, one baseline and two post-intervention surveys (with correlations between follow-ups of 0.5)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
Harvard University-Area Committee on the Use of Human Subjects
IRB Approval Date
2021-08-12
IRB Approval Number
Protocol #: IRB21-0871
IRB Name
Bocconi Research Ethics Committee
IRB Approval Date
2021-10-08
IRB Approval Number
Code: FA000259.01.02
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials