Can a Website Bring Unemployment Down? Experimental Evidence from France

Last registered on March 28, 2022

Pre-Trial

Trial Information

General Information

Title
Can a Website Bring Unemployment Down? Experimental Evidence from France
RCT ID
AEARCTR-0009112
Initial registration date
March 27, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
March 28, 2022, 7:07 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
Harvard Business School

Other Primary Investigator(s)

PI Affiliation
MIT
PI Affiliation
CREST-ENSAE
PI Affiliation
CREST-ENSAE
PI Affiliation
Harvard University
PI Affiliation
Aix Marseille University

Additional Trial Information

Status
Completed
Start date
2017-04-01
End date
2019-02-01
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
We evaluate the impact of an online platform giving job seekers tips to improve their search and recommendations of new occupations and locations to target, based on their personal data and labor market data. Our experiment used an encouragement design and was conducted in collaboration with the French public employment agency. It includes 212,277 individuals. We find modest effects on search methods: the users of the platform adopt some of its tips and they are more likely to use resources provided by public employment services. However, following individual trajectories for 18 months after the intervention, we do not observe any impact on time spent looking for a job, search scope (occupational or geographical), or self-reported well-being. Most importantly, we do not find any effect on any employment outcome, whether in the short or medium run. We conclude that the enthusiasm around the potential for job-search assistance platforms to help reduce unemployment should be toned down.
External Link(s)

Registration Citation

Citation
Ben Dhia, Aicha et al. 2022. "Can a Website Bring Unemployment Down? Experimental Evidence from France." AEA RCT Registry. March 28. https://doi.org/10.1257/rct.9112-1.0
Experimental Details

Interventions

Intervention(s)
We evaluate the impact of "Bob Emploi,'' an online job-search assistance website established to reduce unemployment in France. Bob Emploi was launched in 2016 by Bayes Impact, a private non-profit organization. In designing the website, Bayes Impact benefited from a close collaboration with the French public employment agency, Pôle emploi, which records systematic information on each job seeker's periods of unemployment. Bob Emploi set out to analyze the rich data provided by Pôle emploi with state-of-the-art statistical analysis methods. This innovative partnership sets Bob Emploi apart from other private websites and increases its potential for impact. Bob Emploi endeavors to assist and motivate job seekers by providing personalized and data-informed advice on sectors and locations to target; offering step-by-step planning assistance; sending regular reminders and encouragement messages; and providing general tips, such as how to behave during a job interview.
Intervention (Hidden)
Using the rich data obtained from Pôle emploi, Bob Emploi seeks to help job seekers find a position through four different means.

First, Bob Emploi gives personalized advice based on user profile. When creating a profile on Bob Emploi, each user is asked to fill in their socio-demographic characteristics (gender, age, level of education, and municipality of residency), their desired occupation, and their target geographic area and salary. Based on this information, and leveraging the labor market information provided by Pôle emploi, the website presents the user with a diagnosis of employability in the form of a numerical score based on the labor demand for the target job, the competition for these jobs, and the quality of the match between the user’s profile and the job requirements. The idea underlying this feature, which is supported by a growing literature, is that job seekers tend to have inaccurate beliefs about the situation of the labor market and to be overoptimistic about their return to employment, leading to suboptimal search strategy. The diagnosis of employability was expected to lead job seekers to update their beliefs about their chances to find a job and to adjust their search accordingly.

In addition to the employability diagnosis, the website gives users recommendations to undertake some actions, ranked by order of importance. For instance, Bob Emploi's algorithm may advise users targeting a tight market to explore related careers with higher demand. If the data indicate that the market for that occupation is less tight in neighboring municipalities, users may be advised to widen the geographic perimeter of their search.

To customize its recommendations, Bob Emploi also asks users to indicate how many job opportunities they identified over the past weeks, how many applications they sent, and how many interviews and job offers they received. The website uses the ratio between any two of these indicators to customize pieces of advice focusing on steps to overcome the most important obstacles to finding a job. For example, if the number of interviews received by a user is disproportionately low relative to their number of applications, Bob Emploi may advise them to edit their résumé or to improve other parts of their application file.

Second, Bob Emploi provides general advice, such as how to behave in a job interview. Recommendations are frequently completed by referral to external resources, most often also online. Both the general and personalized advice provided by the website may help job seekers overcome informational barriers and increase their chances to find a job.

Third, beyond information provision, Bob Emploi also provides step-by-step planning assistance to its users. The goal of these action plans is to help job seekers organize their job search efficiently and prioritize the tasks they need to complete. Encouraging individuals to follow such plans has proven effective in many contexts, including job search.

Fourth, Bob Emploi sends regular reminders and messages of encouragement to job seekers as a motivation technique. Users can control how often they receive these emails. This feature is motivated by recent research in behavioral economics showing that present biased time preferences and biased beliefs can otherwise lead job seekers to procrastinate their search effort. In addition, the website also generally uses encouraging language and an optimistic tone, emphasizing the strengths of each job seeker’s profile. The format of the interface is simple and inviting, in contrast with the often austere and complex appearances of other administrative resources. These features may be particularly relevant given the decline in search effort and motivation over the unemployment spell and the increase in sadness documented by previous work.
Intervention Start Date
2017-04-01
Intervention End Date
2017-12-01

Primary Outcomes

Primary Outcomes (end points)
Employment outcomes, including: experiencing any employment episode following the intervention, obtaining a stable job (a contract of more than six months), total number of days in unemployment, and total amount of unemployment bene fits. Up to 18 months after the intervention.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Job-search effort, scope, and expectations, including: job seekers' reemployment expectations and how realistic they proved to be; job search effort (number of hours respondents declare spending on their job search, sending at least one unsolicited application per week, and number of online applications sent through Pôle emploi's job board); and scope of the search (respondents who report applying to jobs located outside the municipality where they live; and number of applications sent by job seekers through Pôle emploi for jobs outside their municipality, outside their target sector, and below their reservation wage).

Job-search methods, including: using one's personal and professional networks when looking for a job; adapting one's CV and cover letters to job listings; following up with recruiting firms; number of Pôle emploi websites and of private websites used by job seekers; and use of public job-search assistance offline services (Pôle emploi workshops and meeting with Pôle emploi caseworkers).

Well-being and life balance: self-declared overall well-being; feeling motivated and supported during one's job search; and participation in non-job-related activities.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Our experiment relied on an encouragement design. Job seekers in the treatment group were invited to an informational session at their local agency. Such sessions are frequently organized by Pôle emploi caseworkers to present free tools and programs to job seekers. In the present case, each meeting was hosted by one or two Pôle emploi caseworkers and lasted approximately 90 minutes. It began by introducing the challenges associated with a job search and the value of widening one’s search methods, notably through personal and professional networking. The caseworkers then briefly presented the "Emploi Store," the Pôle emploi platform that lists job-search services, including Bob Emploi. The session then turned to the presentation of Bob Emploi using screenshots and, sometimes, a real-time simulation. Participating local agencies organized these information meetings at a rate of approximately one per week from April 20 to July 31, 2017. Job seekers in the treatment group were invited to these sessions based on the month in which they were drawn. Specifically, individuals from the first wave, drawn on April 1, were invited to attend meetings taking place between late April and late May. Individuals from the second wave, drawn on May 1, and third wave, drawn on June 1, were invited to attend meetings taking place between late May and late June, and between late June and late July, respectively.

To maximize take-up, we completed the informational sessions by sending three emails encouraging all job seekers in the treatment group to create an account on Bob Emploi. The emails were sent on July 7 and 26 (for individuals drawn in the first two months and in the third month, respectively), September 28, and November 13. The exact text of the emails differed slightly for recipients who had attended an informational session and those who had not.
Experimental Design Details
Randomization Method
Randomization done in office by a computer
Randomization Unit
The randomization was done at the level of the individual job seeker. There was a unique level of randomization.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
212,277 individuals.
Sample size: planned number of observations
212,277 individuals.
Sample size (or number of clusters) by treatment arms
92,752 individuals in the control group, and 119,525 individuals in the treatment group.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Committee on the Use of Human Subjects - Harvard University
IRB Approval Date
2017-08-25
IRB Approval Number
IRB17-0735

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials