Employing the unemployed of Marienthal: Evaluation of a guaranteed job program

Last registered on February 08, 2021

Pre-Trial

Trial Information

General Information

Title
Employing the unemployed of Marienthal: Evaluation of a guaranteed job program
RCT ID
AEARCTR-0006706
Initial registration date
November 04, 2020
Last updated
February 08, 2021, 3:03 AM EST

Locations

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Primary Investigator

Affiliation
University of Oxford

Other Primary Investigator(s)

PI Affiliation
Oxford University

Additional Trial Information

Status
In development
Start date
2020-10-01
End date
2025-04-30
Secondary IDs
Abstract
This documents outlines the study design and statistical analysis of our evaluation of a guaranteed job program that will be piloted in the municipality of Gramatneusiedl in Austria, starting in October 2020.
Our study design is based on two approaches.
The first approach uses individual-level pairwise randomization of participants into waves.
The second approach constructs a synthetic control at the municipality level, to impute counterfactual outcomes for Gramatneusiedl.
We consider multiple different treatment contrasts to separate out direct effects of program participation, anticipation effects of future participation, and municipal level equilibrium effects.
External Link(s)

Registration Citation

Citation
Kasy, Maximilian and Lukas Lehner. 2021. "Employing the unemployed of Marienthal: Evaluation of a guaranteed job program." AEA RCT Registry. February 08. https://doi.org/10.1257/rct.6706-1.1
Experimental Details

Interventions

Intervention(s)
Starting in October 2020, the Public Employment Service for Lower Austria (Arbeitsmarktservice Niederoesterreich (AMS NOE) is piloting an intervention that aims to eradicate long-term unemployment and improve social, health and wellbeing outcomes for people in long-term unemployment, by bringing them back into employment. The intervention will provide a guaranteed job complemented by targeted counseling to support people in long-term unemployment.

The intervention will take place in one town in Lower Austria, Gramatneusiedl.
All residents who have been unemployed for over 9 months are eligible to participate.
The initial period for the project is set until 2024 and budgeted with EUR 7.4 Million for the full duration. The {AMS} calculates that the annual cost of the intervention is EUR 29,841.39 per participant.

Preparatory training

The program is implemented by the private service-provider itworks, which specializes in implementing active labour market programmes for the {AMS}.
{itworks} provides preparatory training for participants, and continues counseling and training after participants have taken up employment. The preparatory training phase is scheduled for a maximum of eight weeks, but durations may vary depending on individual conditions and progress.
Each participant receives a tailored curriculum according to their individual needs.
This may include individual and group counseling, skills development, supporting self-initiative, and assistance with health-related problems. Participants will continue to be encouraged to take up regular employment outside of the program if available.

Guaranteed jobs

After successful completion of the preparatory training phase, participants join the job guarantee program for up to 3 years. Participants are supported to find a job on the regular labor market, for which the {AMS} subsidizes wage costs at a 100\% rate for the first 3 months, and at 2/3 for the subsequent 9 months. Those that remain without job placement will receive an employment offer with a newly established social enterprise operated by {itworks}. All participants will be paid at least minimum wage. The social enterprise implements projects at the municipal and regional level. Tasks may include activities such as working in childcare, gardening, renovation, and carpentry depending on orders acquired by the enterprise. In addition, participants are supported to develop and propose their own ideas for projects of the social enterprise based on their expertise and local knowledge of community needs. Through its business activities, the enterprise is expected to generate revenues of around EUR 383,000 over the project duration.

A specific effort is made to create productive and meaningful employment that is adequate to the participants’ previous jobs in terms of skills and income. The jobs created are tailored to the needs of the recipients. Persons with limited work availability receive targeted job offers. A person only available to work part-time receives a corresponding part-time offer. A person who can carry out only a limited number of tasks (e.g. for health reasons) similarly receives a corresponding offer. Social workers and instructors continue to provide support to employees of the social enterprise as needed. Participants have access to occupational physicians. Those participants that feel ready to work for third-party employers receive targeted support and additional counseling to apply and find employment outside of the program.

Voluntary participation

Work conditionality is eased. Currently, under law Arbeitslosenversicherungsgesetz (AlVG \S 9), people in unemployment are assigned to labour market programs by the AMS. They have the obligation to participate and they have to accept an employment offer that conforms to their skillset. By contrast, within the job guarantee program only participation at the information event and the preparatory training phase is subject to this conditionality, while take-up of employment offered as part of the job guarantee is voluntary (i.e.. without sanctions in case a job offer is declined).
Intervention Start Date
2020-10-01
Intervention End Date
2024-04-30

Primary Outcomes

Primary Outcomes (end points)
Contrasts

In order to assess the impact of the guaranteed job program, we will consider three contrasts.
First, we compare participants in the two waves of the pairwise randomized study.
This comparison delivers credibly identified treatment effects.
It is restricted, however, to short term outcomes measured in early 2021, before the second wave of participants starts their jobs.
Furthermore, the control group might be impacted by the anticipation of future program receipt.

Second, we construct an additional control group from the long term unemployed who live in synthetic control municipalities, and who would have been eligible to participate had they been residents of Gramatneusiedl.
This comparison relies on the validity of the synthetic control.
It allows us, however, to estimate treatment effects which are not contaminated by anticipated program receipt and to compare outcomes over a longer period.

Third and lastly, we estimate municipal level treatment effects by comparing Gramatneusiedl to the synthetic control. This comparison includes residents who would not have been eligible to participate in the program, because they were not long-term unemployed.
This comparison again relies on the validity of the synthetic control. It allows us to estimate equilibrium effects and spillovers at the municipality level, which might for instance be driven by the crowd-out of jobs, or by consumer demand effects of those participating in the program.


Primary outcomes

We will estimate the effect of program participation on a range of economic and social outcomes. To assess the main objective of the intervention - to eradicate long-term unemployment - we will use labor market status as our primary outcome; for participants in the experimental study, and for other residents of Gramatneusiedl in the synthetic control study.
For the experimental study, labor market status will be defined as being in employment vs. not being in employment.
For the synthetic control study, we will estimate the treatment effect on the unemployment rate (unemployed as a share of the working age population), which we expect to exhibit a larger effect. For robustness, we will also estimate the effect on the employment rate.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
To assess the impact of treatment on secondary objectives, we will estimate the treatment effect on a range of outcomes, which are provided in a Table in the pre-analysis plan.
We will use multi-item scales to construct one composite index for each of the secondary outcomes, except for employment and income. We then group those outcomes in four thematic families.
We will register the full questionnaire and methodology for constructing the composite indices before running the first survey.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Our evaluation of the job guarantee program is based on two complementary study designs.
The first design uses pairwise randomization within pairs of participants matched using baseline covariates. Participants are assigned to one of two waves, where the second wave starts the program 4 months after the first one.
This allows us to estimate (short-term) effects of the program by comparing participants across the two waves, around 3-4 months from the start of the first wave.

The second design uses the synthetic control method.
We construct a synthetic control town for Gramatneusiedl, based on other towns in the province of Lower Austria.
The synthetic control town is a convex combination of similar towns.
This method allows us to estimate effects of the program at the town level, including potential spillovers on non-eligible residents.

Please refer to the pre-analysis plan for further details.
Experimental Design Details
Not available
Randomization Method
Pairwise randomization, and pre-registered synthetic control
Randomization Unit
Individuals and municipalities
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
-
Sample size: planned number of observations
62 program participants; 6 control municipalities
Sample size (or number of clusters) by treatment arms
-
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials

Documents

Document Name
Survey February 2021
Document Type
survey_instrument
Document Description
This document is a translation of our survey questionnaire, administered to program participants in February 2021.
File
Survey February 2021

MD5: 4a87b4759ef205e08fb8212e365e0059

SHA1: 9b2ae31b881a0dd1ef19836e8ba051a1872a4f62

Uploaded At: February 08, 2021

IRB

Institutional Review Boards (IRBs)

IRB Name
Departmental Research Ethics Committee at the Department of Economics, Oxford University
IRB Approval Date
2020-09-18
IRB Approval Number
N/A
Analysis Plan

Analysis Plan Documents

Employing the unemployed of Marienthal: Evaluation of a guaranteed job program

MD5: 4eb3c6c685b4e296bc6926a97608e12d

SHA1: d5018d774d5c74f990124581513072c41ed4612f

Uploaded At: November 04, 2020