The Effects of a Supported Employment Program for Disability Insurance Recipients with Mental Health Conditions: A Randomized Controlled Trial Evaluation

Previous studies have shown that Disability Insurance (DI) beneficiaries may have substantial remaining capacity to work (Bound, 1989; French & Song, 2014; Maestas, Mullen, & Strand, 2013). This motivated social security administrations to implement Active Labour Market Policies that encourage DI beneficiaries to return to work.

In this study, we evaluate the effects of an innovative employment support program aimed at DI beneficiaries with a mental condition. The program is based on the “Individual Placement and Support” (IPS) model (treatment condition). We compare this new program with traditional rehabilitation services based on vocational training (control condition).

Vocational training has been traditionally favoured to help DI beneficiaries to return to work. It relies on the assumption that DI recipients should first rebuild a working capacity and afterwards reintegrate the job market. This approach is often called “train-then-place”. In Belgium, it is managed by the public employment centres since 2012. The program includes three phases supervised by employment specialists: 1) assessment of an individual’s rehabilitation needs, 2) vocational training, and finally 3) job search counselling. The whole process might be quite lengthy (in Belgium, it spans on average a period of 12 months, with trainings of about 6 months).

The IPS model of supported employment proposes to inverse this traditional "train-then-place" approach with a "place-then-train" approach that should favour quick insertion into the labour market and afterwards, if necessary, on-the-job training. The program relies on job coaches who help DI beneficiaries to find and retain a job that fits their mental health condition.The IPS model is based on several principles (R. Drake, Bond, & Becker, 2012), including:

- “rapid job search” (i.e. participants should quickly have face-to-face contacts with potential employers);
- “competitive employment” (i.e. no sheltered or voluntary work);
- “follow-along support” (i.e. continuous coaching, even after a job has been found);
- “integrated services” (i.e. job coaches work closely with mental healthcare specialists).

2018-03-26

2022-12-31

All participants complete a baseline survey. Then we will conduct follow-up surveys every six months until the end of the intervention.

Our main outcomes of interest are the following:

- Disability insurance status (binary) and benefits (euros);
- Job search efforts (# hours per week, different channels, # interviews);
- Labour force participation (binary), time to first job, salary (euros), # weeks worked, # hours worked per week, sheltered work (binary);
- Unemployment status (binary) and benefits (euros).

Our secondary outcomes of interest are the following:

- Vocational training (attendance, duration, completion);
- Job quality (type of contract, bonus);
- Over-qualification (match between diploma and ISCO level of occupation);
- Management role (binary);
- Self-esteem (10 items - Rosenberg (1965));
- General Self-Efficacy Scale (10 items - Schwarzer and Jerusalem (1995));
- Health related quality of life (SF12 - Ware, Kosinski, and Keller (1996));
- # sick days, # doctor and hospital visits;
- Household labour supply (binary and # of hours per week) and earnings (euros).

Doctors of the healthcare funds select participants that (1) suffer from moderate-to-severe mental illness and (2) are willing to reintegrate the labour market. After they have given their informed consent, their file is sent to the disability insurance agency. Relevant information is then transmitted to the researchers: individual characteristics (gender and work experience) used for the stratified randomization, as well as names and contact details for the follow-up surveys.

The researchers allocate participants to the IPS or control group using the randomization list of the correct stratum. The result of the allocation is communicated to the disability insurance agency in a password-protected file sent by email. Participants are then invited to a job centre of the region where they live (Job centres in Belgium are a regional competency, supervised in Flanders by the VDAB, in Brussels by Actiris and in Wallonia by the Forem).

Three actors will be in contact with the participants during all the study period: healthcare fund doctors, job centres’ workers and IPS job coaches (only for the treatment group). The allocation sequence will be concealed from them, as well as from the disability insurance agency.

However, blinding the professionals for the randomization result is impossible. Indeed, the IPS program relies on close communication between health and employment specialists. Therefore, they will all be aware of the allocation of a participant to the intervention group. Doctors of the healthcare fund will also know about the control group allocation, as they need to sign a specific document to start the traditional vocational rehabilitation and send it to the disability insurance agency.

On the other hand, participants will ignore in which group they have been allocated (treatment or control). Professionals are instructed to communicate only about the program chosen for the participant and never disclose information about the other program.

The research team is responsible for the randomization and will therefore not be blinded. However, they will never enter directly in contact with the participants who answer the surveys at home (online or in paper form) or in their job centre (online).

Randomization is done in office by computer. The randomization lists are created using the software Stata and the user-written command “ralloc” from Ryan (1998). The command provides a sequence of treatments randomly permuted in blocks of varying size (treatments are balanced within blocks).

Allocation is clustered by region (with varying sample size) and stratified with two individual variables (gender and work experience). The number of strata is calculated “as the product, over all stratification variables, of the levels in each stratification variable" (Ryan, 1998), in our case 2 genders x 3 work experience categories, that is 6 strata.

Randomization takes place at the individual level. As participants will enter the study over several months, we need a predefined allocation sequence in the form of computer-generated randomization lists. We are using a block randomization scheme with permuted blocks of varying sizes.

We are using stratification on two variables that are important confounding factors according to previous studies: gender (Wewiorski & Fabian, 2004) and work experience in previous years (The exact question is: “Have you been working in a paid job in the last two years?” The answer is binary (yes/no) but a third category will be added for the cases when the information is missing) (R. E. Drake, McHugo, Becker, Anthony, & Clark, 1996).

Yes

Participants are recruited in the three regions of Belgium. Each region is a separate cluster. The most populated region is Flanders, followed by Wallonia. The capital, Brussels, and its neighbouring cities form the last region. The region of Wallonia has been separated in two to account for the fact that two different partners are responsible for the implementation of the IPS program in this region. In total, we therefore have four clusters (i.e. Flanders, Wallonia 1, Wallonia 2, Brussels).

We plan to recruit between 500 and 1000 participants. The final sample will depend on the flows of patients that are received by the health fund doctors and are willing to re-enter the labour market. Based on our power calculation analysis (next section) and the previous studies (R. Drake et al., 2012), we estimate that a sample size of 500 participants would be sufficient to detect significant differences for the primary outcomes (especially labour market participation). However, if the flow of participants is sufficient, we would like to recruit up to 1000 participants to allow for observing heterogeneous treatment effects. We are especially interested in studying treatment effects heterogeneity based on local labour market conditions and household characteristics.

The sample size is equally divided between treatment arms.

Our power calculation analysis is based only partially on previous studies of the IPS model. Indeed, the sample sizes and effect magnitudes reported in previous works are so heterogeneous, especially between the United-States and Europe, that there are of little help. Among the 23 studies that have been inventoried by (R. Drake et al., 2012), the sample size range from 41 to 312 participants. As for the effect size, it varies from 11 to almost 40 percentage points with very heterogeneous control group conditions (R. Drake et al., 2012). Our main outcomes of interest are binary (e.g. employment or disability status). We have chosen to run the power calculation with the maximum standard deviation for a binary outcome of 0.5. In the pre-anaysis plan, we provide a graph with the minimum detectable effect for several sample sizes. With 500 participants, we can detect an effect of the treatment of 12.6 percentage points (with alpha 0.05 and power 0.8). This would be the second worst effect of all the studies of the IPS model published so far. With 1000 participants, we can detect an effect of the treatment of 8.9 percentage points. Our main results will use a sample size within this range.

Tojerow & Fontenay - Pre-Analysis Plan