Supervisor Screening and Training

Last registered on July 28, 2023
View Trial History

Pre-Trial

Trial Information

General Information

Title
Supervisor Screening and Training
RCT ID
AEARCTR-0011288
Initial registration date
July 28, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
July 28, 2023, 2:09 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Name
Anant Nyshadham
Affiliation
University of Michigan
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Achyuta Adhvaryu
PI Affiliation
University of San Diego
Contact Investigator
PI Name
Jorge Tamayo
PI Affiliation
Harvard University
Contact Investigator
PI Name
Jean-François Gauthier
PI Affiliation
HEC Montréal
Contact Investigator
PI Name
Emir Murathanoglu
PI Affiliation
University of Michigan
Contact Investigator

Additional Trial Information

Status
In development
Start date
2023-08-31
End date
2024-12-31
Keywords
Firms & Productivity, Labor
Additional Keywords
managerial skills, soft skills, training, absenteeism, attrition, well-being
JEL code(s)
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
The experiment follows Adhvaryu et al. (2023) which demonstrates a critical role of certain managerial dimensions in determining firm productivity, and shows that intervening to improve management practices can generate meaningful impacts. We find that several key dimensions of managerial quality, such as attention, autonomy and locus of control are important for productivity but are not appropriately priced into market pay. Firms could substantially improve productivity at a low cost via psychometric measurement and screening of potential hires, and by providing training to improve managerial deficiencies. Adhvaryu et al. (2023a) follows up on this insight by evaluating a managerial training program based on the learnings from Adhvaryu et al. (2023b) identifying which managerial skills and practices contribute most to the productivity of production lines of India’s largest garment producer, Shahi Exports. The resulting program, called Supervisors Transform Into Change Holders (STITCH), was evaluated via a randomized controlled trial across hundreds of supervisors at Shahi Exports (our partner organization). The training achieved large and persistent impacts: on average efficiency (produced quantity/target quantity) of lines managed by trained line supervisors increased by 7.3% during training and 5.8% over the six months after the training. The proposed study empirically tests a screening and training technology that we have developed to hire and/or promote (garment) production line supervisors based on their psychometric traits and/or managerial practices to improve firm productivity and worker wellbeing. The trial will take place in 62 garment manufacturing units in India among upper- level managers and production line supervisors overseeing more than 70,000 workers of Shahi Exports. We will randomly select roughly a quarter of the units to get the screening technology, a quarter to get the training technology, and a quarter to get both while the remaining units will serve as the control group.
External Link(s)

Registration Citation

Citation
Adhvaryu, Achyuta et al. 2023. "Supervisor Screening and Training." AEA RCT Registry. July 28. https://doi.org/10.1257/rct.11288-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
This study will be conducted in 62 factories of Shahi Exports Pvt Ltd. with a self-administered screening tool and training tool consisting of managerial and leadership skills. We will be doing 3 follow ups.

The factories in the first treatment arm will receive only the screening and training tools (T1)
The factories in the second treatment arm will receive the screening and training tools and the technical tool (T2)
The factories in the third treatment arm will receive only the technical tool (T3)
The control units will not receive any of the tools apart from using the screening tool. We will use the survey app embedded in the survey tool to measure characteristics, traits, and practices at baseline and endline for the control group. The information from these surveys will not be shared with HR as they would in the other treatment arms.


Introduction to the tools:

1. Screening tool: The screening tool contains a series of psychometric questions, questions about management skills, leadership skills and soft skills. The screening tool is app based and self-administered through tablets. The results from this tool will be recorded on an online portal that can be accessed by the HR manager in-charge of hiring. They will see a display of results based on the screening score. At the end of the study, we will also see to what extent the HR department has adhered to our recommendations. This will be a secondary result. At the backend of the screening tool, there is a proprietary formula that scores each section and then weights those scores to decide which training the user would need to undergo. The overall time to undertake the screening tool is 60-90 minutes. This must be done at a stretch.

2. Training Tool: The training tool is a collection of modules and lessons mapped to the screening tool. These lessons are based on leadership, stress management, conflict resolution and other topics that would help make participants into better supervisors. Based on their screening result, each supervisor would have a unique learning path carved out for them in the training tool. Once they have completed their targeted training modules, they will have access to the entire tool including modules that they weren’t recommended. They can do these at their own pace. The training tool has a gamified interface that rewards participants in the form of coins and stars based on their progress in the app. The training app, along with being self-administered, is self-paced. The participants can take their time to complete the modules.

3. Technical tool: The technical tool is a digitized screening test internally developed by Shahi based on the technical assessment done in the existing hiring framework. It has been developed with the objective to assess nine key competencies required for a line supervisor. The tool aims to establish unbiased and standardized hiring and promotion processes across all units in Shahi. This tool is being used for the hiring/promotion process in Shahi currently. The test is divided into two primary sections: Selection and Promotion. The test consists of 50 multiple choice questions broadly covering themes such as professional skill, managerial skill, technical knowledge, and organizational context. This standardized test will administer a set of questions to prospective supervisors, calculate their scores based on their responses, and rank them by their predictive productivity at a supervisory role.
The trial allows us to test for both the effect of the different tools and their complementarity.
Intervention Start Date
2023-10-01
Intervention End Date
2024-10-01

Primary Outcomes

Primary Outcomes (end points)
1. Managerial Composition
We will use data from the baseline and the endline survey to create measures of the managerial skills and traits of supervisors.

2. Productivity
Productivity is calculated at the production line level, as the ratio of actual pieces of garment produced to the target number of pieces that could be produced under optimal conditions in a given day. This number is an exogenous industry standard.

3. Absenteeism
A worker is defined as absent if they were not present for work on a given working day. We will have daily absenteeism data for each worker and supervisor.

4. Worker Satisfaction
All survey outcome variables, like worker satisfaction, are indices generated by normalizing each component of the index to mean zero and standard deviation one, and then averaging across all components.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
For this project, we are going to do an RCT in 62 factories of Shahi Exports Pvt Ltd. with a self-administered screening tool and training tool consisting of managerial and leadership skills. We will be doing 3 follow ups. Our unit of treatment is the entire factory. The unit of analysis differs between different outcomes. For line-level productivity outcomes, the unit of analysis is the production line. For retention, absenteeism, and survey outcomes, the unit of analysis is the individual worker/supervisor.

Experimental Design Details
Surveys will happen for 3 different groups - supervisors, workers and floor-in-charges. Each group will be surveyed 3 times. For supervisors, this will mean interacting with the same supervisor 3 times. For workers, this will mean interacting with each worker in the sample less than or equal to 3 times at random.

Supervisor Surveys: These will be done using an app designed by GBL to screen supervisors for managerial skills, soft skills and leadership skills. These will be self-administered.
FIC Surveys: These will be done by an enumerator using a SurveyCTO form
Worker Surveys: These will be done by an enumerator using a SurveyCTO form

Experimental Design Details
Not available
Randomization Method
We pre-randomize by computer using Stata’s pseudo-random number generator.
Randomization Unit
Unit of randomization is the factory
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
62
Sample size: planned number of observations
1200 supervisors and 4000 workers
Sample size (or number of clusters) by treatment arms
16 factories in screening and training tools arm, 16 factories in the screening, training and technical tool arm, 15 factories in the technical tool arm, 15 factories control

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
GBL
IRB Approval Date
2021-09-13
IRB Approval Number
GBL092021
IRB Name
University of Michigan
IRB Approval Date
2022-08-12
IRB Approval Number
HUM00217038
Analysis Plan