NEW UPDATE: Completed trials may now upload and register supplementary documents (e.g. null results reports, populated pre-analysis plans, or post-trial results reports) in the Post Trial section under Reports, Papers, & Other Materials.
Upgrading Management Technology in Colombia
Initial registration date
October 17, 2014
June 29, 2019 7:35 PM EDT
Other Primary Investigator(s)
Additional Trial Information
The key objectives of this research are i) to test whether a scalable and cost-effective approach to closing the gap in management practices between developing and more developed countries can be implemented; and ii) to measure the impact of closing this management gap on firm productivity and profitability. It will do this by means of a randomized experiment in Colombia, which will allocate firms into three groups: a control group who just receive a diagnostic on their management practices, an individualized treatment group who receive one-on-one consulting to improve 5 different processes: Logistics, Human Resources, Finance, Marketing and Sales, and Production; and a treatment group in which management improvements will be implemented in a small group setting, analogous to agricultural extension, with firms learning from one another. The interventions take place over a 6 month period and are intended to improve growth, productivity, and the scope for innovation in these businesses.
The first phase is common to all firms, and includes a detailed diagnostic of 5 different areas (Logistics, Human Resources, Finance, Marketing and Sales, and Production) implemented by a team of 6 consultants, 5 of them specialists in each specific area analyzed and one team leader coordinating the process. This diagnostic phase lasts 2 full-time weeks and is completed by a report that analyzes both managerial practices for each one of these areas and key performance indicators associated with each one of the areas analyzed
There are two interventions being piloted, which will be received by different treatment groups:
1) Individual consulting services support: Firms assigned to this group will receive individual support of a team of 5 consultants for 5 different processes: Logistic, Human Resources, Finance, Marketing and Sales, and Production during a period of 4-6 months to implement a set of improved managerial practices prioritized for their potential impact on growth and productivity of the businesses based on the diagnostic performed in phase one. 2) Group consulting services support: Firms assigned to this group will receive group support of a team of 6 consultants for 4-6 months to implement a set of improved managerial practices prioritized for their potential impact on growth and productivity of the businesses based on the diagnostic performed in phase one. The difference with the individual support is that groups of 5 or 6 firms located in the same region receive a group intervention. Leaders from the firms in a group sign an agreement to work together and help each other improve. Training in particular topic areas first occurs in a group class setting, with each firm sending the people in charge of that production process along. This is then followed up by in-person demonstrations, with group members being able to learn from each other, so that an improvement in practice in one firm can be observed and mimicked by others in the group. Groups are formed so that members are not direct competitors to one another, but instead are producing complementary products with similar management problems.
Intervention Start Date
Intervention End Date
Primary Outcomes (end points)
We will examine impacts at three key levels of the causal chain: first whether the interventions lead to improvements in management practices; second, whether these improvements in management practices lead to changes in factory operations (such as changes in machine downtime rates, electricity usage, quality defect rates, and labor hours); and thirdly, whether these changes ultimately lead to higher production, improved productivity, greater sales, and higher profitability.
The exact measures will depend on data availability, and particularly on what measures can be reliably compared across different groups without being affected by the intervention. We plan in particular to focus on four key outcomes:
* Management practices
* Production levels
* Employment levels
Primary Outcomes (explanation)
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Following the diagnostic, we dropped firms with fewer than 10 workers, to leave a sample of 159 firms for the experiment. We then formed matched triplets of firms, choosing this grouping in a way to minimize the Mahalanobis distance between firms in a triplet in terms of their geographic location, size, labor productivity, and management practices. This took place in November 2013. Then within each triplet, firms were randomly allocated to a control group and two treatment groups. Each group consists of 53 firms.
Experimental Design Details
Randomization within triplets was done in a semi-public setting, with representatives of government and industry observing.
Randomization at the level of the individual firm.
Was the treatment clustered?
Sample size: planned number of clusters
Sample size: planned number of observations
Sample size (or number of clusters) by treatment arms
53 firms to control, 53 to individual consulting treatment, 53 to group consulting treatment
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
The proposed study details with a single industry, and collects very detailed data at a monthly frequency. These two features help to ensure we have sufficient power to detect meaningful treatment effects of interest. In particular, we have more than 5 times the number of firms in each treatment group as was the case in the Bloom et al. (2013) study, which was still able to detect significant impacts.
First consider our power to detect a change in management practices. During the diagnostic phase, the index collected by CNP of all management practices has a mean of 0.46 (out of 1) and standard deviation of 0.10. Even without any controls we would still have 80 percent power to detect a change of approximately 5.5 percentage points in the proportion of desirable management practices that are implemented – and power to detect even smaller impacts once additional controls are used. Given the intensity of the intervention and the evidence from India, we expect at least this much of a change in management practices.
Second, consider our power to detect an increase in productivity. Labor productivity at baseline has a mean of 30.76 and standard deviation of 18.29. Labor productivity here is annual sales (in millions of Colombian pesos) per employee, so 30.76 indicates sales per worker of approx..US$14,600. We will have this data at a monthly level, with 12 months pre-intervention, and 12 months post-intervention. Assuming an autocorrelation of 0.5 then gives us power of 95.1 percent to detect a 10% increase in productivity, again without taking into account power gains from the random assignment within matched triplets. This allows us to detect an effect smaller than the size measured in Bloom et al. (2013), whilst allowing room for attrition to potentially reduce the sample slightly. Taken together we view these calculations as indicating we have sufficient power to detect meaningful treatment effects of interest.
INSTITUTIONAL REVIEW BOARDS (IRBs)
Post Trial Information
Is the intervention completed?
Intervention Completion Date
April 30, 2016, 12:00 AM +00:00
Is data collection complete?
Data Collection Completion Date
December 31, 2018, 12:00 AM +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
Was attrition correlated with treatment status?
Final Sample Size: Total Number of Observations
159 firms, up to 6 years of monthly data depending on outcome
Final Sample Size (or Number of Clusters) by Treatment Arms
53 firms control, 53 firms individual consulting, 53 firms group consulting
Reports, Papers & Other Materials
Differences in management quality are an important contributor to productivity differences across countries. A key question is how to best improve poor management in developing countries. We test two different approaches to improving management in Colombian auto parts firms. The first uses intensive and expensive one-on-one consulting, while the second draws on agricultural extension approaches to provide consulting to small groups of firms at approximately one-third the cost of the individual approach. Both approaches lead to improvements in management practices of a similar magnitude (8-10 percentage points), so that the new group-based approach dominates on a cost-benefit basis. Moreover, we find some evidence that the group-based intervention led to increases in firm size over the next three years, while the impacts on firm outcomes are smaller and statistically insignificant for the individual consulting. The results point to the potential of group-based approaches as a pathway to scaling up management improvements.
Iacovone, Leonardo, William Maloney and David McKenzie (2019) "Improving Management with Individual and Group-Based Consulting: Results from a Randomized Experiment in Colombia", World Bank Policy Research Working Paper no. 8854
REPORTS & OTHER MATERIALS