A Scientific Approach to SME Productivity

Last registered on November 01, 2023

Pre-Trial

Trial Information

General Information

Title
A Scientific Approach to SME Productivity
RCT ID
AEARCTR-0003875
Initial registration date
February 12, 2019

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
February 12, 2019, 4:57 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
November 01, 2023, 6:54 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Primary Investigator

Affiliation
ICRIOS, Bocconi University

Other Primary Investigator(s)

PI Affiliation
ICRIOS-Bocconi University
PI Affiliation
City, University of London
PI Affiliation
Oxford University
PI Affiliation
ICRIOS-Bocconi University

Additional Trial Information

Status
Completed
Start date
2018-12-01
End date
2019-11-30
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
SMEs are a vital part of the economy of almost all Countries (including the UK). However, their contribution to wealth creation, GDP and employment is lower than expected since the share of high-productivity SMEs remains substantially low (especially in the UK). The UK Government has recognised that SME productivity is a concern and has worked since 2011 on an action plan to improve it. Recent evidence shows that a key barrier to SMEs development is the shortage of programmes that educate and support businesses. Other studies also suggest that the lack of systematic managerial approaches and evidence based decision-making also hinder the growth and productivity of SMEs.
It is therefore important to ascertain what practices might improve SMEs’ decision-making processes - and, therefore, result in higher productivity. Addressing this research question would allow to better focus national and local government policies supporting SMEs, and more specifically to offer management training programs with proven ability to enhance small business owners and managers decision making. This project aims to address this question by testing the effect on SMEs’ productivity and growth of a set of novel business practices labelled ‘a scientific approach to decision making’. Extant studies converge in suggesting that SMEs, and especially microbusinesses, need to improve their ability to make rigorous strategic decisions.
This suggests that there is considerable room for improvement in the way they are managed and that there is a clear need to have them learn effective decision making tools.
The project will focus on micro-businesses, which represent the lion share of all businesses in most Countries (including UK), but account for only a small fraction of turnover. This suggests that there is considerable room for improvement in the way they are managed. Moreover, identifying solutions to improve these firms’ productivity could lead to particularly impactful results and a substantial return on investment. A focus on micro-businesses is also consistent with our experimental research design, with the intervention targeted to business owners or managers whom, in microbusinesses make most decisions (and especially strategic ones) with direct impact on performance.
More specifically we want to know if firms adopting a scientific approach to decision making are more productive than non adopters. Using an experimental research design we will compare two samples of firms, similar -on average- on all covariates. We will intervene on the two groups offering management training and then test if the pre/post-intervention productivity increase of firms treated with the scientific approach is larger than the pre/post productivity increase in the control group (which gets standard management training).

External Link(s)

Registration Citation

Citation
Camuffo, Arnaldo et al. 2023. "A Scientific Approach to SME Productivity." AEA RCT Registry. November 01. https://doi.org/10.1257/rct.3875-1.3
Former Citation
Camuffo, Arnaldo et al. 2023. "A Scientific Approach to SME Productivity." AEA RCT Registry. November 01. https://www.socialscienceregistry.org/trials/3875/history/198902
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
The intervention consists in a management training and mentorship program comprising 8 sessions (3 hours each) plus homeworks and reviews.
The training program offers to all participants state-of-the-art tools (like the business model canvas, customer validation and development, the balanced scorecard and strategy maps) to diagnose their business, assess their strategy/business model and, if necessary, re-orient or change it. However, the treatment group will also be instructed about how to do this following a scientific approach. The Scientific Approach consists of making decisions following a set of behavioural routines – similar to those used by scientists- comprising four key components:
1. Articulation of a “theory” to design a business model grounded on a well-defined framework of the targeted customer problem;
2. Formulation of explicit/testable/falsifiable hypotheses about the consequences of actions;
3. Collection and analysis of evidence and data or design and execution of experiments to test these hypotheses;
4. Open and critical analysis, evaluation and reflection of the evidence.
Our ‘theory of change’ builds on the intuition, supported by prior research, that, under conditions of uncertainty, many business owners overestimate (false positives) or underestimate (false negatives) the potential of the strategies they follow. Instead, microbusinesses that use a scientific approach make evidence-informed decisions and reduce the number of ‘incorrect’ decisions made. This is expected to have a positive impact on their productivity.
Intervention (Hidden)
The administration of the training and mentorship program is organized as follows.
Participants are allocated to 10 classes (approximately 24 participants per class). 5 classes will be held on Wednesdays and 5 on Saturdays to accommodate participants’ needs. 8 instructors will teach the 10 classes (some instructors will teach 2). Each instructor will be rigorously selected and undergo a train the instructor program about how to teach and mentor the training and mentorship program, what is the scientific approach and how to teach and mentor the treatment and the control groups. Each class is split into 2 groups (approximately 12 participants each; the small size of the group is geared towards ensuring an effective learning experience): treatment and control with treatment taught in the morning and control taught in the afternoon (Wednesday or Saturday).
Intervention Start Date
2019-02-13
Intervention End Date
2019-04-13

Primary Outcomes

Primary Outcomes (end points)
1. Level of adoption of the scientific approach. More specifically we will measure the pre-post intervention average level of adoption of the scientific approach in both the treatment and control groups using a purposely designed instrument (interview and survey) which allows to calculate an additive measure of adoption of the scientific approach broken down into its 4 components. We expect to detect a significant difference in the increase in the level of adoption of the scientific approach between treated and control microbusinesses.
2. Number, type and timing of strategic iterations -significant business changes- (short-term, 3 months since the start of the intervention). We expect to detect no difference or a slight difference in the number of strategic iterations during and after the intervention between the treated and the control group, with better adopters iterating earlier and more radically.
3. Productivity, measured according to the guidelines of the ONS productivity handbook
Primary Outcomes (explanation)
1. Level of adoption of the scientific approach. Measured using a purposely designed instrument (interview and survey) which allows to calculate an additive measure of adoption of the scientific approach broken down into its 4 components.
2. Strategic iterations. Measured as a count (and the timing) of the strategic re-orientations (e.g. change in customer segments, product/ service offered etc.)
3. Labor, capital, materials, energy and multiple factor productivity as defined in the ONS productivity handbook

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Parallel experimental design, with two arms, pure randomization at the microbusiness level and a 50%/50% allocation ratio.
Experimental Design Details
We will randomize participants into treatment and control. Once assigned to the treatment or control group, participants will be randomly assigned to instructors/classes. We will run randomization checks on a wide set of covariates (individual and microbusiness characteristics) to ensure balance. Randomization checks will be conducted into treatment (between treatment and control groups) and within and across class/instructors. Further randomization checks will be conducted across intervention calendar dates and between and within registered and non registered microbusinesses.
Randomization Method
Participants randomized into treatment and control using STATA randomization procedure.
Randomization Unit
Firm (microbusiness)
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
240 firms
Sample size: planned number of observations
2400 (240 firms X 10 monthly observation)
Sample size (or number of clusters) by treatment arms
120 microbusinesses control, 120 microbusinesses treatment
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Our power calculations uses a target impact approach and tries to calculate the sample size using as impact estimates the outcome of previous studies in similar contexts (Camuffo et al. 2019) and participants data. However, we also conducted sensitivity analysis simulating how the smallest true impact varies as a function of the sample size. Our target sample size of 240 microbusinesses stems from these analyses. These calculations will be refined in the piloting stage of the project and based on the actual size and past performance of the microbusinesses that will be recruited as participants. Nonetheless, assuming that at the beginning of the RCT the average revenue of the microbusinesses in the treatment and control groups is the same and equal to £70,000, with a standard deviation of £100,000, a sample size of 240 would more than suffice to confidently detect a 0.35 effect size difference, i.e. an increase in the average revenues of the treated microbusinesses of 50% relative to the control group. We plan to re-run our experimental power calculations using both G-Power and STATA as soon as we have the final information about the participants and possibly adjust our numbers. At any rate, our experimental design incudes repeated data collection over time, which will help to increase the power of our tests. We will use ANCOVA and multiple measures to improve power (MacKenzie, 2012).
IRB

Institutional Review Boards (IRBs)

IRB Name
Cass Research Ethics Committee
IRB Approval Date
2018-12-15
IRB Approval Number
Ethics ETH1819-0351: Prof Elena Novelli (Low risk)
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
November 30, 2019, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
November 30, 2019, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
261 entrepreneurs
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
261 entrepreneurs
Final Sample Size (or Number of Clusters) by Treatment Arms
128 control, 133 treatment
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
No
Reports, Papers & Other Materials

Relevant Paper(s)

Abstract
This large-scale replication of Camuffo et al.'s (2020) research, involving 759 firms in four randomized control trials, provides evidence of the teachability and performance benefits of a scientific approach in entrepreneurship. The larger size of the sample used in this paper leads to novel and more precise insights compared to prior work. We observe a positive, well-defined impact on idea termination and a non-linear effect on radical pivots, with treated firms favoring a few over none or repeated pivots. We provide a theoretically-based interpretation of the empirical results: the approach enhances entrepreneurs' efficiency in searching for viable ideas, but, this effect is dominated in this sample by the instilled methodic doubt, which leads to an increased awareness of the contingencies potentially hindering the idea’s success.
Citation
Camuffo, A., Gambardella, A., Messinese, D., Novelli, E., Paolucci, E., Spina, C. 2023. A Scientific Approach to Entrepreneurial Decision Making: Large Scale Replication and Extension, working paper
Abstract
This paper investigates the role of a firm’s degree of business development—defined as the extent to which an entrepreneur has already crystallized the details of their firm strategy, making radical changes unlikely—in moderating the impact of a scientific approach to decision-making on performance. We explore this issue with a question-driven approach based on evidence from a field experiment with 261 UK entrepreneurs. Results show that treated firms at a higher degree of business development perform better than control firms, whereas firms at a lower degree of business development perform worse than control firms. We present qualitative and quantitative evidence to interpret this result. We elaborate on the implications for future research.
Citation
Novelli, E., Spina, C. 2023. When do Entrepreneurs Benefit from Acting Like Scientists? A Field Experiment in the UK

Reports & Other Materials