The effectiveness of public support for high-potential businesses – The Growth Impact Pilot

GIP was a randomised controlled trial run in 2014-2015. The purpose of the programme was to test whether an intensive course of business coaching, when offered alongside leadership and management training, leads to additional growth compared to only offering the leadership and management training. GIP was embedded within the wider GrowthAccelerator programme. Both initiatives were designed to selectively target established small and medium-sized enterprises (SMEs) identified as having high growth potential, rather than being open to all businesses.

2014-04-01

2018-09-01

The primary outcomes for our analysis are computed as follows, using data from the ONS Longitudinal Business Database (LBD):
Cumulative turnover, the sum of annual turnover between the programme year and the programme year + 4, in £0,000. Turnover for years in which a business is marked as “inactive” will be counted as zero. Each annual turnover value will be adjusted for inflation by using the GDP deflator value in that year.
Cumulative employment (“job years”), defined as the sum of people employed by the company between the programme year and the programme year + 4. Employment for years in which a business is marked as “inactive” will be counted as zero.
A proxy measure of productivity, defined as turnover per employee in the programme year + 4. If employment is reported as zero or is missing in a particular year, we will consider productivity that year to be zero.

If some of your outcomes will be constructed (e.g. "women empowerment") please provide a description of how the outcome will be constructed from the main variables.

Our primary outcome measures of firm turnover and employment are constructed by the Office for National Statistics (ONS) and made available to researchers through the Longitudinal Business Database (LBD). While Gross Value Added (GVA) would be the preferred measure of economic contribution, this data is not currently accessible via the LBD, necessitating the use of turnover as the primary measure of firm output.

The secondary outcomes for our analysis are computed as follows, using data from the LBD, and the Department for Business and Trade (DBT):
Turnover in the programme year + 4, in £0,000.
Employment in the programme year + 4.
Survival, defined as whether the business is marked as “active” in the LBD in the programme year + 4, as a binary measure (0 or 1).
“High-growth” status between the programme year and the programme year + 4, defined as whether the business achieved > 20% growth in any consecutive three years, a binary measure (0 or 1).
Log of the two primary measures

We will also examine longer-term outcomes as part of our exploratory analysis, as the continued trajectory of these firms is of great interest. However, we have defined our primary outcome period to conclude before the major economic disruption of the COVID-19 pandemic to ensure our main findings reflect the impact of the programme before this significant and unexpected external shock.

We have selected the nominal cumulative measures for turnover and employment as our primary outcomes. This choice is driven by the need for the results to align with how they would be used to assess value for money, which compares the total nominal gain against the cost of the programme.

However, as part of our secondary, exploratory analysis, we will also assess the log-transformed cumulative outcomes. Our power calculations show the implied nominal MDES for the log measures is greater, this percentage-based approach likely better reflects the plausible, multiplicative patterns of how businesses actually grow in response to support.

GIP was an RCT run in 2014-2015. The purpose of the programme was to test whether an intensive course of business coaching, when offered in addition to leadership and management training, leads to growth.
GIP employed a variation-in-treatment design, in which businesses were randomly assigned to the control or treatment group. Participants in the control group were allocated to receive leadership and management training (matched funding of up to £2,000 per senior manager), while participants in the treatment group were allocated to receive training plus one-to-one business coaching (valued at £3,500). These elements are the same as those provided through the GrowthAccelerator programme (see below).
There were 546 participating SMEs, including 274 in the control group and 272 in the treatment group (roughly ½ allocated to the control group and ½ to the treatment group).
Contributors to GIP included the Department of Business and Trade (DBT), which was then the Department for Business, Innovation and Skills (BIS) and members of the Innovation Growth Lab (IGL).

Public lottery, randomization done in office by a computer, coin flip, etc.

Randomly allocated by computer through a process implemented by academic researchers who were not otherwise involved in the programme or its evaluation.

Randomisation was stratified by number of employees, turnover band and location.

No

Out of 546 participating SMEs (274 in the control group and 272 in the treatment group), we expect to be able to find matches with the important databases (most notably the ONS Longitudinal Business Database) for about 513 firms.

The sample size for our analysis will be reduced because matches must be found to the ONS internal identifiers (entrefs) and outliers will be eliminated. We anticipate a final total sample size of approximately 513 companies. Given that the programme targeted established firms with high growth potential, we had anticipated a near-complete match rate to the ONS administrative data. We have, however, encountered some unexpected attrition, the reasons for which are not fully clear but are perhaps due to administrative errors in how firm identifiers were originally recorded or retained. At the time of registration, the complete matching process has only been completed for the control group, but as the process is independent of treatment status, we do not expect this attrition to be systematic or to introduce bias into our final analysis.

About 256 firms in treatment group, about 257 in control group

Accounting for sample design and clustering, what is the minimum detectable effect size for main outcomes. Specify the unit, standard deviation, and percentage. Our power calculations are based on an initial ingest of data for the GIP control group only. All calculations assume an alpha level of 5% and 80% power. For cumulative turnover, based on a mean of 6003.5 thousand pounds, a standard deviation of 5650.2 thousand pounds, and an R-squared of 36.7% for covariates, our minimum detectable effect size is 1114.2 thousand pounds, or a difference of 18.6% between the treatment and control groups. For cumulative employment (“job years”), based on a mean of 90.5 job years, a standard deviation of 86.2 job years, and an R-squared of 56.0% for covariates, our minimum detectable effect size is 14.2 job years, or a difference of 15.7% between the treatment and control groups. For productivity, based on a mean of 97.5 thousand pounds per employee, a standard deviation of 77.2 thousand pounds per employee, and an R-squared of 3.1% for covariates, our minimum detectable effect size is 18.8 thousand pounds per employee, or a difference of 19.3% between the treatment and control groups. Our updated power calculations, based on the observed variance in the administrative data for the control group, indicate a Minimum Detectable Effect Size (MDES) for our primary nominal outcomes that is substantially higher than was anticipated in the original trial design.