A Scientific Approach to Innovation Management

We recruit participants through an online call for application. Applicants should be aspiring entrepreneurs willing to develop a new venture. Selected participants will be randomly assigned to three groups.

Participants in two treatment groups will receive six sessions of 4-hour in-person training (or 8 sessions of 3-hour in-person training). The sessions will take place every other week for about three months. These sessions will include interactive lectures and coaching by qualified mentors/instructors each working with a subset of the participants. Both treatment groups will receive general training related to entrepreneurial decision making and will learn how to collect and evaluate information about their entrepreneurial ideas. The two treatment groups are identical in terms of the number of classes offered, the basic structure of the course, and the examples used.

Content and instruction in one treatment group will emphasize both the theory and experimental elements within the Scientific Approach. Specifically, this content includes the theory behind hypothesis development, rigorous test design processes, hypothesis testing and experimentation. This will be referred to as the ‘theory-and-evidence-based’ treatment group.
For the second treatment group, greater emphasis will be devoted to experimentation and lesser attention devoted to the theory element. In particular, greater emphasis will be devoted to hypothesis testing and experimentation. This will be referred to as the ‘evidence-based’ treatment group.

Our expectations before conducting the study were that both treatments would be beneficial for entrepreneurs in different ways. We expected entrepreneurs who follow an evidence-based approach to engage in a fast-iterative decision-making process and improve their business idea through data collection and analysis. We expected entrepreneurs who follow a theory-and-evidence-based approach to be slower and more selective, taking more time to pivot to adapted, but more defined and effective, business idea.

The research team will collect various measures related to both participants and their business projects before, during, and after the training using surveys and interviews. Participation in the surveys, interviews and all the other activities of the present study is voluntary. Participants will not be paid for their participation, but they will receive free training and free access to all events and activities.

A third set of participants, the ‘pure control’ group, will not receive any formal training. However, these participants will be provided written materials relevant to decision making and the opportunity to participate in events held during the project.
The performance of participants in all three groups will be monitored over the course of one year and a half.

Update: the last RCT, run in India, has a slightly different design. In particular, interventions will be administered online, and there will not be a ‘pure control’ group.

2022-01-28

2024-02-17

Performance, pivot, project termination

1. Performance: our main dependent variable is the amount of revenue/total sales generated by the startups over time. We also record an alternative measure for performance by recording considering the number of employees and related changes over time.

2. Pivot: this is the cumulative number of times in which a firm makes a major change to its business model. We defined a change to be major by analyzing whether the firm changed the core elements of the product or service. In the panel dataset, pivot is identified by a binary variable.

3. Project Termination: this is a binary variable that takes value 0 until the entrepreneur ceases the startup, 1 in the period in which the startup terminates, and it is a missing value thereafter. To avoid attrition biases, we will check that the firms which leave the program also cease entrepreneurial activity.

The research team will also examine potential mechanisms behind these outcomes by collecting variables related to how and when decisions are made.

The research team plans to also examine additional categories of outcomes of interest:

DOMAINS:
Studying how broad domains and scientific thinking impact entrepreneurial idea quantity (number of alternative frameworks devised for the same idea), breadth (number of contexts of application for the same idea, distance among the contexts of application), and quality (performance and startup size).
Within one of the RCTs (Spain), these relationships will be studied through a 2x2 manipulation entailing a domains treatment (manipulating the breadth of domains) in addition to the main treatment.
Within another RCT (India), these relationships will be studied with respect to one specific domain: social and demographic background. In particular, we will study the impact of social / religious background on the types of ideas generated (target markets, target customers).
Main outcome variables:
- Number of alternative frameworks for the same idea
- Number of ideas developed
- Idea breadth
- Number of application contexts
- Distance among application contexts
- Changes in target markets (niche vs. mass), target customers, solution, value proposition
- Startup size

ENTREPRENEURIAL TEAMS:
Analyzing whether our interventions are associated with a higher likelihood and number of team changes in treated startups and with a higher adoption of a dual team formation strategy rather than resource-seeking or interpersonal attraction strategy alone.
Main survey measures related to this outcome:
- Team formation strategies
- Team member joining/leaving and reasons
- Transactive Memory System.

FUNDING:
Analyzing whether our interventions are associated with a higher likelihood of receiving funding from professional investors.
Main survey measure related to this outcome:
- Received funding from professional investors (binary variable)

GOAL ORIENTATION:
Investigating whether exposure to different types of entrepreneurship training changes entrepreneurs’ goal orientation, and whether different goal orientations affect the relationship between the training received (treatment) and the strategic decisions entrepreneurs make (outcome). Goal orientation is a multi-dimensional construct that includes three related dimensions:
- Performance-approach goal orientation
- Performance-avoid goal orientation
- Learning goal orientation.

MOTIVATIONS:
Studying whether entrepreneurs who are classified as belonging to different categories based on their motivations and growth aspirations have differing outcomes in terms of method application, pivoting and termination. The four categories will be created according to a 2x2 matrix generated from the following survey measures:
- Intrinsic or Extrinsic motivation to entrepreneurship
- High or Low growth aspiration.

PARSIMONY/ SUBTRACTION:
Analyzing the effect of the treatments on the parsimony of the entrepreneurs’ reasoning and formulation of business models, as well as studying the effects of parsimony as a mediator towards performance, pivoting and termination outcomes. The expectation is that a scientific treatment will be positively related to parsimony/ subtraction.
Main measures related to this outcome:
- Output of a computer simulation task where we measure additive and subtractive changes performed by entrepreneurs to a 10x10 digital grid pattern
- Survey measure of self-reported additive or subtractive changes to each entrepreneur’s business idea
- Analysis of the business idea descriptions submitted by entrepreneurs to identify additive or subtractive changes to their business models over time
- Changes over time in the number of elements in the target customers, value proposition, and solution components of the business idea.

PERCEPTION OF CHALLENGES:
Analyzing the effect of the intervention on entrepreneurs’ perceptions of challenges to entrepreneurship and their perceived ability to respond to these challenges. The expectation is towards a higher perceived ability for the theory-driven intervention, especially in relation to challenges related to the business development rather than on the external environment. We ask entrepreneurs in the survey to:
- Indicate the top-3 challenges among a list of proposed challenges
- Indicate, for each of the proposed challenges, their perceived ability to deal with them (Likert scale)
- Open-ended question about challenges and obstacles to business goals in the phone interview, to both validate results from the questionnaire and get richer information.
The analysis will also look at potential moderation effects and correlation with overconfidence traits, specifically using a battery of questions capturing the “illusion of control” bias.

PITCH DEVELOPMENT: study whether training entrepreneurs to think like scientists leads them to produce narratives of their ideas that elicit more positive evaluations of their written ‘pitch’ (i.e., description of their business idea) from external audiences. Main measures related to this outcome (coded by external raters on 1-5 scales):
- Perceived idea novelty
- Perceived cognitive legitimacy
- Perceived business viability.

We conduct a series of 6 RCTs in different Countries (Italy, the UK, The Netherlands, Colombia, Spain, and India) with approximately 150 early-stage entrepreneurs in each location to understand the conditions under which different elements to decision-making sustain entrepreneurial actions. We will randomly assign entrepreneurs who participate in the RCTs in each location to three experimental arms, corresponding to different combinations of the components of the scientific approach (theory-and-evidence, evidence, and a control group). The intervention consists in a three-month entrepreneurship training program, offered free of charge to participants. We will monitor the performance of these three groups over time. Firms in both treatment groups will learn how to collect and evaluate information about their entrepreneurial ideas, though the content of the training is different between the two groups:
1) The first treatment group will receive training focused on the ‘evidence-based’ approach, in which greater emphasis will be devoted to the “evidence” element within the scientific approach rather than devoted to the theory element
2) The second treatment group will receive training based on the ‘theory-and-evidence-based’ approach that emphasizes both the theory and experimental elements within the scientific approach
3) We also have a 'pure control' group, which includes entrepreneurs who will not receive training

We will observe the impact these decisions have over time, as the data collection process will take place over the course of one year

Not available

Participants are assigned to the different treatment groups by simple randomization (done in office by a computer using STATA), or a stratified randomization in the locations where this would be needed (for example, where there is a need to offer training in multiple languages or in different locations within the same country).

Individual

No

Around 900 individuals (i.e., 150 individuals x 6 RCT)

Around 5400 individual-time observations (i.e., 900 individuals x 6 time periods)

Allocation ratio = 37.5% 'theory-and-evidence-based' group, 37.5% 'evidence-based' group, 25% 'pure control' group

We justify the size of the samples based on experimental power calculations conducted using G*Power 3.1 (command F-tests, ANOVA: repeated measures, within-between interactions). We assume we detect a very small effect size (f=0.1), which is in line with the data and the outcomes in Camuffo et al. (2020a). We assume standard type I and type II errors (α err prob = 0.05 Power (1-β err prob) = 0.95). Given the three arms of the RCT, about 6 observations per firm, and making standard assumptions about correlation among repeated measures (0.5) and correction for non-sphericity (ε = 1), we obtain a required total sample size of 150 firms, which is in line with the above-described target sample size. All in all with six locations, 150 firms per location and 6 data points, we aim at a panel dataset composed of 5400 observations. This provides a comfortable buffer suggesting a sufficiently large power of the test even if, for some groups, the estimated effect sizes might be smaller than the anticipated effect used in this case.