|
Field
Abstract
|
Before
Economic growth depends critically on managerial decisions that concern innovation, a context characterized by high uncertainty. Unfortunately, research shows that managers or entrepreneurs do not have good routines or methods to make decisions under uncertainty. This suggests that there might be serious benefits from improving the efficiency of public and private investments to nurture entrepreneurship. The goal of this project is to understand the implications of an approach that teaches managers to start their decision making process with a good framing of the problem, developing models that predict the outcomes of actions from logical
connections between antecedents and consequences, testing the model using existing data or data drawn from well-defined experimental designs and that assess the results in a disciplined way. We call this approach scientific because it overlaps to a significant
extent with the approach that scientists use to develop new knowledge. To this aim, the project designs and implements a large-scale randomized control trial (RCT) conducted in five countries (Italy, UK, Spain, Colombia, the Netherlands) based on an intervention that comprises six/eight training sessions over about three months offered to randomly generated groups of entreprenerurs.
|
After
Economic growth depends critically on managerial decisions that concern innovation, a context characterized by high uncertainty. Unfortunately, research shows that managers or entrepreneurs do not have good routines or methods to make decisions under uncertainty. This suggests that there might be serious benefits from improving the efficiency of public and private investments to nurture entrepreneurship. The goal of this project is to understand the implications of an approach that teaches managers to start their decision making process with a good framing of the problem, developing models that predict the outcomes of actions from logical
connections between antecedents and consequences, testing the model using existing data or data drawn from well-defined experimental designs and that assess the results in a disciplined way. We call this approach scientific because it overlaps to a significant
extent with the approach that scientists use to develop new knowledge. To this aim, the project designs and implements a large-scale randomized control trial (RCT) conducted in six countries (Italy, UK, Spain, Colombia, the Netherlands, India) based on an intervention that comprises six/eight training sessions over about three months offered to randomly generated groups of entrepreneurs.
|
|
Field
Trial End Date
|
Before
April 30, 2023
|
After
February 15, 2025
|
|
Field
Last Published
|
Before
February 24, 2022 01:14 PM
|
After
September 19, 2023 09:57 AM
|
|
Field
Intervention (Public)
|
Before
Start-ups receive 6x4h or 7x3h sessions of training (depending on the constraints of the different institutions that will host the RCT). However, the content of the training program is the same across RCTs/locations regardless of the number and length of the training sessions. These sessions include interactive lectures and coaching by qualified mentors/instructors each working with a subset of the start-up sample. The two different treated groups will receive the same amount of training on entrepreneurial decision making (on topics like business model canvas, customers' interviews, minimum viable products/services, prototype, etc.). The content of one treatment group (theory-driven) will emphasize both the theory and experimental elements within the scientific method. Specifically, this content includes the theory behind hypothesis development, rigorous test design processes, hypothesis testing, collection of evidence and disciplined evaluation. This will be referred to as the ‘theory-driven’ treatment group.
For the second treatment group (evidence-driven), greater emphasis will be devoted to hypothesis testing, collection of evidence and disciplined evaluation rather than the theory element.
A third set of participants, the ‘pure control’ group, will not receive any formal training but will be offered the opportunity to attend a set of informational events about entrepreneurial development and how to solve typical problems. These events will be open and available to entrepreneurs in all groups (at different times to avoid contamination). This will be done to keep up participants’ motivation and reduce the natural attrition that otherwise is likely to characterize this group.
Participants are assigned to the different treatment groups by simple randomization, or a stratified randomization in the locations where this would be needed (for example, where there is a need to offer training in multiple languages or in different locations within the same country).
|
After
We recruit participants through an online call for application. Applicants should be aspiring entrepreneurs willing to develop a new venture. Selected participants will be randomly assigned to three groups.
Participants in two treatment groups will receive six sessions of 4-hour in-person training (or 8 sessions of 3-hour in-person training). The sessions will take place every other week for about three months. These sessions will include interactive lectures and coaching by qualified mentors/instructors each working with a subset of the participants. Both treatment groups will receive general training related to entrepreneurial decision making and will learn how to collect and evaluate information about their entrepreneurial ideas. The two treatment groups are identical in terms of the number of classes offered, the basic structure of the course, and the examples used.
Content and instruction in one treatment group will emphasize both the theory and experimental elements within the Scientific Approach. Specifically, this content includes the theory behind hypothesis development, rigorous test design processes, hypothesis testing and experimentation. This will be referred to as the ‘theory-and-evidence-based’ treatment group.
For the second treatment group, greater emphasis will be devoted to experimentation and lesser attention devoted to the theory element. In particular, greater emphasis will be devoted to hypothesis testing and experimentation. This will be referred to as the ‘evidence-based’ treatment group.
Our expectations before conducting the study were that both treatments would be beneficial for entrepreneurs in different ways. We expected entrepreneurs who follow an evidence-based approach to engage in a fast-iterative decision-making process and improve their business idea through data collection and analysis. We expected entrepreneurs who follow a theory-and-evidence-based approach to be slower and more selective, taking more time to pivot to adapted, but more defined and effective, business idea.
The research team will collect various measures related to both participants and their business projects before, during, and after the training using surveys and interviews. Participation in the surveys, interviews and all the other activities of the present study is voluntary. Participants will not be paid for their participation, but they will receive free training and free access to all events and activities.
A third set of participants, the ‘pure control’ group, will not receive any formal training. However, these participants will be provided written materials relevant to decision making and the opportunity to participate in events held during the project.
The performance of participants in all three groups will be monitored over the course of one year and a half.
Update: the last RCT, run in India, has a slightly different design. In particular, interventions will be administered online, and there will not be a ‘pure control’ group.
|
|
Field
Intervention End Date
|
Before
July 01, 2022
|
After
February 17, 2024
|
|
Field
Primary Outcomes (Explanation)
|
Before
|
After
1. Performance: our main dependent variable is the amount of revenue/total sales generated by the startups over time. We also record an alternative measure for performance by recording considering the number of employees and related changes over time.
2. Pivot: this is the cumulative number of times in which a firm makes a major change to its business model. We defined a change to be major by analyzing whether the firm changed the core elements of the product or service. In the panel dataset, pivot is identified by a binary variable.
3. Project Termination: this is a binary variable that takes value 0 until the entrepreneur ceases the startup, 1 in the period in which the startup terminates, and it is a missing value thereafter. To avoid attrition biases, we will check that the firms which leave the program also cease entrepreneurial activity.
The research team will also examine potential mechanisms behind these outcomes by collecting variables related to how and when decisions are made.
|
|
Field
Experimental Design (Public)
|
Before
We conduct a series of 5 RCTs in different Countries (Italy, the UK, The Netherlands, Colombia,and Spain) with approximately 150 early-stage entrepreneurs in each location to understand the conditions under which different elements to decision-making sustain entrepreneurial actions. We will randomly assign entrepreneurs who participate in the RCTs in each location to three experimental arms, corresponding to different combinations of the components of the scientific approach (theory and experimentation, experimentation and a control group). The intervention consists in a three-month entrepreneurship training program, offered free of charge to participants. We will monitor the performance of these three groups over time. Firms in both treatment groups will learn how to collect and evaluate information about their entrepreneurial ideas, though the content of the training is different between the two groups:
1) The first treatment group will receive training focused on the ‘evidence-driven’ approach, in which greater emphasis will be devoted to the “evidence” element within the scientific approach rather than devoted to the theory element.
2) The second treatment group will receive training based on the ‘theory-driven’ approach that emphasizes both the theory and experimental elements within the scientific approach.
3) We also have a pure control group, which includes entrepreneurs who will not receive training
We will observe the impact these decisions have over time, as the data collection process will take place over the course of one year.
|
After
We conduct a series of 6 RCTs in different Countries (Italy, the UK, The Netherlands, Colombia, Spain, and India) with approximately 150 early-stage entrepreneurs in each location to understand the conditions under which different elements to decision-making sustain entrepreneurial actions. We will randomly assign entrepreneurs who participate in the RCTs in each location to three experimental arms, corresponding to different combinations of the components of the scientific approach (theory-and-evidence, evidence, and a control group). The intervention consists in a three-month entrepreneurship training program, offered free of charge to participants. We will monitor the performance of these three groups over time. Firms in both treatment groups will learn how to collect and evaluate information about their entrepreneurial ideas, though the content of the training is different between the two groups:
1) The first treatment group will receive training focused on the ‘evidence-based’ approach, in which greater emphasis will be devoted to the “evidence” element within the scientific approach rather than devoted to the theory element
2) The second treatment group will receive training based on the ‘theory-and-evidence-based’ approach that emphasizes both the theory and experimental elements within the scientific approach
3) We also have a 'pure control' group, which includes entrepreneurs who will not receive training
We will observe the impact these decisions have over time, as the data collection process will take place over the course of one year
|
|
Field
Planned Number of Clusters
|
Before
750 individuals
|
After
Around 900 individuals (i.e., 150 individuals x 6 RCT)
|
|
Field
Planned Number of Observations
|
Before
7500 individual- month observations
|
After
Around 5400 individual-time observations (i.e., 900 individuals x 6 time periods)
|
|
Field
Sample size (or number of clusters) by treatment arms
|
Before
Allocation ratio= 1.5:1.5:1
|
After
Allocation ratio = 37.5% 'theory-and-evidence-based' group, 37.5% 'evidence-based' group, 25% 'pure control' group
|
|
Field
Power calculation: Minimum Detectable Effect Size for Main Outcomes
|
Before
We justify the size of the samples based on experimental power calculations conducted using G*Power 3.1 (command F-tests, ANOVA: repeated measures, within-between interactions). We assume we detect a very small effect size (f=0.1), which is in line with the data and the outcomes in Camuffo et al. (2020a). We assume standard type I and type II errors (α err prob = 0.05 Power (1-β err prob) = 0.95). Given the three arms of the RCT, about 10 observations per firm, and making standard assumptions about correlation among repeated measures (0.5) and correction for non-sphericity (ε = 1), we obtain a required total sample size of 150 firms, which is in line with the above-described target sample size. All in all with six locations, 150 firms per location and 10 data points, we aim at a panel dataset composed of 9000 observations This provides a comfortable buffer suggesting a sufficiently large power of the test even if, for some groups, the estimated effect sizes might be smaller than the anticipated effect used in this case.
|
After
We justify the size of the samples based on experimental power calculations conducted using G*Power 3.1 (command F-tests, ANOVA: repeated measures, within-between interactions). We assume we detect a very small effect size (f=0.1), which is in line with the data and the outcomes in Camuffo et al. (2020a). We assume standard type I and type II errors (α err prob = 0.05 Power (1-β err prob) = 0.95). Given the three arms of the RCT, about 6 observations per firm, and making standard assumptions about correlation among repeated measures (0.5) and correction for non-sphericity (ε = 1), we obtain a required total sample size of 150 firms, which is in line with the above-described target sample size. All in all with six locations, 150 firms per location and 6 data points, we aim at a panel dataset composed of 5400 observations. This provides a comfortable buffer suggesting a sufficiently large power of the test even if, for some groups, the estimated effect sizes might be smaller than the anticipated effect used in this case.
|
|
Field
Secondary Outcomes (End Points)
|
Before
• Degree of behavioural compliance with the treatment (scientific intensity)
• Number, diversity, distance and sequentiality of applications of the business idea identified by entrepreneurs
• The type of challenges perceived by the founder and perceived confidence in addressing those challenges
• Team dynamics (e.g., formation and changes in the composition of the entrepreneurial team)
• External evaluations of the entrepreneurial ideas
|
After
The research team plans to also examine additional categories of outcomes of interest:
DOMAINS:
Studying how broad domains and scientific thinking impact entrepreneurial idea quantity (number of alternative frameworks devised for the same idea), breadth (number of contexts of application for the same idea, distance among the contexts of application), and quality (performance and startup size).
Within one of the RCTs (Spain), these relationships will be studied through a 2x2 manipulation entailing a domains treatment (manipulating the breadth of domains) in addition to the main treatment.
Within another RCT (India), these relationships will be studied with respect to one specific domain: social and demographic background. In particular, we will study the impact of social / religious background on the types of ideas generated (target markets, target customers).
Main outcome variables:
- Number of alternative frameworks for the same idea
- Number of ideas developed
- Idea breadth
- Number of application contexts
- Distance among application contexts
- Changes in target markets (niche vs. mass), target customers, solution, value proposition
- Startup size
ENTREPRENEURIAL TEAMS:
Analyzing whether our interventions are associated with a higher likelihood and number of team changes in treated startups and with a higher adoption of a dual team formation strategy rather than resource-seeking or interpersonal attraction strategy alone.
Main survey measures related to this outcome:
- Team formation strategies
- Team member joining/leaving and reasons
- Transactive Memory System.
FUNDING:
Analyzing whether our interventions are associated with a higher likelihood of receiving funding from professional investors.
Main survey measure related to this outcome:
- Received funding from professional investors (binary variable)
GOAL ORIENTATION:
Investigating whether exposure to different types of entrepreneurship training changes entrepreneurs’ goal orientation, and whether different goal orientations affect the relationship between the training received (treatment) and the strategic decisions entrepreneurs make (outcome). Goal orientation is a multi-dimensional construct that includes three related dimensions:
- Performance-approach goal orientation
- Performance-avoid goal orientation
- Learning goal orientation.
MOTIVATIONS:
Studying whether entrepreneurs who are classified as belonging to different categories based on their motivations and growth aspirations have differing outcomes in terms of method application, pivoting and termination. The four categories will be created according to a 2x2 matrix generated from the following survey measures:
- Intrinsic or Extrinsic motivation to entrepreneurship
- High or Low growth aspiration.
PARSIMONY/ SUBTRACTION:
Analyzing the effect of the treatments on the parsimony of the entrepreneurs’ reasoning and formulation of business models, as well as studying the effects of parsimony as a mediator towards performance, pivoting and termination outcomes. The expectation is that a scientific treatment will be positively related to parsimony/ subtraction.
Main measures related to this outcome:
- Output of a computer simulation task where we measure additive and subtractive changes performed by entrepreneurs to a 10x10 digital grid pattern
- Survey measure of self-reported additive or subtractive changes to each entrepreneur’s business idea
- Analysis of the business idea descriptions submitted by entrepreneurs to identify additive or subtractive changes to their business models over time
- Changes over time in the number of elements in the target customers, value proposition, and solution components of the business idea.
PERCEPTION OF CHALLENGES:
Analyzing the effect of the intervention on entrepreneurs’ perceptions of challenges to entrepreneurship and their perceived ability to respond to these challenges. The expectation is towards a higher perceived ability for the theory-driven intervention, especially in relation to challenges related to the business development rather than on the external environment. We ask entrepreneurs in the survey to:
- Indicate the top-3 challenges among a list of proposed challenges
- Indicate, for each of the proposed challenges, their perceived ability to deal with them (Likert scale)
- Open-ended question about challenges and obstacles to business goals in the phone interview, to both validate results from the questionnaire and get richer information.
The analysis will also look at potential moderation effects and correlation with overconfidence traits, specifically using a battery of questions capturing the “illusion of control” bias.
PITCH DEVELOPMENT: study whether training entrepreneurs to think like scientists leads them to produce narratives of their ideas that elicit more positive evaluations of their written ‘pitch’ (i.e., description of their business idea) from external audiences. Main measures related to this outcome (coded by external raters on 1-5 scales):
- Perceived idea novelty
- Perceived cognitive legitimacy
- Perceived business viability.
|