|
Field
Trial Title
|
Before
AI and strategic decision-making (pilot)
|
After
AI and strategic decision-making (generation experiment)
|
|
Field
Trial Start Date
|
Before
August 18, 2023
|
After
February 16, 2024
|
|
Field
Last Published
|
Before
October 13, 2023 09:31 PM
|
After
February 16, 2024 04:54 AM
|
|
Field
Intervention Start Date
|
Before
August 18, 2023
|
After
February 16, 2024
|
|
Field
Intervention End Date
|
Before
September 29, 2023
|
After
February 23, 2024
|
|
Field
Primary Outcomes (Explanation)
|
Before
For each business plan, we will measure the following evaluation outcomes:
1. Five factors evaluated on a scale of 1 (low) to 10 (high), analyzed both individually and as an index: (1) writing of business plan; (2) innovation and value proposition; (3) execution plan; (4) potential to invest in this idea; (5) viability. We also plan to explore the standard deviation of these ratings across participants.
2. A binary indicator of whether the participant would accept the startup into the accelerator: "Would you accept this startup into the accelerator program?" Response: yes/no.
3. A binary indicator of whether the participant would be interested in being introduced to the startup: "Would you be interested in being introduced to the startup?" Response: yes/no.
4. Likelihood of the participant investing in the startup: "How likely would you be to invest in the startup? (Click on the slider and then select the number)". Response: range from 0 to 100. Analyzed using both the raw number and a binary indicator above the median.
We plan to analyze treatment effects by regressing each outcome on an indicator of whether the business plan was a GPT-generated version, running this specification also with business plan fixed effects and participant fixed effects to increase precision.
We also plan to analyze heterogeneity in treatment effects along the following dimensions, by interacting each with the treatment indicator of whether the business plan was a GPT-generated version:
1. An indicator of whether the business plan was accepted or rejected from the accelerator program
2. Writing quality of the original version of the business plan (both manually coded as a binary indicator for many grammatical and spelling errors, as well as algorithmically-generated measures)
3. An indicator of the cohort of the business plan application (2021 or 2022)
4. A binary indicator of whether a summary of the solution was provided as part of the problem statement in the original version of the business plan
5. A binary indicator of whether the original version of the business plan includes concrete signals already achieved (e.g., funds raised, product launches, number of partners or customers)
|
After
For each business plan, we will measure the following evaluation outcomes:
1. Five factors evaluated on a scale of 1 (low) to 10 (high), analyzed both individually and as an index: (1) writing of business plan; (2) innovation and value proposition; (3) execution plan; (4) potential to invest in this idea; (5) viability. We also plan to explore the standard deviation of these ratings across participants.
2. A binary indicator of whether the participant would accept the startup into the accelerator: "Would you accept this startup into the accelerator program?" Response: yes/no.
3. A binary indicator of whether the participant would be interested in being introduced to the startup: "Would you be interested in being introduced to the startup?" Response: yes/no.
4. Likelihood of the participant investing in the startup: "How likely would you be to invest in the startup? (Click on the slider and then select the number)". Response: range from 0 to 100. Analyzed using both the raw number and a binary indicator above the median.
We plan to analyze treatment effects by regressing each outcome on an indicator of whether the business plan was a GPT-generated version, running this specification with business plan fixed effects and participant fixed effects to increase precision.
We also plan to analyze heterogeneity in treatment effects along the following dimensions, by interacting each with the treatment indicator of whether the business plan was a GPT-generated version:
1. An indicator of whether the business plan was accepted or rejected from the accelerator program
2. Writing quality of the original version of the business plan (coded as a binary indicator for many grammatical and spelling errors)
|
|
Field
Randomization Method
|
Before
Six business plans are randomly assigned to one of four blocks using a computer. Each participant is randomly assigned to a block and the original or GPT version of each business plan using Qualtrics. The order of each business plan displayed to each participant is randomized using Qualtrics.
|
After
For each business plan, each participant is randomly assigned to the original or GPT version, using Qualtrics. The order of the business plan displayed to each participant is also randomized using Qualtrics.
|
|
Field
Randomization Unit
|
Before
Each business plan evaluation by a participant (i.e., each of the six business plans a participant evaluates is randomly assigned to either the original or GPT-generated version, such that participants are assigned three original and and three GPT-generated versions each).
|
After
Each business plan evaluation by a participant (i.e., each of the ten business plans a participant evaluates is randomly assigned to either the original or GPT-generated version)
|
|
Field
Planned Number of Clusters
|
Before
600 evaluations by 100 investors (from online platform)
|
After
2500 evaluations by 250 investors
|
|
Field
Planned Number of Observations
|
Before
600 evaluations by 100 investors (from online platform)
|
After
2500 evaluations by 250 investors
|
|
Field
Sample size (or number of clusters) by treatment arms
|
Before
300 evaluations assigned to original versions of business plans, 300 evaluations assigned to GPT-generated versions of business plans
|
After
1250 evaluations assigned to original versions of business plans, 1250 evaluations assigned to GPT-generated versions of business plans
|
|
Field
Intervention (Hidden)
|
Before
Each participant evaluates a randomly assigned set of six business plans. For each business plan, each participant is randomly assigned to one of two versions of business plans: the original or a GPT-generated version of the business plan.
|
After
Each participant evaluates a set of ten anonymized business plans. For each business plan, the intervention randomly assigns each participant to one of two versions of business plans: the original version submitted to a leading European accelerator program, or a GPT-generated version of the business plan, where we prompted GPT using the original entrepreneur's response to the ``Problem'' section of their business plan and asked it to generate the remaining plan, as a way to obtain GPT's proposed strategy to solve the problem described.
|