AI and strategic decision-making (generation experiment)

See details below.

Each participant evaluates a set of ten anonymized business plans. For each business plan, the intervention randomly assigns each participant to one of two versions of business plans: the original version submitted to a leading European accelerator program, or a GPT-generated version of the business plan, where we prompted GPT using the original entrepreneur's response to the ``Problem'' section of their business plan and asked it to generate the remaining plan, as a way to obtain GPT's proposed strategy to solve the problem described.

2024-02-16

2024-02-23

See details below.

For each business plan, we will measure the following evaluation outcomes:

1. Five factors evaluated on a scale of 1 (low) to 10 (high), analyzed both individually and as an index: (1) writing of business plan; (2) innovation and value proposition; (3) execution plan; (4) potential to invest in this idea; (5) viability. We also plan to explore the standard deviation of these ratings across participants.

2. A binary indicator of whether the participant would accept the startup into the accelerator: "Would you accept this startup into the accelerator program?" Response: yes/no.

3. A binary indicator of whether the participant would be interested in being introduced to the startup: "Would you be interested in being introduced to the startup?" Response: yes/no.

4. Likelihood of the participant investing in the startup: "How likely would you be to invest in the startup? (Click on the slider and then select the number)". Response: range from 0 to 100. Analyzed using both the raw number and a binary indicator above the median.

We plan to analyze treatment effects by regressing each outcome on an indicator of whether the business plan was a GPT-generated version, running this specification with business plan fixed effects and participant fixed effects to increase precision.

We also plan to analyze heterogeneity in treatment effects along the following dimensions, by interacting each with the treatment indicator of whether the business plan was a GPT-generated version:

1. An indicator of whether the business plan was accepted or rejected from the accelerator program

2. Writing quality of the original version of the business plan (coded as a binary indicator for many grammatical and spelling errors)

See details below.

Following the pilot previously registered as an earlier version, we run the following experiment.

Participants are assigned to evaluate ten business plans each for a leading startup accelerator program in Europe. We source these plans by obtaining 10 original business plans submitted to the program (5 accepted, 5 rejected) and generating GPT-generated versions of these plans.

We randomly assign each participant to the original version or a GPT-generated version of the business plan for each of the ten plans, which are displayed in randomized order.

We recruit approximately 250 investors from an online platform. We recruit as many participants as possible with the pre-screening criteria that they are (1) employed in business management and administration, finance, information technology, or science, technology, engineering, and mathematics; (2) they have experience in one of the following types of investments: angel syndicates, venture capital, or private equity; (3) they have an approval rate of 91-100 on the platform. We also screen out any participants that fail a simple knowledge check relating to start-up investing.

For each business plan, each participant is randomly assigned to the original or GPT version, using Qualtrics. The order of the business plan displayed to each participant is also randomized using Qualtrics.

Each business plan evaluation by a participant (i.e., each of the ten business plans a participant evaluates is randomly assigned to either the original or GPT-generated version)

No

2500 evaluations by 250 investors

2500 evaluations by 250 investors

1250 evaluations assigned to original versions of business plans, 1250 evaluations assigned to GPT-generated versions of business plans