Experimental Design Details
TRAINEE INVESTORS
We will assess the main hypothesis using the regressions:
Y_ip=α_1 y_0p±α_2 F_ip±α_3 〖T1〗_ip ±〖 α〗_4 〖〖T1×F〗_ip±X_j±L_p±ε_ip〗_
Y_ip=〖β_1 y_0p± β〗_2 F_ip±β_3 〖T2〗_ip ±〖 β〗_4 〖〖T2×F〗_ip±X_j±L_p±ε_ip〗_
This is an ANCOVA regression, which we plan to use to increase our statistical power, following McKenzie (2012). Y0 is the baseline measurement of the dependent variable – before the program starts. We will measure the dependent variable three times during each program and pool the three measurements. (We will also measure the dependent variable at specific points in time, which will give us less statistical power, but will produce directional effects over time, which we also plan to analyze.)
Dependent Variable: The dependent variable is Yp – the propensity to invest in a startup by the paired program. The four paired programs will take place in four geographic regions and include entrepreneurs from across those regions: Sub-Saharan Africa, India, MENA and Latin America. This will result in a total sample of eight programs - four treatment programs and four control programs – with one treatment and one control program in each location. We use fixed effects for the paired program in all regressions. (We will only include fixed effects for the investor in pooled regressions when we join up the sample with the professional investors.)
We will measure the dependent variable using 4 methods. We will build our model around the scales and the qualitative proportion measurements. Method 1 and 3 are what we used to build our model:
Scales: Each trainee investor will evaluate each startup on a scale. The baseline evaluation takes place on a 6-point scale. Thereafter, evaluators will use a 24-point scale (control group) and a 32-point scale (treatment group). All scale evaluations are normalized by the program using a z-score.
Binary: Each trainee investor will know that the top 2 rated startups will receive investment. Therefore, each trainee investor will carefully consider who they place in the top 2.
Qualitative: Each trainee investor will be asked what additional information they need from the investor. We will assess the proportion of promotion vs. prevention-focused questions (Kanze et al. 2018).
Performance-reward bias: Normalized scale (Male normalized qualitative proportion – Female normalized qualitative proportion) – see Castilla (2008). Intuitively, if you get the same scale rank, what is the difference between the qualitative score by the gender of the entrepreneur?
Intervention 1: For intervention 1 on prompting confident inquiry, we will have a sample size of at least 216 investor decisions on startups led by a female founder (within 720 total investor decisions on all startups). We will assess the effect of this treatment using a between subject design. Using 1 pre measure and 3 post measures in ANCOVA requires a sample size of 212 total decisions on females (across treatment and control) for a total power calculation of 0.85. We therefore estimate that we will see a significant positive effect of the treatment on the qualitative dependent variable primarily (but we will look for positive and significant effects on other DVs too). Therefore, we expect a positive and significant sign on a4 on all DVs.
Intervention 2: For intervention 2 on evaluating using demonstrated competence, we will have the same sample size as intervention 1, with the same power calculations. We therefore estimate that we will see a significant positive effect of the treatment on the scale dependent variable primarily (but we will look for positive and significant effects on other DVs too). Therefore, we expect a positive and significant sign on a4 on all DVs.
Intervention 3: TBD.
Control variables. In the equations where we do not use investor fixed effects, we will include potential control variables at the investor level including the gender of the investor (Greenberg & Mollick, 2018), and experience of the investor (Clingingsmith & Shane, 2018), which we will measure by a binary variable: having participated in another accelerator, and continuous variables: years of experience in the market; and
In the equations where we do not use startup fixed effects, we will include control variables for the startups including:
Evidence of their underlying quality: whether they were accepted by Connect or waitlisted (binary); their average score in Connect due diligence (1-4 continuous)
Evidence of their maturity: whether they were in the most popular geographic market for the program from the finalist group i.e., Egypt for MENA (binary); number of founders (categorical); total employees (categorical); the log of funds raised (continuous).
POOLED TRAINEE AND PROFESSIONAL INVESTORS
Ascertaining separate effects of interventions 1 from intervention 2. There are several ways to ascertain the separate effects of interventions 1 and 2. First, we are separating the independent variables, and will use the qualitative variable as the primary DV for prompting consistent enquiry, and the scale variable as the primary DV for evaluating using demonstrated competence.
Second, we will run a cross-sectional regression for additional information in a pooled sample across trainee and professional investors, and will have a total sample size of at least 456 decisions on female founders (240 from professional investors and 216 from trainee investors) and at least 1,500 decisions on all founders (780 from professional investors and 720 from trainee investors) all during one time period. To detect an effect, we will require d = 0.309 without clustering. Therefore, we will be looking for only a positive sign on a4 using this model.
Third, we will conduct a diff-in-diff on the additional information questions for all 360 decisions for all entrepreneurs in the treatment program at two time periods (across evaluations of male and female founders). This will be a within subject regression. We expect to see only a positive sign on a4.
PROFESSIONAL INVESTORS
We will assess the first main hypothesis using the OLS regression:
Y_ijp=α_1 F_ijp±α_2 〖T1〗_ijp ±〖 α〗_3 〖〖T1×F〗_ijp±L_i±L_j±L_p±ε_ip〗_
Dependent Variable: The dependent variable is Yijp – the propensity to invest in a startup by i – the investor, j – the startup, and p – the paired program.
We will measure the dependent variable using two methods. We will build our model around the scales and the qualitative proportion measurements. Method 1 and 2 are what we used to build our model:
Binary: Before meeting each company, each professional investor will be asked “Do you want to meet this company?” [Definitely want to meet, Would meet if there’s time, No thank you]. The first two options will be treated as binary responses. First, 1 = definitely want to meet and 0 = other. Second 1 = meet / other, 0 = No thank you.
Qualitative: Each professional investor will be asked what additional information they need from the investor. We will assess the proportion of promotion vs. prevention-focused questions (Kanze et al. 2018).
Performance-reward bias: Normalized scale (Male normalized qualitative proportion – Female normalized qualitative proportion) – see Castilla (2008). Intuitively, if you get the same scale rank, what is the difference between the qualitative score by the gender of the entrepreneur?
Intervention 1: For intervention 1 on prompting confident inquiry, we will have a sample size of at least 216 investor decisions on startups led by a female founder (within 720 total investor decisions on all startups). We will assess the effect of this treatment using a between subject design. Using 1 pre measure and 3 post measures in ANCOVA requires a sample size of 212 total decisions on females (across treatment and control) for a total power calculation of 0.85. We therefore estimate that we will see a significant positive effect of the treatment on the qualitative dependent variable primarily (but we will look for positive and significant effects on other DVs too). Therefore, we expect a positive and significant sign on a4 on all DVs.
Intervention 2: TBD.
PROESSIONAL INVESTORS
In the equations where we do not use investor fixed effects, we will include potential control variables at the investor level including the gender of the investor (Greenberg & Mollick, 2018) and experience of the investor (Clingingsmith & Shane, 2018). We will also track organizational variables about the organization the investor works for, to assess potential directional heterogenous effects of treatment based on: the size of the investment organization; the date it last raised funds (e.g. Hochberg, Serrano, Ziedonis, 2017); its investment thesis i.e. sector and geography; stated interest in gender-based investments; and motivation of the firm to focus on gender (e.g. Ely & Thomas, 2001), to examine organizational variables and heterogenous results.
We predict that the gender of the investor will have no effect, that the experience of the investor and a stated interest in gender-based investments will result in higher evaluations for female founders.