Experimental Design Details
We will explore the effect of changing two elements of investors’ evaluation processes on individual investor decisions. One intervention changes the evaluation practice used within an organization, and one intervention changes the information received from the environmental level. Following Yang and Aldrich (2014), We conceptualize organization-level practices and environmental information as frames or inputs to decision-making. By testing changes at each level, we will begin to assess the level and type of intervention that may affect investor decision-making.
Evaluation practice. For the first intervention, we draw from research on diversity in hiring decisions (Kalev, Dobbin, & Kelly, 2006; Dobbin, Kim, & Kalev, 2011). Many diversity training programs have been ineffective at cultivating diversity (Kalev et al., 2006), and an overt focus on meritocracy can even result in more bias (e.g. Castilla & Bernard, 2010). When there is ambiguity in hiring criteria, not only do hiring managers fill in the blanks with stereotypes, but the criteria used to assess merit “can be defined flexibly in a manner congenial to the idiosyncratic strengths of applicants who belong to desired groups” (Uhlmann & Cohen, 2005: 474). Uhlmann and Cohen designed an experiment that reduced the opportunity to retroactively construct criteria for hiring managers, which resulted in more female candidates being recruited. By asking evaluators to commit to placing weights on a set of hiring criteria before assessing applications, candidates were less able to define merit based on the applications they saw compared with a control group. We will replicate this study with equity investors who will weigh criteria before assessing startup briefs. This leads to two hypotheses:
Hypothesis 1a: Investors that weigh investment criteria before evaluating startup applications will be more likely to invest in startups founded by female CEOs than those that weight criteria after evaluating startup applications
Hypothesis 1b: Investors that consider and weigh investment criteria before evaluating startup applications will prioritize different criteria than those weighing criteria after evaluating startup applications (who will retroactively construct criteria)
Legitimating information. For the second intervention, we draw from the entrepreneurial finance literature, which has found that investors must consider how to obtain follow-on investment to succeed (e.g. Gompers, 1995), that they are more likely to invest when they receive information that the startup has a prominent affiliate (Stuart, Hoang, & Hybels, 1999), or when they gain information about a startup from a trusted syndicate partner (Sorenson & Stuart, 2001). The literature on equity investments made by public actors is rare, and evidence on policies designed to increase equity investment is mixed (Lerner, 2012; Lerner & Nanda, 2020). However, in some contexts, equity investors are more likely to invest in early-stage startups when they receive legitimating information about startup quality i.e. when startups receive publicly funded R&D grants (Howell, 2017). This effect could be even more valuable to startups founded by female founders, particularly if women are less likely to fit into investor heuristics (e.g. Huang, 2018; Kanze et al., 2018). This leads to two hypotheses:
Hypothesis 2a: Investors will be more likely to invest in startups that have previously received investment from a legitimate publicly-funded source
Hypothesis 2b: Investors will be more likely to invest in startups founded by female CEOs that have previously received investment from a legitimate publicly-funded source
We will assess the main hypotheses using the regression:
Yijt = a0 + a1.Gijt + a2.T1ijt + a3.T2ijt + a4.[T1ijt.Gijt] + a5.[T2ijt.Gijt] + Li + Lj + Lt + eijt
The dependent variable is Yijt, the propensity to invest in a startup led by a female CEO by the investor (i), the startup pairing (j) and the order in which the investment is made (t). I use fixed effects for the startup pairing (j) and the order in which the investment is made (t).
Our variables of interest are startups led by a female CEO (G), and two treatments: how organizations ask investors to apply the criteria they use (T1); and the type of legitimizing information investors receive from the field (T2). Our hypotheses predict that both treatment 1 and treatment 2 will increase the propensity to invest in firms founded by female CEOs - T1ijt.Gijt and T2ijt.Gijt. Therefore, we will be looking for a positive sign on a4 and a5.
For treatment 1, we will have a sample size of 100 investor decisions on startups led by a female CEO (25 treatment investors making two decisions each and weighting the criteria before investing, and 25 investors each making two decisions without any treatment, and weighting their criteria after investing). We will assess the effect of this treatment using a between subject design. To analyze our main hypothesis 1a, we expect a positive sign on the interaction between startups founded by a female CEO and the treatment. We predict only a directional effect, because the expected effect of this treatment is d=0.4 (Uhlmann & Cohen, 2005), for which we need a sample size of 272 decisions to find evidence of a significant effect. We will causally test this hypothesis further in study 2.
We will assess the effect of treatment 2 as a whole using a within treatment design, which will result in a sample size of 200 decisions (each of the 100 investors will see one investment with treatment information on the term sheet, and one without). While the expected effect of this treatment is unknown, it is likely to be more effective than a research grant at d=0.1 (Howell, 2017), because all startups in the sample are actively pursuing equity investment, and we will increase the value of the investment. We predict a positive significant effect on coefficient a3. This will test hypothesis 2a, and requires an expected effect size of d=0.23. Therefore, we predict a significant positive effect. For hypothesis 2b, we predict a positive, directional effect on coefficient a5, because with a sample of 100 decisions made on startups with female CEOs, we would need an expected effect size of 0.33 – which is a larger effect than we expect. We are very unlikely to find a significant effect for the size of public investment that is most likely to change the propensity of investment, but will again receive some directional information. Again, we can causally test these hypotheses further in study 2 with a larger sample.
We will also analyze some key mechanism variables. First, following hypothesis 1b, we predict that the T1 group will weigh certain criteria differently to the control group. In the control group, we predict that investors will more heavily weigh criteria based on attributes of the startup with a male CEO: they will retroactively construct criteria to fit their perception of what a good founder looks like (Uhlmann & Cohen, 2005).
Second, we will analyze what additional information investors would request from startups before making an investment, which will provide some information on mechanisms that might lead more investors to invest. We predict that startups with female CEOs will be asked to provide more information, following work that suggests that investors require more information from startups presented by a female founder (Kanze et al., 2019).
We will measure other potentially important variables at the investor level including the gender of the investor (Greenberg & Mollick, 2018) and experience of the investor (Clingingsmith & Shane, 2018). We will also measure organizational variables about the organization the investor works for, to assess potential directional heterogenous effects of treatment based on: the size of the investment organization; the date it last raised funds (e.g. Hochberg, Serrano, Ziedonis, 2017); its investment thesis i.e. sector and geography; stated interest in gender-based investments; and motivation of the firm to focus on gender (e.g. Ely & Thomas, 2001).
Using this data, we will assess potential directional heterogenous effects of treatment. We predict that some investors would be naturally more likely to invest in female founders: female investors; investors working for an organization with a founder diversity mandate. We predict that these investors would be less likely to be affected by all treatments.
We predict that treatment 2 will be more effective for experienced investors following Clingingsmith and Shane’s finding that experienced investors responded more to an information shock than other investors. We predict that treatment 1 will be more effective for less experienced investors, as those with experience may be more likely to rely on their existing heuristics when thinking about criteria.