Market-based Information to Predict Small Firms’ Marginal Returns to Capital and Other Business Support Services

This RCT is part of a larger research project that explores the extent to which information that is valuable for predicting entrepreneurs’ future outcomes is embedded in business networks. By business networks, we mean networks of entrepreneurs, suppliers of entrepreneurs, and customers of entrepreneurs. The research project explores information related to both entrepreneurs’ future and past absolute and relative performance and marginal returns (MR) to capital, managerial consulting, and networking with other businesses.

We are conducting this research project as part of a related study that uses a RCT to evaluate the impact of the Enterprise Grow Programme, a large-scale business support program being implemented by the Government of Ghana’s Ghana Enterprises Agency (GEA) and the World Bank. In the RCT, small enterprises (sized 6-30 employees), are randomly assigned to grant, group-based managerial consulting, group-based peer learning, and control conditions. The RCT enables us to obtain unbiased estimates of the MRs to these interventions across the distribution of predicted returns, allowing us to test the accuracy of MR predictions, and to measure the predictive power of information gathered about entrepreneurs’ absolute and relative performance.

In the current study, we leverage the fact that the Enterprise Grow Programme brings together entrepreneurs for four days of general business training before conducting a final screening of applicants to the program. Among those who pass the final screening, we conduct a baseline survey and organize a peer networking event with the GEA. In the networking event, we group entrepreneurs in groups of 6 and conduct one-on-one peer networking. At the end of the event, we conduct a survey in which entrepreneurs make absolute and relative predictions about their and their group members’ past and future performance.

We conduct a RCT for a subset of questions in the survey; we use randomized assignment to high- and low-stakes to examine how incentives to strategically misreport to increase one’s chances of receiving a grant through the Enterprise Grow Programme affect the accuracy of reports. We hypothesize that entrepreneurs will distort their predictions to favor themselves in the high stakes condition; as predictions should be unambiguously worse (less accurate) in the high stakes condition compared to the control, we will conduct a one-sided hypothesis test.

We then explore the extent to which two simple tools from mechanism design that aim to improve the accuracy of reports can help: the first is a split-sample ranking mechanism, and the second is payments for accuracy. In the split sample ranking mechanism, entrepreneurs’ rankings of themselves cannot influence the allocation of the grant. Given this, we expect rankings in the stakes condition to be weakly more accurate when this mechanism is used; as predictions should be weakly better (more accurate) in the split sample mechanism compared to the stakes only condition, we will conduct a one-sided hypothesis test.

The main experimental conditions are:

1. Control:
a. No stakes: Respondents informed that this survey is for research and learning purposes only.
b. No split sample ranking.

2. Stakes only:
a. Respondents informed that the peer ranked highest by others in their assigned group will have a greater chance of obtaining a grant through the Enterprise Grow Programme.
b. No split sample ranking.

3. Stakes + split sample ranking:
a. Respondents informed that the peer ranked highest by others in their assigned group will have a greater chance of winning the grant.
b. Split sample ranking: Respondents informed that the split sample ranking mechanism will be used.

Within these 3 groups, we cross-randomize payments for accurate prediction for a subset of questions in the survey. In the control condition for the payments for accuracy, we do not provide any payments for accuracy. In the treatment condition, we provide a payment to the group member whose ranking is closest to the true ranking among group members. We expect rankings in the payments condition to be weakly more accurate when this mechanism is used; as predictions should be unambiguously better (more accurate) in the payments condition compared to the no payments condition, we will conduct a one-sided hypothesis test.

2024-06-18

2024-12-31

In the survey, the 3 experimental conditions (control, stakes only, stakes + split sample) are introduced before six questions in which entrepreneurs rank their group members:
a. Total sales in 2023;
b. Future sales growth (growth rate in terms of sales in next 12 months);
c. Total profit in 2023;
d. Future profit (total profit in next 12 months);
e. Number of full-time employees at time of baseline survey;
f. Future number of full-time employees (number of employees measured in 12 months).

To reduce the number of hypothesis tests for primary outcomes, our main specifications will group these questions into two primary outcomes:
1. Past performance (questions a, c, and e);
2. Future performance (questions b, d, and f).

For comparability across outcomes, the variables will be percentilized prior to grouping.

The payments for accuracy condition is only applied for the past performance questions (a, c, and e), so we expect it may only be relevant for outcome #1. It’s possible, though, that the payments for accuracy treatment generates spillovers to questions b, d, and f, so we will also present results for this outcome.

We will also report results separately for each question a-f under Primary Outcomes.

Prior to randomization, we assign entrepreneurs to peer groups for the networking sessions. The target group size is 6, although there will be groups sized smaller than 6. To the extent possible, we form groups based on entrepreneurs’ cohort in the four-day general business training. We combine groups of entrepreneurs across cohorts based on similar locations.

We randomize entrepreneurs to the experimental conditions in the following steps:
1. Stakes vs. no stakes: Stratified, cluster-based randomization by group. Stratification is done by geographic region.
2. Split-sample treatment: Stratified, individual-level randomization within stakes groups-only. Stratification is done by group. Within group, 50% of entrepreneurs will be assigned to stakes.
3. Payments for accuracy treatment: Stratified, individual-level randomization. Stratification is done by group x treatment condition. Within group x treatment condition, 67% of entrepreneurs will be assigned to the payments for accuracy treatment.

We plan to conduct heterogeneous treatment effect (HTE) analysis by the strength of entrepreneurs’ self-reported social ties with their group members. Conceptually, we are interested in this dimension of heterogeneity because we hypothesize that the primary incentive for entrepreneurs to strategically misreport in the ranking exercise in the stakes condition is to improve their own chance of obtaining a grant. If entrepreneurs are socially-connected, though, and have preferences for their socially-connected peers to obtain the grant, it’s possible that they may strategically misreport to increase the chances of the group members with whom they are socially connected obtaining a grant. The split-sample mechanism that we consider does not help to address this type of strategic misreporting concern. We will also implement a data-driven approach to testing for HTEs using machine learning methods from (Chernozhukov et al., 2020).

We will also explore whether the predictive accuracy of reports can be improved by weighting reports by group members’ characteristics, such as their trustworthiness (as reported by their group members).

Not available

The randomization is done by a computer (using a Stata do-file with a seed drawn from random.org).

For the stakes condition, the unit of randomization is the group of up to 6 entrepreneurs formed for the peer networking session. For the split-sample and payments for accuracy mechanisms, the unit of randomization is the individual entrepreneur.

Yes

We expect 88 – 92 clusters. The final count may be outside of this range for two reasons: first, the Enterprise Grow Programme is being rolled out in batches, and we do not know the size of the final batch. We assume the same batch size in the third and fourth (and final) batches to estimate the number of clusters. Second, we need to create groups of entrepreneurs at the peer networking sessions, and take-up of the sessions will not be 100%; as such, we may not always be able to create groups of entrepreneurs (our protocol allows for smaller groups, but this might not always be possible).

The expected number of firms is 210 firms for batch 3, and another 210 firms for batch 4. For batch 3, 262 firms will be invited, but the actual number of firms will be lower, due to non-participation as well as difficulties in forming groups that consist of at least two entrepreneurs.

For the stakes condition, clusters will be assigned to the stakes condition with 67% probability.
For the split-sample mechanism, individuals in stakes groups will be assigned to the split-sample mechanism with 50% probability.
For the payments for accuracy mechanism, individuals will be assigned to this condition with 67% probability.

We conducted power calculations using the data from our pilot entrepreneur peer prediction survey to inform our assumptions. The outcome in the power calculation is the mean percentile ranking on profits and employees from the pilot survey. For the stakes versus no stakes condition, we assume: 90 clusters of size 6 (or in some specifications, 3 in the treatment condition), with 33% of clusters assigned to the control group. We assume 80% power and 5% statistical significance level. We assume an intraclass correlation coefficient of 0.05. The mean and SD for the control group are based on the pilot survey. We assume a one-sided hypothesis test in the lower direction (i.e., testing that accuracy of rankings in the stakes conditions will be lower than accuracy in the no stakes condition). With these assumptions, we are powered to detect a decrease in accuracy of 9.2% (8.2%) of a standard deviation (sd) for cluster size 3 (6) in treatment condition. For the stakes only versus stakes + split sample condition, we conservatively assume only use the participants in the stakes groups. We maintain the same assumptions for power, statistical significance, and mean and SD of the control group. We assume 360 participants, with equal-sized treatment and control groups. We assume a one-sided hypothesis test in the upper direction (i.e., testing that accuracy of rankings in the stakes + split sample condition will be higher than accuracy in the stakes only condition). With these assumptions, we are powered to detect an increase in accuracy of 8.5% of a sd. We expect that our actual power may be higher for two reasons: We will stratify the randomized assignment in both cases, and we will include control variables selected by the post double selection lasso (PDS lasso) procedure, which will explain some of the residual variation in the outcome. In addition, we assume a conservative level of take-up for the peer networking sessions; if take-up is higher than expected, our sample size will be larger than assumed in the calculations.