|
Field
Trial Status
|
Before
in_development
|
After
on_going
|
|
Field
Abstract
|
Before
Many small businesses have already been deeply affected by the COVID-19 crisis, leading them to lay off employees, temporarily close shop, or go out of business completely. In response, governments have passed specific provisions to help support small businesses as part of large aid packages. However, we document in a baseline survey that many small businesses may not understand who is eligible for aid, what the aid is available or how to apply. This project evaluates the roles of information frictions on small businesses’ utilization of newly available government assistance. To begin, we survey small businesses in the United States and Latin America about how they have been affected by the COVID-19 crisis. Next in the first round of the pilot outreach program, we randomly assign small businesses to informational interventions with different levels of intensity. Businesses will then be followed-up to see if they successfully applied for aid, and how the aid affected layoffs and expectations of going out of business. Using the experimental design, we will study the role of information frictions on differences in applying for aid by firm size, and how that aid affects business survival. The more effective measures will then be scaled up if any are found to be cost-effective.
|
After
Many small businesses have already been deeply affected by the COVID-19 crisis, leading them to lay off employees, temporarily close shop, or go out of business completely. In response, governments have passed specific provisions to help support small businesses as part of large aid packages. However, we document in a baseline survey that many small businesses may not understand who is eligible for aid, what aid is available or how to apply. This project evaluates the roles of information frictions on small businesses’ utilization of newly available government assistance. To begin, we survey small businesses in the United States and Latin America about how they have been affected by the COVID-19 crisis. Next in the first round of the pilot outreach program, we randomly assign small businesses from eight Latin American countries to informational interventions with different levels of intensity. Businesses will then be followed up to see if they successfully applied for aid, and how the aid affected expectations, layoffs and probability of going out of business. Using the experimental design, we will study the role of information frictions on differences in applying for aid by firm size, and how that aid affects business survival. The more effective measures will then be scaled up if any are found to be cost-effective.
|
|
Field
Last Published
|
Before
April 02, 2020 12:16 PM
|
After
July 29, 2020 02:02 AM
|
|
Field
Intervention (Public)
|
Before
We survey small businesses in the United States and Latin America about how they have been affected by the COVID-19 crisis. Next, we randomly assign small businesses to different informational interventions with different levels of intensity. Businesses will then be followed-up to see if they successfully applied for aid, and how the aid affected layoffs and expectations of going out of business. Using the experimental design, we will study the role of information frictions on differences in applying for aid by firm size, and how that aid affects business survival. The more effective measures will then be scaled up in a second-round if any are found to be cost-effective.
|
After
We survey small businesses in Latin America about how they have been affected by the COVID-19 crisis. Next, we randomly assign small businesses to different informational interventions with different levels of intensity. Businesses will then be followed up to see if they successfully applied for aid, and how the aid affected owner's expectations, layoffs and probability of going out of business. Using the experimental design, we will study the role of information frictions on differences in applying for aid by firm size, and how that aid affects business survival. The more effective measures will then be scaled up in a second round if any are found to be cost-effective.
|
|
Field
Intervention Start Date
|
Before
April 08, 2020
|
After
April 22, 2020
|
|
Field
Intervention End Date
|
Before
May 31, 2020
|
After
July 17, 2020
|
|
Field
Primary Outcomes (End Points)
|
Before
Take up of government aid, layoffs, and firm survival.
|
After
1. Firm survival, 2. Number of layoffs
|
|
Field
Primary Outcomes (Explanation)
|
Before
The main outcomes are the number of workers laid off. These will be determined using a followup survey.
|
After
1. Firm survival: whether the firm is still open (or temporarily closed but plans to reopen). It will be measured using a follow up survey, and eventually administrative data.
2. Number of workers laid off: this is the number of full-time equivalent workers (full-time + 0.5*part-time) laid off since January (i.e. pre-covid). It will be determined using a follow up survey, and eventually administrative data. Notes on how we count jobs lost in case of business closure:
If the firm had 0 employees in January and it closes, then this counts as 1 job lost (the owner’s)
If the firm had N full-time equivalent employees in January, and it closes, then that is N+1 jobs lost
|
|
Field
Experimental Design (Public)
|
Before
Small businesses are recruited through a baseline survey conducted through social media. A random subsample is chosen to be assigned to treatment "intense assistance", another is assigned to "intermediate assistance", and the last group will be kept as control.
|
After
Small businesses are recruited through social media to take our baseline survey conducted in Qualtrics. Those who share contact information with us are included in the experiment sample. Firms are randomly assigned to the "low-intensity" information intervention group, or to the "high-intensity" information intervention group. In Mexico, there is also a "medium-intensity" information intervention group. Randomization is stratified by country and date of response.
|
|
Field
Randomization Method
|
Before
The sample is stratified by country, state, size of the firm, prior stated beliefs and layoffs at baseline. Randomization will be done by a computer.
|
After
Randomization was done by a computer. It was stratified by country and date of response. For firms in Mexico, we further stratified by State, as there was substantial variation in policies across states.
|
|
Field
Randomization Unit
|
Before
The firm.
|
After
The firm
|
|
Field
Planned Number of Clusters
|
Before
5000 firms
|
After
To assess the impact of T2 vs. T1 the sample size is 10,895 firms (Sample 1). To assess the impact of T3 vs. T1 the sample size is 2,804 firms (Sample 2). Finally, with the 3,334 firms from Mexico (Sample 3) we can test both T4 vs. T1, and T4 vs. T2. As per our intervention description, a group of Mexican firms belong to both Sample 1 and Sample 3.
Overall, there are 15,367 firms in our experiment sample. More details can be found in the Pre Analysis Plan
|
|
Field
Planned Number of Observations
|
Before
5000 firms
|
After
To assess the impact of T2 vs. T1 the sample size is 10,895 firms (Sample 1). To assess the impact of T3 vs. T1 the sample size is 2,804 firms (Sample 2). Finally, with the 3,334 firms from Mexico (Sample 3) we can test both T4 vs. T1, and T4 vs. T2. As per our intervention description, a group of Mexican firms belong to both Sample 1 and Sample 3.
Overall, there are 15,367 firms in our experiment sample. More details can be found in the Pre Analysis Plan
|
|
Field
Sample size (or number of clusters) by treatment arms
|
Before
1666 firms as control, 1666 firms receive the information in a one on one conversation, 1666 firms receive the information through e-mail.
|
After
Email (T1): 8,921 firms
Email + Phone session + Chatbot (T2): 3,252 firms
Email + Phone session (T3): 1,526 firms
Email + Chatbot (T4, only in Mexico): 1,668 firms
|
|
Field
Power calculation: Minimum Detectable Effect Size for Main Outcomes
|
Before
|
After
The Pre Analysis Plan provides a more comprehensive account of the power calculations we have performed. We list here the Minimum Detectable Effects for T1 vs T2. Consider the following to interpret the MDE calculations:
For all calculations, our alpha was set at the standard 0.05 level, power was set at 0.8. and we considered critical values for a one-sided test.
For each outcome, we show (where possible -and relevant) the baseline mean of the control group and the Minimum Detectable Effect of our intervention.
Some outcomes (Business is open, Business applied for help, Business got help) do not have a measure in the world without COVID, so we calculate MDEs with respect to 3 possible scenarios for the control mean.
For the outcome “jobs lost from January to follow up”, we also make an assumption about the jobs that will be lost in the control group. We know the average number of jobs lost in the control group from January to the moment of the baseline. The approximation we use is 1.5 times this number.
In general, we expect the take up of the treatment to be 60% in the treatment group (0.6 compliance) and 0% in the control group (0% spillover). We adjust our MDEs taking this into account (multiplying by 1/0.6 = 1.66)
MDE calculations T1 vs T2 in 5 countries, primary outcomes:
(1) Business is open, MDE relative to 3 possible values for control mean
if control mean = 0.98, MDE = 0.0111
if control mean = 0.95, MDE = 0.0179
if control mean = 0.90, MDE = 0.0251
(2) Number of jobs lost from baseline to follow up
control mean (assumed) = 3.5, MDE = -0.0864
(3) Proportion of owners aware of programs
control mean = 0.21, MDE = 0.0357
(4) Proportion that applied for help, MDE relative to 2 possible values for control mean
if control mean = 0.1, MDE = 0.0267
if control mean = 0.15, MDE = 0.0315
(5) Proportion that got help, MDE relative to 2 possible values for control mean
if control mean = 0.03, MDE = 0.0156
if control mean = 0.06, MDE = 0.0213
|
|
Field
Additional Keyword(s)
|
Before
COVID-19, CARES Act, Financial relief programs, Small business, Latin America
|
After
COVID-19, Financial relief programs, Small business, Latin America
|
|
Field
Intervention (Hidden)
|
Before
The interventions include a one on one conversation about the new policies that the small business may be eligible for and resources on how to find out more information and apply. This counselor will guide the firm owner thought the steps needed to apply and access to the programs. Another less intense treatment consists of sending emails with the information and links to more details.
|
After
To evaluate whether the access to information can increase the take-up of government relief programs and change expectations and employment outcomes in small firms, we proposed an experimental design in which each participant firm was randomly assigned to receive one or more of the following: (i) an informational email containing a description of their country’s current policies, (ii) a one-on-one phone session with a trained staff member in which business owners are walked through the characteristics of the policies, and/or (iii) support from a digital assistant (chatbot) that tries to replicate the experience of the one-on-one telephone informational session.
In Argentina, Bolivia, Brazil, Chile, Colombia, the Dominican Republic, Mexico, and Peru, firm owners were randomly assigned to one of the following arms:
Treatment arm 1: Email
Treatment arm 2: Email + Phone session + Chatbot
Resource constraints prevented us from implementing the chatbot in Argentina, Bolivia, and the Dominican Republic. Firms from those three countries, however, did receive the one-on-one phone session. Therefore, we have (in practice) an “Email + Phone session” arm:
Treatment arm 3: Email + Phone session
Since Mexico was the country with the most respondents, there is an additional treatment arm which consists only of receiving the email and chatbot. This arm was part of the original design: business owners in Mexico were randomly assigned to Treatment arm 1 or Treatment arm 2, previously mentioned, or to:
Treatment arm 4: Email + Chatbot
In sum, there are three comparisons we will be evaluating:
Email (T1) vs. Email + Full information (T2), leveraging our samples from Brazil, Chile, Colombia, Peru, and a subset of the firms from Mexico (N: 10,895)
Email (T1) vs. Email + Phone session (T3), which involves business owners from Argentina, Bolivia, and the Dominican Republic (N:2,804), and
Email (T1) vs. Email + Full information (T2) vs. Email + Chatbot (T4), which involves all Mexican firms (N:3,334).
In the baseline survey we asked business owners if they wanted to be contacted by a NGO to receive information about policies that had been announced in their countries. This offer was intended to increase the likelihood that business owners would share their contact information with the research team. Since we generated these expectations, we decided that everyone who showed willingness to be contacted would receive at least the summary of policies through email. For this reason, we do not have a pure control group.
|
|
Field
Did you obtain IRB approval for this study?
|
Before
No
|
After
Yes
|
|
Field
Secondary Outcomes (End Points)
|
Before
Whether the firm applied for aid. These will be determined using a followup survey.
|
After
1. Awareness about government relief policies, Take up of government aid (2. Whether the firm applied for aid, 3. Whether the firm got aid), 4. Expectations of the owner about recovery
|
|
Field
Secondary Outcomes (Explanation)
|
Before
Whether the firm has filed for bankruptcy or gone out of business. These will be determined using a followup survey.
|
After
2. Whether the firm applied for any of the relief policies available in their country: This will be asked directly in the follow up survey.
3. Whether the firm got aid from such relief policies: This will be asked directly in the follow up survey, and we also expect to determine it by looking at administrative government records of firms that received aid, when they become available.
4. Expectations of the owner about recovery: This will be measured using three questions of the follow up survey
a) Do you think your business will recover in the next two years?
b) How many months do you think it will take for your business to fully recover?, and
c) What is the probability (1-100) that your business will go bankrupt/permanently close in the next 6 months?
|