|
Field
Abstract
|
Before
This impact evaluation aims to measure the effect of a program that combines business training, mentoring, and a large cash transfer on high-potential small and medium businesses in Chile. 250 out of the top 500 firms participating in a business plan competition will be randomly selected to receive all three components of the program, while the remaining firms will receive none of them.
In-person surveys with the entrepreneurs will be conducted before and 12 months after the randomization program, which will measure the program's impact on key indicators such as monthly sales, monthly profits, and the number of employees a business has. Other changes, such as the welfare of entrepreneurs, their business practices, and learning, will also be measured. If available, administrative data coming from Chilean tax and unemployment insurance authorities will be used to track the long-term growth with these firms.
|
After
This impact evaluation aims to measure the effect of a program that combines business training, mentoring, and a large cash transfer on high-potential small and medium businesses in Chile. 250 out of the top 500 firms participating in a business plan competition will be randomly selected to receive all three components of the program, while the remaining firms will receive none of them.
In-person surveys with the entrepreneurs will be conducted before and 12 months after the program. The surveys will measure the program's impact on key indicators such as monthly sales, monthly profits, and the number of employees in the business. Other outcomes, such as the welfare of entrepreneurs, their business practices, and learning, will also be measured. In addition to the survey data, we will use available administrative data coming from Chilean tax and unemployment insurance authorities to analyze the firm’s profits, sales, costs and networks. The tax data will also be used to track the long-term growth of these firms. Finally, we use data from the videos submitted by the entrepreneurs as a part of the selection process to study potential treatment heterogeneities.
|
|
Field
Last Published
|
Before
November 30, 2022 05:09 PM
|
After
December 14, 2023 01:57 PM
|
|
Field
Intervention (Public)
|
Before
Impulso Chileno is a program created by a Chilean NGO to help high-potential Chilean firms grow faster by giving them business training, one-on-one mentoring, and a sizable monetary transfer. This program has been implemented yearly since 2018, and its fifth iteration will be evaluated with a randomized control trial.
The NGO identifies high-potential small and medium enterprises (SMEs) through a business contest in which entrepreneurs present their business, its financial statements, and a business plan for what they would do with their monetary endowment.
Firms that participate in the contest face a selection process with several stages. Ultimately an evaluation committee will choose the 500 firms they believe show the most promise. 250 of these firms will be randomly assigned to the treatment group, while the rest will form a control group.
The treatment group will then receive the three primary components of the program:
(1) A monetary transfer:
- This cash transfer can range between 3 and 5 million Chilean pesos (approximately USD 3,500 to USD 5,800).
- The average value of the transfer is USD 5,300. This transfer can only be used to buy physical assets, working capital, digital tools, and fund marketing campaigns.
(2) A business training program:
- The training program consists of 30 hours of online, group-based, real-time business training sessions.
- The Business Administration School of the Pontificia Universidad Católica de Chile (PUC Chile) imparts this program.
(3) Individual mentoring:
- At least eight one-on-one mentoring sessions, which have a free-form structure and usually last about an hour.
- The mentors usually lead or work for larger Chilean companies in the same sector as the mentees. They are volunteers from Corporación Simon de Cirene’s mentoring network.
Meanwhile, the control group will receive none of these components. However, we plan on offering three optional online talks to entrepreneurs on both the treatment and control groups throughout the year to maintain a base level of engagement with both groups and minimize the risk of attrition bias. The proposed subjects for these talks are (1) empowerment, (2) business experiences and (3) work-life balance. We expect these talks to not have a significant impact on outcome variables, but we will keep track of entrepreneur’s attendance to the events in order to control for them.
|
After
Impulso Chileno is a program created by a Chilean NGO aimed to help high-potential Chilean firms grow faster by giving them business training, one-on-one mentoring, and a sizable monetary transfer. This program has been implemented yearly since 2018, and its fifth iteration will be evaluated with a randomized control trial.
The NGO identifies high-potential small and medium enterprises (SMEs) through a business plan competition in which entrepreneurs present their business, its financial statements, and a business plan detailing how they would invest the monetary transfer.
Firms that participate in the contest face a selection process with several stages. Ultimately, an evaluation committee will choose the 500 firms they believe show the most promise. During this year of the program, 250 of the 500 finalist firms will be selected at random from and assigned to the treatment group; the remaining 250 finalists will form a control group.
The treatment group will receive the three primary components of the program:
A monetary transfer
This cash transfer can range between 3 and 5 million Chilean pesos (approximately USD 3,500 to USD 5,800). The average value of the transfer is USD 5,300.
This transfer can only be used to buy physical assets, working capital, digital tools, and fund marketing campaigns.
A business training program
The training program consists of 30 hours of online, group-based, real-time business training sessions.
The Business Administration School of the Pontificia Universidad Católica de Chile (PUC Chile) delivers this program.
Individual mentoring.
At least eight one-on-one mentoring sessions, which have a free-form structure and usually last about an hour.
The mentors usually lead or work for larger Chilean companies in the same sector as the mentees. They are volunteers from Corporación Simon de Cirene’s mentoring network.
Additionally, two optional online talks were offered to entrepreneurs in both the treatment and control groups throughout the year to maintain a base level of engagement with both groups and minimize the risk of attrition bias. The subjects for these talks were: (1) empowerment, (2) business experiences and (3) work-life balance.
|
|
Field
Primary Outcomes (Explanation)
|
Before
Sales (survey)
- Monthly Sales for November - Estimated using three different approaches, then verified by the entrepreneur.
- Other Monthly Sales - Directly asked from August to October.
- Yearly Sales - An estimate of the sales in the last 12 months.
Costs (survey)
- Monthly Costs per Item - Asked how much money was spent during November for the following items: merchandise, payment of infrastructure services (electricity, water, etc.), gasoline, payment of debts and interest rates, cost of salaries and wages, renting equipment, establishment’s rent, cost of telecommunication services, tax payments, repair and maintenance of equipment, professional services (ie. lawyers, accountants), transportation costs and others.
Profits (survey)
- Monthly Profits - Profits for November are calculated by subtracting monthly costs from their estimated monthly sales. This calculation is done via survey software and shown to the entrepreneur so they can verify it. Changes are made to each sub-section if the entrepreneur sees a problem with the final result.
- Yearly Profits - Directly estimated by the entrepreneur taking into account the last 12 months.
Number of Employees
- Number of Employees - how many employees did the business have during November. An employee is someone who works in their business and receives a recurring salary or wage.
- Number of Full-Time and Part-Time Employees - how many of the employees are full-time, and how many hours a week do the part-time employees work relative to a typical 44 hour a week full-time job.
- Number and Percent of Formal Employees - how many of these employees are formal employees. By this, we mean that they are currently subscribed to social security and health care systems through this business.
- Number and Percent of Related Employees - how many of these employees are family members of the entrepreneur?
- Number of unpaid employees - apart from the employees numbered, how many other people help out at the business that does not receive a recurring wage or salary. As an example, we can think of the son of an entrepreneur who is involved in the business but doesn’t get paid as a helper.
Tax data (if available)
- Yearly reported profit
- Yearly reported cost
- Yearly reported sales
- Unemployment insurance data
- Number of formal employees
|
After
Monthly sales, costs, and profits from survey data; Number of employees and type (family members, formal employees, full time vs part-time employment, unpaid helpers). From tax data: sales, taxable profits, costs. From unemployment insurance data: number of formal employees. We will also report the Anderson Index of these outcomes.
From survey data
Sales
Monthly Sales for November
Yearly Sales - An estimate of the sales in the last 12 months.
Costs
Monthly Costs per Item - Asked how much money was spent during November for the following items: merchandise, payment of infrastructure services (electricity, water, etc.), gasoline, payment of debts and interest rates, cost of salaries and wages, renting equipment, establishment’s rent, cost of telecommunication services, tax payments, repair and maintenance of equipment, professional services (i.e. lawyers, accountants), transportation costs and others.
Profits
Monthly Profits - Profits for November are calculated by subtracting monthly costs from their estimated monthly sales. This calculation is done via survey software and shown to the entrepreneur, so they can verify it. Changes are made to each sub-section if the entrepreneur sees a problem with the final result.
Yearly Profits - Directly estimated by the entrepreneur, taking into account the last 12 months.
Number of Employees
Number of Employees - how many employees did the business have during November. An employee is someone who works in their business and receives a recurring salary or wage.
Number of Full-Time and Part-Time Employees - how many of the employees are full-time, and how many hours a week do the part-time employees work relative to a typical 44 hour a week full-time job.
Number and Percent of Formal Employees - how many of these employees are formal employees. By this, we mean that they are currently subscribed to social security and health care systems through this business.
Number and Percent of Related Employees - how many of these employees are family members of the entrepreneur?
Number of unpaid employees - apart from the employees numbered, how many other people help out at the business that does not receive a recurring wage or salary. As an example, we can think of the son of an entrepreneur who is involved in the business but doesn’t get paid as a helper.
From tax data
Tax data for 2021 and 2022 for baseline, and 2023 for follow-up. If there is an impact in 2023, we will study the impact in 2024, then up to 2029 and furthermore 2033-34.
Yearly reported profit
Yearly reported cost
Yearly reported sales
Number of employees
|
|
Field
Experimental Design (Public)
|
Before
The Impulso Chileno program is divided into two stages. The first stage consists of a business plan competition to identify high-potential Chilean small and medium enterprises (SMEs). The program's second stage is where the top-rated firms from the competition receive the intervention itself: a business training course, individual mentoring, and a large cash transfer to invest in their businesses.
Our impact evaluation will randomly assign the top 500 firms participating in the competition into treatment and control groups to measure the program’s effect on the firm's growth.
Business Plan Competition:
The business plan competition will be held between September 2022 and January 2023. During this period, we expect around 20,000 firms to apply to the contest through an online application that gathers basic firm and entrepreneur-level data and their business plan proposal.
Firms must be formal and have annual sales within a pre-specified bracket [~$7,600 to $380,000 USD] to qualify as a high-potential SME. Firms that do not comply with these prerequisites are disqualified. Firms who fit these criteria will be scored based on their business model, entrepreneur’s profile, and business plan proposal.
The top-rated 2,500 firms will then be asked to submit tax statements for the previous four months and a 3-minute video explaining their business model and business plan. An evaluation committee (formed by representatives of the implementation partners) will watch the videos, review the documents, and re-rank the contenders to select the top 700 of them as semi-finalists.
Each semi-finalist will have a 30-minute interview with members of the evaluation committee, where entrepreneurs will present their business plan proposal. Based on this interview, the 700 firms will be re-ranked once again to find the top 500 finalists for the program.
Randomized Assignment:
Toward the end of January 2023, 250 finalists will be randomly assigned to a treatment group, while the rest will form a control group. Between February and July 2023, the treatment group will receive the three components of the program: a monetary transfer, business training and mentoring. Meanwhile, the control group will receive none of these components.
This randomization will be stratified based on the entrepreneur’s gender and the macro-zone from where the business is operated from. A macro-zone is an aggregation of different Chilean regions (analogous to US states), which split the country into 6 distinct areas; North, Center, Center-South, South and Austral Macrozones, along with the Metropolitan region of Santiago de Chile.
Data Collection:
In-person baseline surveys will be carried out at each finalist's place of work throughout December 2022 and January 2023, before the randomized selection process is carried out (N=500). Follow-up surveys will be conducted 12 months later. Most of the medium-term effects of the program will be measured using these surveys. We plan to use administrative data from unemployment insurance and tax authorities to measure the long-term impact of Impulso Chileno, if available.
Regression Discontinuity Design
Apart from the randomized control trial involving the top 500 firms, we will also survey and gather data from the next 200 firms who did not reach the business plan competition’s threshold to become finalists. We plan on using the scores and ranking within the business plan competition as the running variable in a regression discontinuity design, comparing the 200 non-finalist with the 250 control group firms. This data will be used to evaluate the contest’s effectiveness in identifying top firms and comparing the program’s impact on the average firm to those firms closer to the cut off.
|
After
The Impulso Chileno program is divided into two stages. The first stage consists of a business plan competition to identify high-potential Chilean small and medium enterprises (SMEs). The program's second stage is where the top-rated firms from the competition receive the intervention itself: a business training course, individual mentoring, and a large cash transfer to invest in their businesses.
Our impact evaluation randomly assigned the top 500 firms participating in the competition into treatment and control groups to measure the program’s effect on the firm's growth.
Business Plan Competition:
The business plan competition was held between September 2022 and January 2023. During this period, 21,780 firms applied to the contest through an online application that gathers basic firm and entrepreneur-level data and their business plan proposal.
Firms had to be formal and have annual sales within a pre-specified bracket [~$7,600 to $380,000 USD] to qualify as a high-potential SME. Firms that did not comply with these prerequisites were disqualified. Firms that fit these criteria were scored based on their business model, entrepreneur’s profile, and business plan proposal.
The top-rated 2,500 firms were then asked to submit tax statements for the previous four months and a 3-minute video explaining their business model and business plan. An evaluation committee (formed by representatives of the implementation partners) watched the videos, reviewed the documents, and re-ranked the contenders to select the top 700 of them as semi-finalists.
Each semi-finalist had a 30-minute interview with members of the evaluation committee, where entrepreneurs presented their business plan proposals. Based on this interview, the 700 firms were re-ranked once again to find the top 500 finalists for the program.
Randomized Assignment:
Toward the end of January 2023, 250 finalists were randomly assigned to a treatment group, while the rest were assigned to the control group. Between March and July 2023, the treatment group received the three components of the program: a monetary transfer, business training and mentoring. Meanwhile, the control group received none of these components.
This randomization was stratified based on the entrepreneur’s gender and the macro-zone from where the business is operated. A macro-zone is an aggregation of different Chilean regions (analogous to US states), which split the country into 6 distinct areas; North, Center, Center-South, South, and Austral Macrozones, along with the Metropolitan region of Santiago de Chile.
Data Collection:
In-person baseline surveys were carried out at each finalist's place of work throughout December 2022 and January 2023, before the randomized selection process was carried out (N=500). Follow-up surveys will be conducted 12 months later. Most of the medium-term effects of the program will be measured using these surveys. We plan to use administrative tax data to measure the short and long-term impact of Impulso Chileno.
Impact Evaluation - OLS:
We will estimate the average treatment effects of the program by conducting an Ordinary Least Squares (OLS) regression, where the assignment variable will serve as the key predictor. To enhance the precision of our analysis, we will employ Machine Learning algorithms, specifically, Post-Double Selection LASSO to select controls in the main regressions; and causal forests to study heterogeneous effects, thereby adopting a data-driven approach.
In specific, regarding causal forests, we will study treatment heterogeneities estimating the Conditional Average Treatment Effect (CATE) including, in the causal forest, the following baseline characteristics: entrepreneurs’ gender, age of the entrepreneur, entrepreneur’s education level, business sector, years since business creation, and region of operation. Furthermore, we will use the predictions on the expected treatment effect for each individual, given the covariates, to investigate treatment heterogeneity.
We will divide the sample in two groups, that is, top and bottom half of the predictions. Then, we will report the balance test for the CATE and p-values adjusted for multiple hypothesis testing. Finally, using that input, we will further detect covariate differences between the two groups to know which are the groups that benefit the most from participating in the Impulso Chileno Program.
To increase power, we will control for the baseline value of dependent variables if available. We will also control by strata and survey specific differences (such as in-person or virtual follow-up).
We will also analyze (without the experimental variation) the program implementation (type of investment funded, attendance to training sessions).
Note that in a companion paper (Assessing Entrepreneurship Potential: Evaluating Video Submissions for Selection Process) we will analyze how applicant’s videos affect the selection process.
|
|
Field
Randomization Method
|
Before
The randomization process will be made through a series of 12 public lotteries. Each of the lotteries corresponds to a different strata given our stratification design. For example, there will be a lottery for female entrepreneurs that hail from the Austral macro-zone, where half of the lottery’s participants will be chosen at random to participate in the treatment group. The main implementing partner intends to live-stream this event and to have a public notary certify the transparency and fairness of it.
|
After
The randomization process was made through a series of 12 public lotteries. Each of the lotteries corresponded to different strata given our stratification design. For example, there was a lottery for female entrepreneurs that hail from the Austral macro-zone, where half of the lottery’s participants were chosen at random to participate in the treatment group. The main implementing partner lived-streamed this event and had a public notary to certify the transparency and fairness of it.
|
|
Field
Sample size (or number of clusters) by treatment arms
|
Before
250 firms in the treatment group, 250 firms in the control group.
200 firms below the ranking threshold for randomization, which receive no treatment components.
|
After
250 firms in the treatment group, 250 firms in the control group.
|
|
Field
Additional Keyword(s)
|
Before
|
After
Microentrepreneurship
|
|
Field
Secondary Outcomes (End Points)
|
Before
Business assets, networks, and business practices knowledge. The number of hours worked by the entrepreneur; entrepreneur’s compensation. If available from tax data: transactions, number of vendors/buyers.
|
After
Business assets, networks, and business practices knowledge. The number of hours worked by the entrepreneur; entrepreneur’s compensation. From tax data: transactions, number of vendors/buyers.
|
|
Field
Secondary Outcomes (Explanation)
|
Before
Entrepreneur’s...
- Compensation - Track whether or not the entrepreneur gives himself a salary and its amount. Include business profits withdrawals made with other than monthly frequency.
- Weekly Hours Worked - How many hours a day does the entrepreneur work on his business for each day of the week. An indicator variable is created if the sum total of hours worked surpass 48 hours, which is the mark at which the ILO defines excessive working hours.
- Savings - Number of weeks of household consumption the entrepreneur could afford with the amount of money he has saved.
- Access to Healthcare and Social Security - Check whether the entrepreneur is currently subscribed to healthcare and social security services and if these are public or private.
Business Practices:
- Digital Tools and Online Presence - Indicator based on whether or not the entrepreneur partakes in a number different business practices related to the areas of usage of digital tools for marketing and record-keeping, along with presence in different social networks and online stores.
- Marketing - Indicator based on whether or not the entrepreneur partakes in 7 different business practices related to the areas of Marketing
- Record Keeping - Indicator based on whether or not the entrepreneur partakes in 8 different business practices related to the areas of record keeping
- Financial Planning - Indicator based on whether or not the entrepreneur partakes in 8 different business practices related to the areas of financial planning
- Stock Control - Indicator based on whether or not the entrepreneur partakes in 3 different business practices related to the areas of buying and keeping control of stock.
Learning:
- The Business Administration School of PUC Chile developed a 10-question diagnostic test that asks questions on the topics covered during the training sessions. The entrepreneurs will fill out a selection of these questions during the in-person surveys to measure changes in learning about business administration throughout the program
Business Assets:
- Current Business Assets
Networks
- Number of people the entrepreneurs can contact for business-related questions
Monitoring Data
- Spending on Investments and Capital - Entrepreneurs assigned to the treatment group need to send all invoices to the Luksic Foundation to ensure all purchases with the money handed out were made in the pre-approved categories. We plan to create a database with these invoices to complement our analysis and see which types of investments were the most common and most effective.
- Attendance to Training Sessions - We will keep track of individual entrepreneurs’ assistance to the Training Sessions to complement our analysis.
- Topics Covered during Mentoring Sessions - Mentors will fill out a form each time a mentoring session happens. In this form, we will know the session's length and the topics covered during that time. We aim to use this information to complement our analysis and see which topics were considered the most useful for entrepreneurs.
Tax data (if available)
- Number of vendors: number of firms from where the firm buys in the last year
- Number of buyers: number of different buyers in the previous year
- Number of transactions: number of sales and buys in the previous year
|
After
From survey data
Entrepreneur’s
Compensation - Track whether the entrepreneur gives himself a salary and its amount. Include business profits withdrawals made with other than monthly frequency.
Weekly Hours Worked - How many hours a day does the entrepreneur work on his business for each day of the week. An indicator variable is created if the sum total of hours worked to surpass 48 hours, which is the mark at which the ILO defines excessive working hours.
Savings - Number of weeks of household consumption the entrepreneur could afford with the amount of money he has saved.
Access to Healthcare and Social Security - Check whether the entrepreneur is currently subscribed to healthcare and social security services and if these are public or private.
Business Practices
Digital Tools and Online Presence - Indicator based on whether the entrepreneur partakes in a number of different business practices related to the areas of usage of digital tools for marketing and record-keeping, along with presence in different social networks and online stores.
Marketing - Indicator based on whether the entrepreneur partakes in 7 different business practices related to the areas of Marketing
Record Keeping - Indicator based on whether the entrepreneur partakes in 8 different business practices related to the areas of record keeping
Financial Planning - Indicator based on whether the entrepreneur partakes in 8 different business practices related to the areas of financial planning
Stock Control - Indicator based on whether the entrepreneur partakes in 3 different business practices related to the areas of buying and keeping control of stock.
Learning
The Business Administration School of PUC Chile developed a 10-question diagnostic test that asks questions on the topics covered during the training sessions. The entrepreneurs will fill out a selection of these questions during the in-person surveys to measure changes in learning about business administration throughout the program
Business Assets
Current Business Assets
Networks
Number of people the entrepreneurs can contact for business-related questions
From Tax data:
Number of vendors: number of firms from where the firm buys in the last year
Number of buyers: number of different buyers in the previous year
Number of transactions: number of sales and buys in the previous year
Additional Data From Implementing Organization
Spending on Investments and Capital - Entrepreneurs assigned to the treatment group need to send all invoices to the Luksic Foundation to ensure all purchases with the money handed out were made in the pre-approved categories. We plan to create a database with these invoices to complement our analysis and see which types of investments were the most common and most effective.
Attendance to Training Sessions - We will keep track of individual entrepreneurs’ assistance to the Training Sessions to complement our analysis.
Topics covered during Mentoring Sessions - Mentors will fill out a form each time a mentoring session happens. In this form, we will know the session's length and the topics covered during that time. We aim to use this information to complement our analysis and see which topics were considered the most useful for entrepreneurs.
This additional data will be used for compliance (the first two) or give qualitative data that might be useful in unpacking channels (the third)
|