|
Field
Last Published
|
Before
June 24, 2021 09:42 AM
|
After
March 21, 2022 02:28 PM
|
|
Field
Primary Outcomes (Explanation)
|
Before
Willingness to emigrate is measured through the answer to three questions in cascade:
1. “Ideally, if you had the opportunity, would you like to move permanently to another country, or would you prefer to continue living in this country?”
2. Those who answer yes previously receive a follow-up question about concrete plans to emigrate—“Are you planning to move permanently to another country in the next 12 months, or not?”.
3. Those who answer yes to both the questions about desire and plan are asked a third question about active preparation to emigrate: “Have you done any preparation for this move (for example, applied for residency or visa, purchased the ticket, etc.)?”
• Measuring the answers to question 1: No=0, Yes=1/3
• Measuring the answers to question 2: No=1/3, Yes=2/3
• Measuring the answers to question 3: No=2/3, Yes=1
Scoring the indicator:
• 0= no willingness to emigrate
• 1/3= low willingness to emigrate
• 2/3= moderate willingness to emigrate
• 1= high willingness to emigrate
|
After
Willingness to emigrate is measured through the answer to three questions in cascade:
1. “Ideally, if you had the opportunity, would you like to move permanently to another country, or would you prefer to continue living in this country?”
2. Those who answer yes previously receive a follow-up question about concrete plans to emigrate—“Are you planning to move permanently to another country in the next 12 months, or not?”.
3. Those who answer yes to both the questions about desire and plan are asked a third question about active preparation to emigrate: “Have you done any preparation for this move (for example, applied for residency or visa, purchased the ticket, etc.)?”
• Measuring the answers to question 1: No=0, Maybe=1/6; Yes=1/3
• Measuring the answers to question 2: No=0, Yes=1/3
• Measuring the answers to question 3: No=0, Yes=1/3
Scoring the indicator:
• 0= no willingness to emigrate
• 1/6 to 1/3= low willingness to emigrate
• 1/3 to 2/3= moderate willingness to emigrate
• 2/3 to 1= high willingness to emigrate
|
|
Field
Experimental Design (Public)
|
Before
A) Main research question
Do vocational training programs improve labour market outcomes, leading to a decrease in willingness to emigrate, in rural areas of low-income countries?
B) Timeline
Exact date – baseline data collection:
• August-September 2020: design of the evaluation
• September-October 2020: promotion of the courses between potential eligible candidates
• September-October 2020: checking for candidates’ eligibility and registering consent, running the baseline survey and assigning the selected candidates to the treatment and control groups.
• November 2020: running the market study survey to contribute to a better understanding of local markets, including internship and job opportunities.
Approximate date – start of intervention:
• November-December 2020 (exact date): those participants selected to the vocational training receive 160 hours vocational training at the CIFAP of Bula.
• January-March 2020: participants undertake either a traineeship in a local firm located in Cacheu or Oio or an entrepreneurship support program.
• November 2020 -September 2021: the beneficiaries of the scholarships attend a vocational training course in a centre of their choice within Guinea-Bissau.
Approximate date – follow-up data collection:
• January-March 2022: running the end-line survey
C) Application mechanism
1. Sampling: the sample was obtained through a promotional campaign on the national radio and Facebook.
2. Shortlisting and screening: interested candidates called our team, being screening for eligibility. Those fulfilling the eligibility criteria were included in the randomization. The individuals in the treatment group were invited to take an examination in person. The day of the examination the fulfilment of the mentioned criteria was checked upon administrative documents and personal interviews.
3. Allocation to the courses: the matching between the available positions in each course was done balancing the performance of the test-takers and their preferences, as previously explained
D) Eligibility mechanism
1. Eligibility criteria:
The GOT programme targeted young people living in the regions of Cacheu and Oio who have some history or internal or international migration or consider the possibility of migrating in the future. The existence of the programme was advertised in the main municipalities in the regions of Cacheu and Oio, including a list of phone numbers to which those interested could call and apply.
Hence, only men satisfying these conditions were considered eligible to take part in the project for the sake of the present impact evaluation. We considered only candidates answering positively the following questions:
- Willingness to take part in at least one of the offered courses, travelling to the municipality where the vocational training takes place for the duration of the courses.
• Region of residence: the experiment was run only in the regions of Cacheu and Oio since it aims to isolate the impact of vocational training in rural areas. Only candidates that had their residence in one out of the two regions passed to the next phase of the interview. In order to control for the truthfulness of statements in the baseline survey, potential participants were asked to bring an official identity document the examination day and their region of residence was checked.
• Migratory profile: the effective implementation of this criteria was done through the establishment of three questions directed to identify the three relevant types of migrants: internal, returned, and potential migrants. Only men who could be identified at least as one of these types of migrants were considered eligible. From our experience with previous surveys, we know we can trust people’s word on this and, moreover, since the candidates did not know that there selection was conditional on having a migratory profile they are not expected to lie about it.
• Age: only individuals ranging between 18 and 34 years old were considered eligible to participate in the courses. This criterion was set after consultation with local consultants and ENGIM, aiming to address one of the main problems of the labour market in Guinea-Bissau: the lack of opportunities and high unemployment levels among young people. In order to control for the truthfulness of the age declared in the baseline survey, potential participants were asked to bring an official identity document the examination day and their age was checked.
• Other situations of vulnerability: given the large list of vulnerabilities among the target population and the subjectivity that would be required to rank people based on the different types of vulnerabilities, it was decided to proxy the multi-dimensional nature of vulnerability using per capita household income. Despite its limitations, this simple measure of poverty is widely recognised as a useful proxy for vulnerability by international organizations and development experts. The threshold was set at 25,000 FCFA/month per person, after consulting with local experts in the matter. This threshold was calculated through two question which ask potential candidates how many members were in their households and their approximate monthly household income. Follow up, consistency questions and embedded controls were included in the survey instrument to identify and exclude the individual that were providing unreliable answer. Moreover, the fact that potential candidates did not know the set threshold for participation or that participation was conditional on income should reduce their incentives to lie.
If these conditions were met, the enumerator will explain the candidate the basic elements of the vocational training program and asked If the candidate agrees to participate. Those who consented would then complete a baseline survey.
E) Allocation to treatment among the treated
Out of the 151 male candidates with no disability that were invited to take the examination, 56 were finally selected to participate in the vocational training courses at the CIFAP in Bula and 35 were given a grant to choose a training institution, for a total of 91 participants allocated to the treatment. The allocation to treatment was done according to the following procedure. First, the allocation was done trying to match the available courses and the candidates’ preferences. Among the people having one course as their first choice, the women got a spot with preference to men. If there were more women willing to do a course as first option than available spots, the available spots were allocated to them having into account their grades. If after the female allocation, there were free spots for a course those were distributed among the men with the best grades who had it as their first option. Such process repeated for people having the course as second option, third and so on until all spots were covered.
Afterwards, the scholarships were allocated to the people who did not get a spot in the vocational courses at Bula because of the lack of matching between their preferences and the course offer, but who still had a good performance at the exam. Thus, the scholarship was granted following the ranking of the candidates according to their grades, without taking in consideration their gender. In the cases where a candidate refused the offer, it went to the next one in the list. This process has resulted in 35 men getting a scholarship.
Summing up:
- Allocated to the treated group: 151.
- Allocated to treatment: 91.
o Allocated to courses in Bula: 56.
Going to an entrepreneurship training: depending on the results of the courses
Going to an internship: depending on the results of the courses
o Given a grant and freedom to choose training institution: 35.
|
After
A) Main research question
Do vocational training programs improve labour market outcomes, leading to a decrease in willingness to emigrate, in rural areas of low-income countries?
B) Timeline
Exact date – baseline data collection:
• August-September 2020: design of the evaluation
• September-October 2020: promotion of the courses between potential eligible candidates
• September-October 2020: checking for candidates’ eligibility and registering consent, running the baseline survey and assigning the selected candidates to the treatment and control groups.
• November 2020: running the market study survey to contribute to a better understanding of local markets, including internship and job opportunities.
Approximate date – start of intervention:
• November-December 2020 (exact date): those participants selected to the vocational training receive 160 hours vocational training at the CIFAP of Bula.
• January-March 2020: participants undertake either a traineeship in a local firm located in Cacheu or Oio or an entrepreneurship support program.
• November 2020 -September 2021: the beneficiaries of the scholarships attend a vocational training course in a centre of their choice within Guinea-Bissau.
Approximate date – follow-up data collection:
• January-March 2022: running the end-line survey
C) Application mechanism
1. Sampling: the sample was obtained through a promotional campaign on the national radio and Facebook.
2. Shortlisting and screening: interested candidates called our team, being screening for eligibility. Those fulfilling the eligibility criteria were included in the randomization. The individuals in the treatment group were invited to take an examination in person. The day of the examination the fulfilment of the mentioned criteria was checked upon administrative documents and personal interviews.
3. Allocation to the courses: the matching between the available positions in each course was done balancing the performance of the test-takers and their preferences, as previously explained
D) Eligibility mechanism
1. Eligibility criteria:
The GOT programme targeted young people living in the regions of Cacheu and Oio who have some history or internal or international migration or consider the possibility of migrating in the future. The existence of the programme was advertised in the main municipalities in the regions of Cacheu and Oio, including a list of phone numbers to which those interested could call and apply.
Hence, only men satisfying these conditions were considered eligible to take part in the project for the sake of the present impact evaluation. We considered only candidates answering positively the following questions:
- Willingness to take part in at least one of the offered courses, travelling to the municipality where the vocational training takes place for the duration of the courses.
• Region of residence: the experiment was run only in the regions of Cacheu and Oio since it aims to isolate the impact of vocational training in rural areas. Only candidates that had their residence in one out of the two regions passed to the next phase of the interview. In order to control for the truthfulness of statements in the baseline survey, potential participants were asked to bring an official identity document the examination day and their region of residence was checked.
• Migratory profile: the effective implementation of this criteria was done through the establishment of three questions directed to identify the three relevant types of migrants: internal, returned, and potential migrants. Only men who could be identified at least as one of these types of migrants were considered eligible. From our experience with previous surveys, we know we can trust people’s word on this and, moreover, since the candidates did not know that there selection was conditional on having a migratory profile they are not expected to lie about it.
• Age: only individuals ranging between 18 and 34 years old were considered eligible to participate in the courses. This criterion was set after consultation with local consultants and ENGIM, aiming to address one of the main problems of the labour market in Guinea-Bissau: the lack of opportunities and high unemployment levels among young people. In order to control for the truthfulness of the age declared in the baseline survey, potential participants were asked to bring an official identity document the examination day and their age was checked.
• Other situations of vulnerability: given the large list of vulnerabilities among the target population and the subjectivity that would be required to rank people based on the different types of vulnerabilities, it was decided to proxy the multi-dimensional nature of vulnerability using per capita household income. Despite its limitations, this simple measure of poverty is widely recognised as a useful proxy for vulnerability by international organizations and development experts. The threshold was set at 25,000 FCFA/month per person, after consulting with local experts in the matter. This threshold was calculated through two question which ask potential candidates how many members were in their households and their approximate monthly household income. Follow up, consistency questions and embedded controls were included in the survey instrument to identify and exclude the individual that were providing unreliable answer. Moreover, the fact that potential candidates did not know the set threshold for participation or that participation was conditional on income should reduce their incentives to lie.
If these conditions were met, the enumerator will explain the candidate the basic elements of the vocational training program and asked If the candidate agrees to participate. Those who consented would then complete a baseline survey.
E) Allocation to treatment among the treated
Out of the 151 male candidates with no disability that were invited to take the examination, 57 were finally selected to participate in the vocational training courses at the CIFAP in Bula and 28 were given a grant to choose a training institution, for a total of 85 participants allocated to the treatment. The allocation to treatment was done according to the following procedure. First, the allocation was done trying to match the available courses and the candidates’ preferences. Among the people having one course as their first choice, the women got a spot with preference to men. If there were more women willing to do a course as first option than available spots, the available spots were allocated to them having into account their grades. If after the female allocation, there were free spots for a course those were distributed among the men with the best grades who had it as their first option. Such process repeated for people having the course as second option, third and so on until all spots were covered.
Afterwards, the scholarships were allocated to the people who did not get a spot in the vocational courses at Bula because of the lack of matching between their preferences and the course offer, but who still had a good performance at the exam. Thus, the scholarship was granted following the ranking of the candidates according to their grades, without taking in consideration their gender. In the cases where a candidate refused the offer, it went to the next one in the list. This process has resulted in 35 men getting a scholarship.
Summing up:
- Allocated to the treated group: 151.
- Allocated to treatment: 85.
o Allocated to courses in Bula: 57.
Going to an entrepreneurship training: 14 out of which 3 got an startup grant
Going to an internship: 43
o Given a grant and freedom to choose training institution: 28.
|
|
Field
Sample size (or number of clusters) by treatment arms
|
Before
151 individuals allocated to the control group. 151 individuals allocated to the treated group out of which 91 were allocated to treatment. Among those allocated to treatment 56 individuals took a 160 hours vocational training at the CIFAP of Bula and 35 individuals received a scholarship to attend a vocational training in other centre within the country.
|
After
151 individuals allocated to the control group. 151 individuals allocated to the treated group out of which 85 were allocated to treatment. Among those allocated to treatment 57 individuals took a 160 hours vocational training at the CIFAP of Bula and 28 individuals received a scholarship to attend a vocational training in other centre within the country.
|
|
Field
Power calculation: Minimum Detectable Effect Size for Main Outcomes
|
Before
The minimum detectable effect size has been estimated for personal income, probability of active employment status and willingness to migrate. All estimations are based on -balanced- sample data of the control and treatment group, at baseline, setting the significance level of Type I errors to 0.05 and the probability of avoiding Type II errors, or power, to 0.80. MDES are calculated for the whole 151 individuals included in the treatment group, although only 91 of them were finally allocated to treatment.
The mean personal income at baseline is 13,497 XOF, with standard deviation of 22,518 XOF for the control group and 20,853 XOF for the treatment one. The estimated minimum detectable effect size equals 7,058.3 XOF using a two-sided test. In other words, after the intervention the difference in average income across groups should be over 7,058.3 XOF or, more precisely, 52.3% of the average income for the control group to be detected.
The mean willingness to emigrate equals 0.2605 with standard deviation of 0.2826 for the control group, while the standard deviation of the treated group is 0.3130. The estimated minimum detectable effect size equals 0.0965 using a two-sided test. In other words, after the intervention the difference in the willingness to emigrate index across groups should be over 0.0965 or, more precisely, 37% of the average willingness to emigrate index for the control group to be detected.
Finally, the mean of the control group probability of active employment status equals to 23.84% and the standard deviation is 42.75% for both groups. The estimated minimum detectable effect size equals 13.83% using a two-sided test. In other words, after the intervention the difference in the probability of having an active employment status across groups should be over 13.83 percentage points or, more precisely, 58% of the average probability for the control group to be detected.
|
After
The minimum detectable effect size has been estimated for personal income, probability of active employment status and willingness to migrate. All estimations are based on -balanced- sample data of the control and treatment group, at baseline, setting the significance level of Type I errors to 0.05 and the probability of avoiding Type II errors, or power, to 0.80. MDES are calculated for the whole 151 individuals included in the treatment group, although only 85 of them were finally allocated to treatment. During the endline we spotted that some interviewers recorded the income data of 5 candidates in thousands XOF. After confirming with the subjects we recoded the values in units, which has insignificantly changed MDES.
The mean personal income at baseline is 13,554 XOF, with standard deviation of 22,579 XOF for the control group and 20,701 XOF for the treatment one. The estimated minimum detectable effect size equals 7,006 XOF using a two-sided test. In other words, after the intervention the difference in average income across groups should be over 7,006 XOF or, more precisely, 49.2% of the average income for the control group to be detected.
The mean willingness to emigrate equals 0.2737 with standard deviation of 0.2870 for the control group, while the standard deviation of the treated group is 0.3130. The estimated minimum detectable effect size equals 0.0971 using a two-sided test. In other words, after the intervention the difference in the willingness to emigrate index across groups should be over 0.0971 or, more precisely, 36.96% of the average willingness to emigrate index for the control group to be detected.
Finally, the mean probability of active employment equals to 23.84% and the standard deviation is 42.75%, for both groups. The estimated minimum detectable effect size equals 13.83% using a two-sided test. In other words, after the intervention the difference in the probability of having an active employment status across groups should be over 13.83 percentage points or, more precisely, 58% of the average probability for the control group to be detected.
|
|
Field
Additional Keyword(s)
|
Before
Migration, Applied Econometrics, Active Labour Market Policies, Vocational Training, Economic Development
|
After
Migration, Applied Econometrics, Active Labour Market Policies, Vocational Education and Training, Economic Development
|
|
Field
Intervention (Hidden)
|
Before
This intervention seeks to improve the matching process between the labour supply and demand of young male workers in the countryside of a low-income country, specifically in the Guinea-Bissau’s regions of Cacheu and Oio. We implemented this RCT to evaluate the impact of ENGIM’s (an Italian NGO specialized on entrepreneurship and vocational training) GOT project, which aims to create employment opportunities in those regions to prevent population’s engagement in irregular migration practices. In order to do that we randomized a sample of youth male workers from both regions, fulfilling some selection criteria (low-income, age between 18-34 and migratory profile), into a treated and a control group. The treated group was invited to take an exam in Portuguese and Mathematics at a vocational centre (CIFAP of Bula) on 26th and 27th of October. The results of the examination, together with participants’ declared preferences and previous working experience, were used as criteria to allocate part of them to the different courses .
The intervention is structured in two stages. First, the beneficiaries received either a 160 hours vocational training at the CIFAP of Bula (56 participants) or a scholarship to attend a vocational training in other centre within the country (35 participants). The first group can undertake their training in fruticulture, beekeeping, horticulture, locksmith, civil construction, business management and IT or hospitality and cooking, and receive 45,000 XOF per person to cover their attendance expenses. The second group can take a vocational training in any subject of their interest. They receive 110,000 XOF via Orange Money and personalised tutorship. The first arm of the experiment runs from November 3rd to the 23rd of December, Monday to Friday, between 14:30 and 18.30 p.m. The second arm has a more flexible schedule, with the requirement that the courses must be completed between November 2020 and September 2021.
Secondly, the laurates of the CIFAP can go into an entrepreneurship support program or into a traineeship in a local firm. In this case, the first group includes the best 4 students of each course for a total of 28 beneficiaries. They receive a 60 hours course in entrepreneurship and business model implementation in January 2021 plus 30,000 XOF extra. The second group will benefit from an internship in local firms for 2 months, approximately between January and March, plus 76,000 XOF extra. The internships will be organised and monitored by a specialised tutor, they will be goal-oriented and will follow the personal vocation of each beneficiary.
Additionally, our research team conducted an enterprise survey in the regions of Cacheu and Oio in December 2020 in order to identify the skills needed by local firms, potential placements for the trainees, and areas that presented business opportunities for entrepreneurship.
|
After
This intervention seeks to improve the matching process between the labour supply and demand of young male workers in the countryside of a low-income country, specifically in the Guinea-Bissau’s regions of Cacheu and Oio. We implemented this RCT to evaluate the impact of ENGIM’s (an Italian NGO specialized on entrepreneurship and vocational training) GOT project, which aims to create employment opportunities in those regions to prevent population’s engagement in irregular migration practices. In order to do that we randomized a sample of youth male workers from both regions, fulfilling some selection criteria (low-income, age between 18-34 and migratory profile), into a treated and a control group. The treated group was invited to take an exam in Portuguese and Mathematics at a vocational centre (CIFAP of Bula) on 26th and 27th of October. The results of the examination, together with participants’ declared preferences and previous working experience, were used as criteria to allocate part of them to the different courses .
The intervention is structured in two stages. First, the beneficiaries received either a 160 hours vocational training at the CIFAP of Bula (57 participants) or a scholarship to attend a vocational training in other centre within the country (28 participants). The first group can undertake their training in fruticulture, beekeeping, horticulture, mechanics, electricity, locksmith, civil construction, business management and IT or hospitality and cooking, and receive 45,000 XOF per person to cover their attendance expenses. The second group can take a vocational training in any subject of their interest. They receive 110,000 XOF via Orange Money and personalised tutorship. The first arm of the experiment runs from November 3rd to the 23rd of December, Monday to Friday, between 14:30 and 18.30 p.m. The second arm has a more flexible schedule, with the requirement that the courses must be completed between November 2020 and September 2021.
Secondly, the laurates of the CIFAP can go into an entrepreneurship support program or into a traineeship in a local firm. In this case, the first group includes the best 4 students of each course for a total of 28 beneficiaries, including non-randomized female beneficiaries. They receive a 60 hours course in entrepreneurship and business model implementation in January 2021 plus 30,000 XOF extra. The second group will benefit from an internship in local firms for 2 months, approximately between January and March, plus 76,000 XOF extra. The internships will be organised and monitored by a specialised tutor, they will be goal-oriented and will follow the personal vocation of each beneficiary.
Additionally, our research team conducted an enterprise survey in the regions of Cacheu and Oio in December 2020 in order to identify the skills needed by local firms, potential placements for the trainees, and areas that presented business opportunities for entrepreneurship.
|
|
Field
Pi as first author
|
Before
No
|
After
Yes
|
|
Field
Building on Existing Work
|
Before
|
After
No
|