The Role of Hard and Soft Skills in Entrepreneurial Success: Experimental Evidence from Uganda

Last registered on November 07, 2014

Pre-Trial

Trial Information

General Information

Title
The Role of Hard and Soft Skills in Entrepreneurial Success: Experimental Evidence from Uganda
RCT ID
AEARCTR-0000550
Initial registration date
November 07, 2014

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
November 07, 2014, 8:58 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
University of Connecticut

Other Primary Investigator(s)

PI Affiliation
PI Affiliation

Additional Trial Information

Status
On going
Start date
2013-01-01
End date
2015-12-31
Secondary IDs
Abstract
This project will investigate the relative importance of hard skills versus soft skills in business success and more broadly in the youth’s school-to-work transition. The project will strengthen, experimentally, the training of these skills among recently graduated Ugandan secondary school students. The goal of the current endeavor is to: (a) leverage insights from the economic and social sciences toward the goal of improving entrepreneurial success in developing countries, and (b) test whether “hard” versus “soft” skills better predict consequential entrepreneurial and business longevity outcomes, and a better performance in the labor market, particularly among young people and women.
External Link(s)

Registration Citation

Citation
Carney, Dana, Nathan Fiala and Paul Gertler. 2014. "The Role of Hard and Soft Skills in Entrepreneurial Success: Experimental Evidence from Uganda." AEA RCT Registry. November 07. https://doi.org/10.1257/rct.550-1.0
Former Citation
Carney, Dana, Nathan Fiala and Paul Gertler. 2014. "The Role of Hard and Soft Skills in Entrepreneurial Success: Experimental Evidence from Uganda." AEA RCT Registry. November 07. https://www.socialscienceregistry.org/trials/550/history/3058
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
The role of the entrepreneur in business and development success has grown in interest among policy makers and researchers in both developed and developing economies. While often left out of formal theories, the entrepreneur plays a key role in the exchange of goods and development of innovation. The role of the entrepreneur in development is of especially important concern in regions where formal sector employment is scarce.
Uganda, like most low-income countries in the world, has a large share of youth who are either unemployed or underemployed. This is not only a concern of the welfare of these youth, but also a waste of valuable human capital available for economic growth. Youth living in these countries encounter numerous challenges entering into and succeeding in the labor market. Living in economies where employment opportunities are scarce and self-employment often the only option, youth need the right combination of human, financial, and social capital that will give them a chance to turn their energy and ideas into entrepreneurship opportunities that lead to the creation of more and better jobs.
Individuals often lack knowledge of the best practices of running of business, especially in developing countries, where education in general is often limited. An intervention growing in popularity is the introduction of business skills training. The goal is to increase the knowledge individual business owners have about doing business, and while doing so, also increase their returns and welfare.
A number of recent research projects have explored the impact of enterprise training and have found mixed results. Oosterbeek, van Praag and Ijsselstein (2010) find that business skills training has no effect on business outcomes. Bjorvatn and Tungodden (2010) find that business training in Tanzania has some effect on business knowledge, but no effect on business outcomes. Karlan and Valdivia (2006) include business training in a microfinance institute in Peru. The program lead to increased business knowledge, revenue and repayment rates, but had no effect on profits. Attanasio et al (2011) subsidized vocational training in Colombia and found income and employment increases for women, but none for men. Blattman, Fiala and Martinez (2011) find high returns to a grant program in Uganda for business training and capital for men and women, with large long-term effects for women. They find though that alleviating capital constraints were the major reason behind business success, with a small role for business training.
At best the current literature presents conflicting results of the usefulness of entrepreneurship training. At worst it suggests such programs are not sustainable. The questions being asked in the literature though are unsatisfactory when it comes to answering definitively the value of entrepreneurship training, in part because they are asking too broad of a question. Each of the evaluations described focus on a package of entrepreneurship training initiated by one or more NGOs or training facilities. It is not clear what exactly is in this package, whether it is comparable across programs or countries, or even whether it can be replicated in the same area. Rather than asking what makes a good entrepreneur, they are throwing a black box program at individuals and seeing if that particular program works in that particular place. It is not surprising then that the results are somewhat contradictory. Entrepreneurship training thus works for women only in some programs, or men only in others. Some increase revenues, other have no effect.
In order to determine if entrepreneurship can be taught, and if that teaching can then lead to real improved real world outcomes, a clear theory of the entrepreneur needs to be developed. The theory should then be tested through precise experiments that parse out the individual components of entrepreneurship to determine what, if anything, can be taught. This will lead to a clearer understanding of what enterprise training works, how it works, and if it can be replicated.
This pre-analysis plan is for an experiment to determine a major component of what type of training improves business development. We focus on the differential effect of what is commonly referred to as “hard” skills (e.g., financial decision making) and “soft” skills (e.g., negotiation and communication). This project will investigate the relative importance of hard skills versus soft skills in business success and more broadly in the youth’s school-to-work transition. The project will strengthen, experimentally, the training of these skills among recently graduated Ugandan secondary school students. The goal of the current endeavor is to: (a) leverage insights from the economic and social sciences toward the goal of improving entrepreneurial success in developing countries, and (b) test whether “hard” versus “soft” skills better predict consequential entrepreneurial and business longevity outcomes, and a better performance in the labor market, particularly among young people and women.
The hard skills training was built upon a currently existing hard skills-based curricula developed by the ILO called Know About Business (KAB), which has been run in over 40 countries worldwide. The newly developed curriculum will focus on financial literacy, the role of budgeting, calculating and keeping track of expenses, revenue and profits, the importance of re-investment for business growth, and a number of other topics.
The soft skills training was built upon existing soft skills-based curricula harvested from both the Young Entrepreneurs Program (YEP) and Educate!, a local NGO that mentors young people in starting businesses. These existing curricula will be refined through guidance from economic and social science research pointing to the critical importance of a few core social skills areas including: building and accessing social networks, how to influence and persuade, how to perceive and listen to others, and how to effectively and creatively negotiate. The full development of this component will build on the existing undergraduate business curriculum at major business school and will be adapted and rigorously piloted tested to match the Ugandan context.
As the target population of the evaluation is youth who have recently finished school, the research will first measure the effect of entrepreneurship training on attitudes and skills. The project will also measure the training on positive labor market outcomes and business success (such as self-employment, business creation, job creation through start-ups, and earnings). The project will then ultimately assess the importance of soft skills versus hard skills. The research will provide evidence on how these skills help to build better, sustainable and bankable businesses. To capture the chain of effects on young people’s decisions and outcomes and measure labor market outcomes, the first follow-up data collection will be conducted in the middle of 2015, two years after participants have finished the training.

Intervention Start Date
2013-02-01
Intervention End Date
2015-07-31

Primary Outcomes

Primary Outcomes (end points)
The evaluation team decided to recruit students for the program and administer the baseline survey at the same time due to the difficulty of tracking students once they leave secondary school. Enumerator teams visited over 200 schools to recruit students and administer baseline surveys. Teams first asked school administrations for permission to advertise the program and asked them to gather all Senior 6 students in one room at the appointed time. School staff members were asked to leave the classroom to preserve confidentiality and to avoid putting undue pressure on students. When students were assembled, the enumerators gave a brief introduction to SEED, highlighting key information that would influence their decision to apply for the program: enrollment was free; a round-trip transport refund would be provided; the training was a residential program that provided meals; students would have the chance to write business plans and enter a competition for a large cash prize.
Enumerators explained to students that as there was limited space in the program, not everyone would be admitted, but that they all have an equal chance of admission into SEED.
All students interested in the program were asked to fill out both a simple application form and a baseline survey, which was framed as purely research-oriented and with no bearing on a student’s chances of being accepted. The application form took about five minutes to fill out and asked students to provide detailed contact information. It also included the question “If you participate in the SEED training, what business would you start and how would you start & run it?” This question was included to indicate to students that SEED would accept serious applicants who have given some thought to the idea of starting their own businesses. After data entry and randomization into treatment and control groups, teams were re-deployed to schools to post lists of admitted students and distribute personalized acceptance letters.
The baseline questionnaire included modules on household information, economic activities, and psycho-social indicators, as well as measures of cognitive ability, risk behavior, and time preferences. Each scale was validated using data from pre-testing on Ugandan secondary school students. The evaluation team adjusted wording, formatting, question phrasing and answer choices in an iterative process based on feedback from focus group discussions and classroom pilots, in order to make the questionnaire understandable and easy to fill out for upper-secondary students.
Due to the infeasibility of administering one-on-one surveys to large numbers of students in a school setting, the baseline survey was self-administered. Each student received a copy of the questionnaire and completed it independently, though the process was guided by an enumerator team who read and explained certain sections aloud and proctored the remainder. The survey consists of 10 sections and took 1-1.5 hours to administer. Enumerators were trained to walk around the room and monitor students closely while they were completing the survey to check for student errors and to make sure students were not copying their neighbors. They were instructed not to assist students in answering knowledge-based questions, such as calculation of profit or interest. Enumerators were, however, permitted to help student in cases that would enable them to better understand a question by defining a commonplace word that the student had trouble with, for example.
Enumerator team leaders audited a random selection of ten surveys in each batch, counting numbers of errors in student completion of the survey. These include blanks, circling errors (i.e. circling too many options), skip pattern errors, illogical values, and illegible writing. Team leaders then gave feedback to enumerator teams on how to improve their monitoring during the survey.
Enumerators encountered some variation in the ability of S.6 students to understand the survey, based on their familiarity with English and prior exposure to similarly structured exams. It should also be noted that though enumerators monitored students during the survey, the evaluation team cannot rule out the possibility that questions were filled out improperly by students who did not ask questions or who encountered difficulty when an enumerator was in another part of the room, particularly in larger schools.
Some enumerators reported that they suspected students thought that IPA was "testing" them, and that students would be admitted based on whether they filled out the survey completely or had the "right" answers, for instance. Enumerators had stressed prior to the survey administration that all students have an equal chance of admission and the survey is merely for research purposes.
Some enumerators reported that some students perceived some of the psychosocial questions as too personal. Enumerators had indicated that students were not obligated to answer questions that made them uncomfortable; this was also printed in the consent form. They also had the impression that students became fatigued toward the end of the psychosocial section because of the large number of questions.
Understanding of the question "Would you rather receive 10,000 today or 30,000 in a month" may have been inconsistent; some students were initially unsure about whether it meant they would receive 10,000 every day or just once. A question that asked students to indicate the extent to which they agreed with the statement "I have restless nights" also caused initial confusion and differing interpretations among some students.
Primary Outcomes (explanation)
The hard skills training was built upon a currently existing hard skills-based curricula developed by the ILO called Know About Business (KAB), which has been run in over 40 countries worldwide. The training focused on the following standard modules for business skills training:
• Financial literacy, including calculating interest rates and general business numeracy.
• Developing a business proposal. By the end of the course, the students are expected to have completed a basic business plan. The business plan will serve as a blue print for the students in their business and help in accessing finance.
• The role of budgeting in the successful development and implementation of a business.
• How to calculate and keeping track of expenses, revenue and profits.
• The importance of re-investment in the business for business growth.
• How to draft a marketing strategy.
• Methods for planning staff needs.
• How to cost goods and services effectively.
• Optimal legal form of the business including process for getting licenses and the permits.
• How to assess of the environmental impact of planned business.
• Forecasting finances.
• A business game that simulates business operations to experiment on the challenges of managing small businesses with product portfolios in different markets.
The soft skills curriculum was developed by the researchers for the purpose of this experiment, using an existing curriculum from the NGO Educate! as a base model. For 100 years psychologists have theorized that there is much more to human intelligence than “hard” or quantitative intelligence. The concept of “Social Intelligence” began to gain traction in the 1920’s with E. L. Thorndike’s seminal work. Since then, all models and theoretical treatments of emotional and social intelligence factor out to a few fundamental underlying dimensions (Riggio, 1986 ; Salovey & Mayer, 1990 ; Thorndike, 1920 , 1936 ). These dimensions all describe (1) the ability to appraise one’s own and others’ emotional and motivational states, and (2) the ability to regulate or control these states within oneself and in others and (3) use such emotional and motivational information toward advancing an a social goal such as to influence, persuade, transact effectively, communicate more clearly, or befriend more quickly.
The training was designed to focus on a suite of trainable skills harvested from psychology, sociology, and economics (named “soft skills” by the business community) which predict facets of entrepreneurial success (e.g., Moss & Tilly, 1996 ; Nickson, Warhurst, Commander, Hurrell, & Cullen, 2011 ). Many scientists and practitioners have argued strongly for the increased reliance upon soft-skills training and education in business education (e.g., Burke, Drasgow, & Edwards, 2004 ; Navarro, 2008 ). For example, having a particular structure to one’s social network leads to an increased likelihood of having entrepreneurial success (Stuart & Sorenson, 2005 ). Thus, knowing how to build and structure one’s social network is of utmost importance to increasing the likelihood of entrepreneurial success.
The four most critical and predictive dimensions of “soft skills” build on each other and are taught in the following order: (1) Building and Maintaining Social Networks, (2) How to Perceive and Listen to Others, (3) How to Influence & Persuade Others, and (4) How to Effectively and Creatively Negotiate. The last module, negotiation, pulls together many of the skills taught over the course of the teaching term. Below is a brief background and pedagogical plan for each. The content for these 4 modules will be taken directly from the YEP and Educate! Curricula and supplemented and edited to reflect the most cutting edge insights from business education. Materials will be grounded deeply by the cultural fabric understood by Ugandan youth by a hired teaching assistant who has had extensive experience with the Ugandan culture and teaching in Uganda.
In addition, special attention was paid to social networks. To be a successful entrepreneur, the people you know are as important as your idea is (Stuart & Sorenson, 2005 ). In addition, it is not only who you know, but who THEY know. Below are diagrams of two kinds of networks. The diagram on the left (Panel A) depicts an “embedded network.” Families, classes, and small work-groups are described by embeddedness: everyone knows everyone fairly well. This kind of network is not particularly effective for launching an entrepreneurial idea because these networks are small, informationally and resource redundant, and you can’t “buy” others’ help by introducing them to anyone (i.e., “brokering” between otherwise unconnected networks). In contrast, the network on the right (Panel B) depicts what has come to be known as an “entrepreneurial network.” In this kind of network, you know many different kinds of KEY people who have access to whole other networks non-redundant with your own. Principles taught in this module include: (1) what does your current social network look like? (2) What kinds of additions do you need to your network? (3) How to add different kinds of people to your network. (4) How to broker connections between people (and their associated networks). (5) Strategies to maintain contacts after they have been established. (6) How to call on people when you need them. (7) What is the difference between weak and strong social network connections?
One of the common assumptions made about influence is that “some people are just good at it.” This is incorrect. One of the common assumptions made about persuasion is that only some people can be persuaded—this is also incorrect. This module teaches Cialdini’s (e.g., Cialdini, 2001 ) 6 building blocks of persuasion: (1) reciprocation, (2) liking, (3) commitment and consistency, (4) authority, (5) social proof, and (6) scarcity. Students are given in-class and out-of-class assignments in which they will practice their influence and persuasion skills to: (a) persuade others of their point of view on an issue, (b) sell goods, and (c) trade upwards from a small object to as valuable an object as possible in 5 trades (the winner of this context receives a prize and public recognition).
The ability to read other people and know their intentions, goals, preferences and emotions is a skill that helps you manage them, persuade them, coordinate them, manage conflicts between them, know when they are lying to you, and enlist them toward your vision and goals. At the most basic level, watching and listening to others and what they say, is a critical skill that most of us do not have the tendency to do. This is a skill which can be taught through practice in a classroom setting with materials (e.g., photos of African faces expressing different emotions) train the accurate detection of emotions such as: happiness and sadness. Antecedents and consequences of emotional experiences are discussed at length. Further, “people reading” skills are also taught in dyads (people working with each other to predict and then test how each other is feeling in various role-plays and improvisations) and in small groups. Role plays to practice listening skills will be used to exercise the critical point that a person cannot know what another person wants or feels unless you stop your own mind from thinking what you want and feel and listen to and think about their wants and feelings.
The final module builds on the previous three modules and also contains unique value-added hands-on learning. The primary assumption shattered in this module is that all negotiations are about a fixed resource (i.e., “the myth of the fixed pie”). Negotiations are NOT always a zero-sum game. In fact, more often than not there are creative and integrative solutions to problems such that both parties can trade what they are least interested in for what they are most interested in (provided the other party has the reciprocal valuation of the same commodities). These critical learning points are covered. Students are given opportunities to practice utilizing the skills in the classroom (in dyadic simulated negotiations) and in the community with strangers. They are also given skills to learn how to “expand the pie” and think creatively about solutions to common problems. Principles taught are: (1) how to expand the goods being discussed (i.e., the myth of the “fixed-pie”), (2) how to build relationships during a negotiation (and before and after), (3) how to be more directive and assertive when appropriate, (4) different types of issues facing negotiators, Students will engage in a number of in-class and out-of-class negotiations in order to demonstrate and practice each concept.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Between November 2012 and April 2013, IPA sent 3 rounds of mass SMS reminders and did 2 rounds of intensive calling (i.e. called all individuals in the treatment group). The last round took place shortly before the program launch. Efforts to reach students or their close relatives on the phone included calling every phone number the student had provided at different times of day, and calling the school administrators and student leaders (whom IPA designated to help mobilize their peers).
Once the program had launched, teams in schools with low turnout also called students who didn't turn up. IPA offered a transport refund to each student as a part of efforts to encourage students to participate, a fact which was emphasized during initial recruitment, as many students had to travel long distances to attend the training.
To further reduce costs incurred by students, IPA organized mass transportation rather than expect students to find their own way to the training sites, some of which were relatively remote. The team asked students to meet at a well-known location in a nearby town, where IPA team members waited to greet students and load them onto buses to the training sites. About two-thirds of individuals in the treatment group eventually attended training.
To train SEED instructors, IPA partnered with Educate!, an NGO that delivers social entrepreneurship training and mentorship to Ugandan secondary-school students. Educate! periodically trains groups of potential teachers in general techniques for teaching entrepreneurship skills. Educate! recruited and trained the majority of the SEED teachers in two rounds in October 2012 and February 2013. After the pilot, the implementation team realized that it was necessary to increase the number of teachers at each training site in order to make their workload more manageable. Time constraints made it difficult for Educate! to train additional teachers, so IPA asked another education NGO called the Private Education Development Network (PEDN) to recruit and train a third group. After all teachers had undergone a general training in teaching methods, IPA trained them on the curriculum to which they had been assigned as well as the business plan curriculum.
The student trainings took place at secondary schools with which Innovations for Poverty Action partnered to implement the program. Students slept and boarded at the schools, attending training sessions during the day. Each student also wrote a business plan guided by daily instruction in the business plan curriculum; a cash grant of UGX 1,000,000 ($400) was awarded to the author of the highest quality business plan in each school (one out of every 100-120 students).
In addition to instructors, IPA trained two “camp directors” per training site to run the day-to-day operations, including handling finances, managing members of the school’s support staff that IPA retained, enforcing discipline, and organizing sports and social activities. The camp directors were supervised by a Field Officer, who was in turn supervised by a regional Field Manager. In January 2013, IPA carried out a pilot in a non-sample district to test the logistical arrangements and curriculum quality, making adjustments to the implementation plan accordingly.
A team of IPA-trained auditors observed teachers’ lessons on a rotating basis, using forms that recorded teachers’ adherence to lesson content, application of quality teaching methods, and details on the learning environment. In addition, students were asked to complete a variety of self-administered evaluations over the course of the training that asked about teaching quality, relevance of lessons, student characteristics, and their experience of SEED camp, among others. They also completed a knowledge assessment covering both hard and soft skills at the beginning and end of the camp. Auditors conducted focus groups at the end of the first and final weeks of the training that delved more deeply into student opinions on the lessons, teaching, and overall experience. Finally, teachers recorded student attendance at the start of each lesson.
A variety of problems arose over the course of the training. In some schools, students complained about the food provided, poor dormitory conditions, and particular camp rules. This was typically addressed by having one of the higher-level SEED supervisors step in, for instance to monitor the school support staff to improve food quality. Some students had been under the impression that SEED would be a relatively luxurious training given the involvement of foreign donors and the way it had been advertised (a free training with an opportunity to win cash grants) and were disappointed with the food quality and actual number of cash grants to be awarded, for instance. At one school, students briefly went on strike over such issues before higher-level SEED organizers were able to calm them down.
In one school the living conditions were very poor, which had demoralizing effects on the teachers and students. The implementation team worked quickly to shift all the students to a nearby SEED school that could accommodate them. In the same school shortly before the move, a student went to the nearby river to swim, despite warnings by school administrators, and drowned. The incident caused significant distress among teachers and students; lessons were canceled for two days and began again the following Monday.
One the final night of training in one of the rural schools, one of the auditors reportedly was out in the nearby town until very late at night with the school doctor and three female students. He was also observed on multiple occasions assisting one of the students with her business plan. This was the only reported case of inappropriate conduct between IPA staff and trainees.
Experimental Design Details
Randomization Method
Randomization on computer
Randomization Unit
Power calculations indicated that 1200 students were required in each arm (hard skills treatment, soft skills treatment, and control group), or 30 students in each of the 80 classes across 20 training sites. Taking into account attrition estimates from similar programs and the long waiting time between recruitment and program implementation, the evaluation team over-enrolled by one-third and accepted 1600 students in each treatment arm, or 40 students per class. The control group remained at 1200 students.
The random assignment of students to hard skills treatment, soft skills treatment, and control was carried out for each region individually using the same series of commands. First the random-number seed was set to ensure the randomization could be replicated. Each student was assigned a uniformly distributed random number between 0 and 1. The sample was then sorted by the random number in ascending order within the strata of school and gender. Each student received a treatment status of 0 (control), 1 (hard skills), 2 (soft skills), or 3 (not in sample). In order to achieve the desired number of students in each status, a particular fraction of each specified group (i.e. students by gender by school) was multiplied by the total number of students in that group.
Of the 200 schools visited during data collection and recruitment, 20 eventually served as training sites. The distribution of training sites were chosen based on the number of individuals in the sample surveyed in each region in order to achieve fairly even numbers across regions. Roughly 40% of the sample attended school in the West, 40% from the East (including 20% in the Jinja area and 20% in the Mbale area), and 20% from the North. As a result, 8 sites were located in the West, 8 in the East, and 4 in the North. The evaluation team divided the sampled schools in each region into geographical clusters, with a pair of host schools/training sites – one devoted to hard skills and one to soft skills – at the center of each cluster. This arrangement minimized the distance students would have to travel to get to the SEED training while serving the evaluation design.
Within each training site, each student was randomly assigned to a particular class for the duration of the training. Roughly 160 students were assigned to each site and were divided into 4 classes of 40 each. The class randomization code first assigns a random number to each student. Then it sorts all students by that number in ascending order, within each host school. The first 25% are assigned to class #1, the second 25% to class #2, etc.
Teachers were randomly assigned to teach hard skills, soft skills, or the business plan curriculum. The randomization was then checked for balance on a range of indicators, including gender, ethnic group, whether the teacher had prior teaching experience, education level, cognitive ability, score received during the general teacher training (if they were trained by Educate!), and aptitude for hard or soft skills (based on a short questionnaire). Teachers who were trained by Private Education Development Network (PEDN) were all assigned to teach the business plan curriculum due to the systematic difference between PEDN and Educate!’s training styles.
The evaluation team introduced a cross-cutting design in which half of the students received a lesson on how to use technology in business and the other half received a lesson on sexual and reproductive health (SRH). Half of the hard skills teachers taught one and the other half taught the other; the same was true for soft skills teachers. Since the technology and SRH modules were the last lesson in the main hard/soft skills curriculum, business plan teachers did not teach the special modules and were excluded from the randomization. Hard/soft skills teachers were first sorted in random order within their curriculum groups; the first 10 were assigned to SRH and #11-20 were assigned to technology. The evaluation team then checked that each group was balanced on average on the same group of indicators used in the randomization of teachers to curricula.
Randomizing teacher allocation to districts was performed last. Teachers were asked to indicate their top three district preferences. Because teachers overwhelmingly preferred urban districts to rural ones, the evaluation team also used data on teachers’ language abilities. First, because few teachers were interested in working in the Northern region, teachers who did prefer to work there were assigned to Northern schools. The remaining teachers were randomly assigned to districts according to language group, regardless of their district preferences. Teachers in smaller language groups were assigned first; Luganda speakers were assigned last.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
Power calculations indicated that 1200 students were required in each arm (hard skills treatment, soft skills treatment, and control group), or 30 students in each of the 80 classes across 20 training sites. Taking into account attrition estimates from similar programs and the long waiting time between recruitment and program implementation, the evaluation team over-enrolled by one-third and accepted 1600 students in each treatment arm, or 40 students per class. The control group remained at 1200 students.
Sample size: planned number of observations
2400
Sample size (or number of clusters) by treatment arms
Power calculations indicated that 1200 students were required in each arm (hard skills treatment, soft skills treatment, and control group), or 30 students in each of the 80 classes across 20 training sites. Taking into account attrition estimates from similar programs and the long waiting time between recruitment and program implementation, the evaluation team over-enrolled by one-third and accepted 1600 students in each treatment arm, or 40 students per class. The control group remained at 1200 students.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
As an initial test of program impacts of the program, we will compare pre- and post-test results for tests given to the students who participated in the program as follows: Si,t - Si,t+1 = α + βT + γXi + δR + εi (1) where i refers to an individual, S is the score of the test delivered before the program (time t) and after the program (time t+1), T is a matrix of dummy variables for which treatment an individual belonged to, X is a matrix of individual controls, R is a matrix of region and school dummies, and ε is the error term. This estimation will test for the changes in knowledge of hard and soft skills. To test the differential effects of each program on the hypotheses presented in Section 4 and Table 2, we will run the following intention to treat (ITT) regression model: Yi = α + βT + γXi + δR + εi (2) where i refers to an individual, Y is the outcome of interest, T is a matrix of dummy variables for which treatment an individual belonged to, X is a matrix of individual controls, R is a matrix of region and school dummies, and ε is the error term. The standard errors will be clustered at the school of origin level. In addition to the outcomes in Equation 2, we will also explore the effect of a number of heterogeneities as discussed in Section 4 and Table 2. This will be done using the following regression model: Yi = α + βT + λT*H + ωH + γXi + δR + εi (3) which includes the addition of the interaction term H, which is the heterogeneity of interest. H will be represented as a dummy variable and will also be included as a control in the regression. Again, standard errors will be clustered at the school of origin level. A final analysis will focus on social network effects. Students were randomly placed into classrooms with other students. The interactions that likely occurred within the classrooms could affect what is commonly referred to as the strong and weak ties of individuals. To test for this, a social network analysis approach will be used. This analysis will include individual and similarity tests. The individual test will use the following regression model: Yi = α + βT + γXi + δR + φX-i + εi (4) The difference between this test and Equation 2 is that the characteristics of the other students in the class, X-i, have been included. These characteristics will allow for a test of the role of other student characteristics in outcomes for student i. Following Fafchamps and Soderbom (forthcoming), we will also look at the difference between student i and the other students in the same classroom though a distance measure as follows: |Yi – Ŷ-i|= α + βT + γXi + δR + εi (5) where Ŷ-i is the average outcome of all other students in the class of student i. This will test for whether there is an increase or decrease in outcome differences between those in the same classrooms.
IRB

Institutional Review Boards (IRBs)

IRB Name
IPA
IRB Approval Date
2013-04-09
IRB Approval Number
483.12January-001
Analysis Plan

Analysis Plan Documents

SEED+pre-analysis+plan.pdf

MD5: cd9ae0e71f05ec31bea9dd6eba1d98e1

SHA1: 662c4acceb176b5c46e196d3081ad24778d8c207

Uploaded At: November 07, 2014

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials