Evaluation of a Program for the Professionalization of Artisans (ProfArts) in Ghana

The project encompasses two interventions. First, we assess the effects of different recruitment content on outreach and program outcomes. Second, we examine the impacts of ProfArts program participation on a comprehensive set of employment-related indicators.

For the first intervention, different mobilization content is randomly assigned to communities where artisans are to be recruited. The ProfArts project's overall recruitment strategy consists of (a) a default outreach campaign, which sees radio spots and social media content disseminated nationwide, and (b) a flyer campaign, with three different versions randomly assigned to communities. The different versions emphasize either (i) the potential for higher earnings as a result of the program, or (ii) the fact that artisans may learn new or update existing skills, or (iii) the program's technological aspects and offer of a digital platform. All versions of the mobilization campaign explain the core program structure and provide information on where and when to register.

For the second intervention, the ProfArts benefits package is randomly assigned to up to 10,000 eligible artisans out of a pool of up to 20,000 applicants. The program consists of a variety of benefits, including skills and financial literacy training, certification and licensing, provision of state-of-the-art tools and personal protective equipment, and access to a mobile application to connect artisans with clients. If program partners are unable to deliver a complete set of benefits to all beneficiaries, we may randomize the composition of benefits packages.

2021-01-01

2021-12-31

For the recruitment content intervention, the primary outcomes are (i) the number of artisans recruited per flyer of each type distributed, or an equivalent metric, (ii) the make-up of the recruited applicant pool, including characteristics such as age, employment status, educational attainment, social preferences, and personality traits, and (iii) downstream outcomes, including program completion rates for selected beneficiaries from different communities, and economic outcomes such as those used to evaluate the ProfArts program itself.

For the ProfArts program intervention, the primary outcomes are (i) employment, measured e.g. in terms of job retention, acquisition and lengths of employment spells, (ii) job quality and quality of life, including e.g. earnings and workplace conditions, (iii) mobility, e.g. migration intentions and behaviors, as well as (iv) firm-level outcomes, e.g. performance and employment at the firm level.

Concerning the possible effects of different recruitment content (i.e. messaging displayed on flyers), we measure (i) uptake rates to help us better understand applicants' motivations and how to design resource-efficient and well-targeted interventions, (ii) applicant pool characteristics to help explain how recruitment campaigns can lead to certain person types being represented disproportionately among beneficiaries (e.g. flyers highlighting monetary gains could attract relatively more profit-oriented individuals with lower expressed levels of altruism), and (iii) downstream outcomes to better understand how selection effects limit the external validity of RCTs and the long-term effects of recruitment content.

Concerning program effects, our measures reflect the program's own key performance targets and our expectation that the intervention will affect economic livelihoods in a variety of ways, in particular concerning employment, job quality, and mobility.

Secondary outcomes include variables that we will use to investigate mechanisms underpinning the interventions' effects, such as the role of knowledge and information, product and production innovations, usage of digital platforms for client and order acquisition, changes in subjects' professional networks, their client base, or their ability to acquire inputs at low cost, etc.

Concerning the analysis of recruitment strategies, we randomly assign one of three flyer types at the community level, within spatiotemporal blocks given by administrative areas and scheduled days of flyer distribution. Outreach and recruitment activities will initially focus on 113 communities in Accra, Kumasi, and Tamale (in 2020), and will subsequently expand to other locations in Ghana (in 2021). Each type of flyer will be distributed in about the same number of communities. The actual locations where flyers are distributed will be tracked using GPS information collected by the team handing out flyers.

A default outreach campaign will be carried out in Accra, Kumasi and Tamale (in 2020) and nationwide (in 2021) using radio commercials and social media activity. All campaign materials contain key information about the ProfArts program and information on where and when to register. The mobilization campaign is implemented by local partner Lyme Haus Solutions, under contract with GIZ Ghana and in cooperation with Ghana Institution of Engineering (GhIE).

Mobilized artisans register at centers in and near target communities, and they complete a baseline questionnaire at that time. Artisans may also be registered and interviewed at construction sites or other places of artisan employment. The focus of the ProfArts program is on artisans in the Ghanaian construction sector, with a focus on masonry, plumbing, steel bending, metal work/welding, electrical installation, roofing/carpentry, glazing, tiling, painting, and AC technology, and artisans need to have obtained a Proficiency I certificate or have completed an apprenticeship in order to be eligible for program benefits. The registration process is organized and carried out in cooperation with our local partner GhIE.

For the analysis of the effects of the ProfArts program, we will then randomly assign eligible registered artisans to control and treatment groups, within blocks formed based on location and respondents' socioeconomic characteristics. If not all beneficiaries can be provided a complete set of planned benefits, we may randomly assign artisans to specific program components.

Selected artisans will be offered a number of benefits over a time period of about six months. These include (i) top-up training opportunities, including financial literacy training, (ii) certification and licensing through a national craftsmen's register (GhIE), (iii) access to a mobile application to connect with clients (Vodafone Ghana), (iv) state-of-the-art tools and related training (Robert Bosch Ghana), and (v) personal protective equipment and technical advice on occupational health and safety measures (UVEX Arbeitsschutz).

Approximately six months after the end of each benefits period, we conduct endline interviews with the relevant batch of treated artisans and those in the control group, provided they have agreed to be contacted again. We will also receive administrative data provided by our partners concerning beneficiaries' program participation.

The random assignment will be done by PIs with a replicable procedure using statistical software.

For the assignment of recruitment content, the unit of randomization is the community. For the assignment of benefits to eligible artisans, the unit of randomization is the individual artisan. Depending on the extent to which artisans belong to the same community or business, assignment may be clustered at an appropriate geographical unit/business.

Yes

The assignment of recruitment strategies is clustered at the community level. We plan to include 113 different communities in Accra, Kumasi and Tamale in 2020 and a substantially larger number in locations throughout Ghana in 2021.

For the evaluation of ProfArts program benefits, we will enroll up to 20,000 artisans over the course of the study, which will possibly but not necessarily be assigned separately and in as many clusters. The assignment of program benefits may be clustered if registered artisans come from the same geographical unit or business and benefits cannot be effectively administered to only some of these artisans.

We anticipate registering up to 20,000 artisans over the course of the project in 2020 and 2021, drawn from a large number of communities throughout Ghana.

For the intervention concerning recruitment strategies, each of the three flyer designs will be distributed in a third of the target communities. For the intervention concerning ProfArts program benefits, half of the eligible artisans will be offered treatment, i.e. up to 10,000 artisans, with 10,000 artisans in the control group.

In a basic group means comparison with 80% power, a significance level of 0.05, and unit standard deviations, the MDE is .04 for the total sample of 20,000 artisan subjects.

Pre-analysis Plan, Evaluation of ProfArts