Evaluation of a Program for the Professionalization of Artisans (ProfArts) in Ghana

Last registered on December 02, 2020

Pre-Trial

Trial Information

General Information

Title
Evaluation of a Program for the Professionalization of Artisans (ProfArts) in Ghana
RCT ID
AEARCTR-0006842
Initial registration date
December 01, 2020
Last updated
December 02, 2020, 11:23 AM EST

Locations

Region

Primary Investigator

Affiliation
RWI - Leibniz Institute for Economic Research

Other Primary Investigator(s)

PI Affiliation
German Institute for Global and Area Studies (GIGA)
PI Affiliation
German Institute for Global and Area Studies (GIGA)
PI Affiliation
German Institute for Global and Area Studies (GIGA)
PI Affiliation
German Institute for Global and Area Studies (GIGA)
PI Affiliation
RWI - Leibniz Institute for Economic Research

Additional Trial Information

Status
In development
Start date
2020-11-23
End date
2022-12-31
Secondary IDs
Abstract
Training and professionalization interventions are an important vehicle for economic support within the development assistance landscape. Yet rigorous quantitative impact evaluations of such programs remain scarce, especially in developing countries. In order to help fill this research gap, we will conduct a rigorous impact evaluation of a program for the Professionalization of Artisans (ProfArts) in Ghana. The program will deliver top-up training, licensing, certification, and related benefits to up to 10,000 artisans drawn from the Ghanaian construction sector, with beneficiaries to be randomly selected from up to 20,000 baseline respondents. In a first step, we examine the effects of randomly assigned recruitment content on application rates, the composition of the applicant pool, and downstream program outcomes. In a second step, we use a randomized controlled trial to estimate effects of the program on four groups of outcomes: (i) employment, measured e.g. in terms of job retention, acquisition and lengths of employment spells, (ii) job quality and quality of life, including e.g. earnings and workplace conditions, (iii) mobility, e.g. migration intentions and behaviors, and (iv) firm-level outcomes, e.g. firm performance and employment.
External Link(s)

Registration Citation

Citation
Beber, Bernd et al. 2020. "Evaluation of a Program for the Professionalization of Artisans (ProfArts) in Ghana." AEA RCT Registry. December 02. https://doi.org/10.1257/rct.6842-1.0
Experimental Details

Interventions

Intervention(s)
The project encompasses two interventions. First, we assess the effects of different recruitment content on outreach and program outcomes. Second, we examine the impacts of ProfArts program participation on a comprehensive set of employment-related indicators.

For the first intervention, different mobilization content is randomly assigned to communities where artisans are to be recruited. The ProfArts project's overall recruitment strategy consists of (a) a default outreach campaign, which sees radio spots and social media content disseminated nationwide, and (b) a flyer campaign, with three different versions randomly assigned to communities. The different versions emphasize either (i) the potential for higher earnings as a result of the program, or (ii) the fact that artisans may learn new or update existing skills, or (iii) the program's technological aspects and offer of a digital platform. All versions of the mobilization campaign explain the core program structure and provide information on where and when to register.

For the second intervention, the ProfArts benefits package is randomly assigned to up to 10,000 eligible artisans out of a pool of up to 20,000 applicants. The program consists of a variety of benefits, including skills and financial literacy training, certification and licensing, provision of state-of-the-art tools and personal protective equipment, and access to a mobile application to connect artisans with clients. If program partners are unable to deliver a complete set of benefits to all beneficiaries, we may randomize the composition of benefits packages.
Intervention Start Date
2021-01-01
Intervention End Date
2021-12-31

Primary Outcomes

Primary Outcomes (end points)
For the recruitment content intervention, the primary outcomes are (i) the number of artisans recruited per flyer of each type distributed, or an equivalent metric, (ii) the make-up of the recruited applicant pool, including characteristics such as age, employment status, educational attainment, social preferences, and personality traits, and (iii) downstream outcomes, including program completion rates for selected beneficiaries from different communities, and economic outcomes such as those used to evaluate the ProfArts program itself.

For the ProfArts program intervention, the primary outcomes are (i) employment, measured e.g. in terms of job retention, acquisition and lengths of employment spells, (ii) job quality and quality of life, including e.g. earnings and workplace conditions, (iii) mobility, e.g. migration intentions and behaviors, as well as (iv) firm-level outcomes, e.g. performance and employment at the firm level.
Primary Outcomes (explanation)
Concerning the possible effects of different recruitment content (i.e. messaging displayed on flyers), we measure (i) uptake rates to help us better understand applicants' motivations and how to design resource-efficient and well-targeted interventions, (ii) applicant pool characteristics to help explain how recruitment campaigns can lead to certain person types being represented disproportionately among beneficiaries (e.g. flyers highlighting monetary gains could attract relatively more profit-oriented individuals with lower expressed levels of altruism), and (iii) downstream outcomes to better understand how selection effects limit the external validity of RCTs and the long-term effects of recruitment content.

Concerning program effects, our measures reflect the program's own key performance targets and our expectation that the intervention will affect economic livelihoods in a variety of ways, in particular concerning employment, job quality, and mobility.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary outcomes include variables that we will use to investigate mechanisms underpinning the interventions' effects, such as the role of knowledge and information, product and production innovations, usage of digital platforms for client and order acquisition, changes in subjects' professional networks, their client base, or their ability to acquire inputs at low cost, etc.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Concerning the analysis of recruitment strategies, we randomly assign one of three flyer types at the community level, within spatiotemporal blocks given by administrative areas and scheduled days of flyer distribution. Outreach and recruitment activities will initially focus on 113 communities in Accra, Kumasi, and Tamale (in 2020), and will subsequently expand to other locations in Ghana (in 2021). Each type of flyer will be distributed in about the same number of communities. The actual locations where flyers are distributed will be tracked using GPS information collected by the team handing out flyers.

A default outreach campaign will be carried out in Accra, Kumasi and Tamale (in 2020) and nationwide (in 2021) using radio commercials and social media activity. All campaign materials contain key information about the ProfArts program and information on where and when to register. The mobilization campaign is implemented by local partner Lyme Haus Solutions, under contract with GIZ Ghana and in cooperation with Ghana Institution of Engineering (GhIE).

Mobilized artisans register at centers in and near target communities, and they complete a baseline questionnaire at that time. Artisans may also be registered and interviewed at construction sites or other places of artisan employment. The focus of the ProfArts program is on artisans in the Ghanaian construction sector, with a focus on masonry, plumbing, steel bending, metal work/welding, electrical installation, roofing/carpentry, glazing, tiling, painting, and AC technology, and artisans need to have obtained a Proficiency I certificate or have completed an apprenticeship in order to be eligible for program benefits. The registration process is organized and carried out in cooperation with our local partner GhIE.

For the analysis of the effects of the ProfArts program, we will then randomly assign eligible registered artisans to control and treatment groups, within blocks formed based on location and respondents' socioeconomic characteristics. If not all beneficiaries can be provided a complete set of planned benefits, we may randomly assign artisans to specific program components.

Selected artisans will be offered a number of benefits over a time period of about six months. These include (i) top-up training opportunities, including financial literacy training, (ii) certification and licensing through a national craftsmen's register (GhIE), (iii) access to a mobile application to connect with clients (Vodafone Ghana), (iv) state-of-the-art tools and related training (Robert Bosch Ghana), and (v) personal protective equipment and technical advice on occupational health and safety measures (UVEX Arbeitsschutz).

Approximately six months after the end of each benefits period, we conduct endline interviews with the relevant batch of treated artisans and those in the control group, provided they have agreed to be contacted again. We will also receive administrative data provided by our partners concerning beneficiaries' program participation.
Experimental Design Details
Not available
Randomization Method
The random assignment will be done by PIs with a replicable procedure using statistical software.
Randomization Unit
For the assignment of recruitment content, the unit of randomization is the community. For the assignment of benefits to eligible artisans, the unit of randomization is the individual artisan. Depending on the extent to which artisans belong to the same community or business, assignment may be clustered at an appropriate geographical unit/business.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
The assignment of recruitment strategies is clustered at the community level. We plan to include 113 different communities in Accra, Kumasi and Tamale in 2020 and a substantially larger number in locations throughout Ghana in 2021.

For the evaluation of ProfArts program benefits, we will enroll up to 20,000 artisans over the course of the study, which will possibly but not necessarily be assigned separately and in as many clusters. The assignment of program benefits may be clustered if registered artisans come from the same geographical unit or business and benefits cannot be effectively administered to only some of these artisans.
Sample size: planned number of observations
We anticipate registering up to 20,000 artisans over the course of the project in 2020 and 2021, drawn from a large number of communities throughout Ghana.
Sample size (or number of clusters) by treatment arms
For the intervention concerning recruitment strategies, each of the three flyer designs will be distributed in a third of the target communities. For the intervention concerning ProfArts program benefits, half of the eligible artisans will be offered treatment, i.e. up to 10,000 artisans, with 10,000 artisans in the control group.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
In a basic group means comparison with 80% power, a significance level of 0.05, and unit standard deviations, the MDE is .04 for the total sample of 20,000 artisan subjects.
IRB

Institutional Review Boards (IRBs)

IRB Name
German Institute for Global and Area Studies (GIGA), Ethics Committee
IRB Approval Date
2020-11-24
IRB Approval Number
03/2020
IRB Name
University of Ghana, Ethics Committee for the Humanities (ECH)
IRB Approval Date
2020-11-25
IRB Approval Number
ECH068/20-21
Analysis Plan

Analysis Plan Documents

Pre-analysis Plan, Evaluation of ProfArts

MD5: b74fb0be465a91c02c4f3fc3bf295fe2

SHA1: 70bcaf4d1554239eccdfcbf264d7dda48b50be56

Uploaded At: December 01, 2020