Impact Evaluation: Data Analytics Training-to-Jobs Pathway Program

The “Tech for Jobs” program is delivered in Jordan by Correlation One, a U.S.-based workforce development organization. The program combines technology skills training, professional development, and job matching and placement services. The training component aims to enhance technology skills related to data science and data analytics, a set of skills well in demand in the ICT sector, which is an area of rapidly growing employment in the region, particularly for youth and women. It also aims to develop accurate information about the qualifications of applicants through assessments the results of which are shared with global IT employers, prepare job seekers to present themselves effectively in this job market through professional development, and provide job matching and placement services to overcome frictions.

2024-10-17

2025-03-01

We will examine three primary outcomes: employment, monthly earnings, and employment formality.

Employment will be a binary variable equal to one for a yes response to either: (1) “Have you participated in any employment during the past seven days?” Or (2) “Were you attached to a job in the past 7 days but were temporarily absent from it?”. Monthly earnings will be zero for those not currently employed. For those who are employed, monthly earnings will be based on the question “What are/were your typical net monthly earnings? In Jordanian dinar.” We will use the inverse hyperbolic sine transformation for earnings to account for potentially zero earnings and attenuate the effect of any outliers. Following International Labour Organization guidance, employment formality will be a binary variable defined based on social insurance coverage (question: “Do/did you have social insurance coverage connected to this job?” and employment (=1 if currently employed and “yes” to have social insurance connected to the job; =0 otherwise).

We will examine a number of secondary outcomes that may mediate or be downstream effects of our primary outcomes. We will specifically consider: employment with an offshore multinational corporation; employment in ICT occupations/industry, remote work, flexible hours, life satisfaction, subjective wellbeing, gender role attitudes, decision making, reservation wages, and reservation working conditions.

Employment with an offshore multinational corporation will be a binary variable, equal to zero if not employed or employed with a Jordanian company. This variable will equal one if the response is “Foreign” to “Is/was the firm you work for a Jordanian firm or a foreign firm?” Employment in ICT (occupations or industry) will be based on occupations (4-digit International Standard Classification of Occupations [ISCO] 2008 codes) and industries (1-digit International Standard Industrial Classification [ISIC] revision 4 codes). If an individual is classified as currently employed, in either an ICT industry or occupation, employment in ICT will equal one. Otherwise, it will be zero.

In terms of the extensive margin of work and working conditions we will investigate hours per week based on the question “What is/was your typical number of hours of work per week?.” This will be set to zero if not currently employed. We will also investigate the variation in working hours; this will be calculated based on two questions (asked of the currently employed) of “How many hours did you work last week (Sunday-Saturday)?” and “How many hours did you work in the week before last (Sunday-Saturday)?”. We will calculate variation as the absolute value of ((last weeks hours-previous week’s hours)/previous week’s hours). Those not employed will be given the mean of the employed in the control group, so that any changes in employment do not confound this variable. We will measure remote/work-from-home/hybrid work based on two questions: (1) “Do/did you work remotely?” (2) “Do/did you work from home?” which both have the responses: (1) “Yes, all of the time” (2) “Yes, some of the time from home and some in office (hybrid)” (3) “No.” Either of the yes responses to either of the questions for the currently employed will be coded as a 1 for this indicator; all other states are zero. Flexible hours will be measured based on the question “Are/were your hours flexible.” A yes response for the currently employed will be coded as a one and all other states zero.

We will measure wellbeing through scales for life satisfaction and subjective wellbeing. Specifically, we will measure life satisfaction based on Cantril’s ladder, which is described as “Please imagine a ladder with steps numbered from 0 at the bottom to 10 at the top. Suppose we say that the top of the ladder represents the best possible life for you and the bottom of the ladder represents the worst possible life for you.” Current life satisfaction using this prompt is based on the question “On which step of the ladder would you say you personally feel you stand at this time, assuming that the higher the step the better you feel about your life, and the lower the step the worse you feel about it?” Future life satisfaction will be based on the question “On which step of the ladder would you say you will stand on 5 years from now?” Subjective wellbeing will be based on the World Health Organization - Five Wellbeing Index (WHO-5) (Topp et al., 2015). The WHO-5 is based on the following prompts. “Over the past two weeks…
I have felt cheerful and in good spirits.
I have felt calm and relaxed.
I have felt active and vigorous.
I woke up feeling fresh and rested.
My daily life has been filled with things that interest me.”
Possible responses are:
All of the time = 5
Most of the time = 4
More than half of the time = 3
Less than half of the time = 2
Some of the time = 1
At no time = 0.
Adding the values across all five questions, and multiplying by four, the scale ranges from 0-100 (worst-best possible wellbeing).

The intervention may lead to changes in gender roles within the household. We will therefore explore gender role attitudes and decision-making measures. These items are based on questions previously used in Jordan to assess gender roles (Gauri et al., 2019; Krafft et al., 2021).
We will create a standardized gender role attitude factor using principle factor analysis, based on the following questions:
“Do you agree or disagree with the following statements?
1. A woman's work harms her reputation
2. Women should return from work before 5pm
3. It is okay to leave child under 5 years old with nursery for women to go to work
4. Men and women should be separated in the workplace
5. A woman’s place is not only in the household, but she should also have the option to work.
6. The husband should help his working wife with household chores.
7. When jobs are scarce, men should have more right to a job than women.”
With responses of (1) Strongly disagree (2) Disagree (3) Neutral (4) Agree (5) Strongly agree. For questions 1, 2, 4, and 7 we will reverse code before factoring so that higher values consistently mean more gender equitable attitudes. We will likewise construct a decision-making factor based on the following questions:
“Who in your family usually has the final say on the following decisions?
Purchasing major household items (refrigerator, stove, furniture)
Making household purchases for daily needs
Visits to family, friends or relative
Buying personal clothes”
With the possible responses:
“1. You alone
2. Spouse
3. Parents
4. You and your spouse jointly
5. Parents, In-laws, Relatives
6. You, your spouse, with parents or in-laws jointly
7. You and your parents
8. Others
98. Don't know
99. Not applicable.”
We will code any response that includes the individual him or her self (1, 4, 6, 7) with a dummy for being involved in the decision and conduct principal factor analysis to create a standardized factor from the four dummies.

The interventions may change the reservation wages, occupations, and working conditions of participants, which we will assess as secondary outcomes. We will assess changes in public and private sector reservation wages (separately) based on the questions (1) “What is the minimum net monthly income you would need to accept a position in the government or public enterprise sector?” (2) “What is the minimum net monthly income you would need to accept a position in the private sector?” Responses will be in Jordanian dinar. We will assess reservation occupations based on the number of yeses (response options are yes and no) to:
“Would you accept a job as a:
Public Sector Employee
Administrative Assistant
Data Entry
Bank Teller
Teacher
Customer Service
Telemarketing
Data analyst
Software developer
Business Intelligence Analyst
Data Scientist
Product Analyst
Operations Analyst
Financial Analyst
Data Engineer

We will assess reservation working conditions based on the number of dummies that change (either increases or decreases) and, alternatively, the number of yeses, for each of the following questions (response options are yes and no):
“Would you accept a job that:
Is entirely remote/Work-from-home
Requires being on your computer and/or phone all day
Requires traveling to an office but less than 30 minutes away
Requires traveling to an office but more than 30 minutes away
Requires working with the general public in person
Requires working with the general public online or on the phone
Is part-time (fewer than 30 hours per week)
Is more than 40 hours per week
Requires early or late or weekend work or meetings (before 8am or after 5pm or on the weekend)
Has flexible hours that you choose
Does not provide child care
Requires working in person with the opposite gender
Requires working in person in a very small workplace (fewer than 5 employees)
Requires working in person at different locations (outside a fixed establishment)
Does not provide social insurance”

Lastly, we will examine months of work since the date T1 started. This will be based on repeated questions at baseline of:
“We are now going to ask you about each job, lasting one month or more you had since you started working (or your current job).
What was the date of the start of job number [#]? (month, year)
Is this job your current job?
What is the date you left this job? (month, year)
Did you have another job after this job?
To collect a full labor market history. For subsequent waves we will collect information on whether continuing the same employment since baseline, and if not, any jobs in the interim using the same questions.

Program administrative data and sample

The top 3,300 female and top 3,300 male applicants based on the application assessment and passing the interview will be the sample for the RCT and randomized into the two treatments and control. Correlation One will provide contact and demographic information for applicants and their application assessment score and percentile, as well as the applicant identifier that is used throughout Correlation One’s administrative data (e.g., to track enrollment and attendance). Correlation One will share enrollment and attendance data for everyone included in the sample treatment arms.

Randomization

Randomization will take place after the baseline survey is completed. Although Correlation One will provide 6,600 contacts, we expect only 6,000 will successfully complete the baseline survey and agree to participate in the study. Randomization will be stratified by sex (3,000 top male and 3,000 top female applicants), in order to randomize each group equally across control, T1, and T2 (1,000 men, 1,000 women in each arm). Randomization is phased in. Control individuals will not be offered any Tech for Jobs training from Correlation One during the study period. Treatment individuals in T1 will be offered training promptly after baseline, and T2 will be offered their (reduced) training one month later.

Control individuals will be offered the full training, and reduced intervention (T2) individuals the technical training, if it is the more cost-effective option, after the entire study period is complete. Control individuals will therefore not have received Tech for Jobs training before endline data collection, but may seek out other trainings. Treatment individuals may also, in the end, decline to enroll, or fail to attend or complete the trainings, behaviors that Correlation One will track administratively and share with the research team.

Not available

Randomization done in office by a computer

Individuals

No

No clusters.

Although Correlation One will provide 6,600 contacts, we expect only 6,000 will successfully complete the baseline survey and agree to participate in the study.

Randomization will be stratified by sex (3,000 top male and 3,000 top female applicants), in order to randomize each group equally across control, T1, and T2 (1,000 men, 1,000 women in each arm).

The hypotheses guiding the power calculation include targeting a power of 80%, a significance level of 5%, complete program take-up, a total target sample size of 6,000 individuals (conservatively, ideally 6,600) equally distributed among two treatment groups and one comparison group, and accounting for a 5% attrition rate at each survey wave. We use the means of three outcomes derived from the Jordan Labor Market Panel Survey of 2016 for individuals aged 18-34 with a university degree. These outcomes include employment during the last 7 days, the inverse hyperbolic sine transformation of monthly wages (set as zero if not employed or not a wage-worker), and formal employment status (0 if not employed or not a wage-worker and 1 if having a social insurance coverage). According to these hypotheses, the power calculation reveals that the minimum detectable effect for the three outcomes across the four survey waves (baseline, end-of-training, 12 months after the start of training, and 20 months after the start of training surveys) ranges between 8% and 11%. This indicates that the study is well-equipped to identify significant changes in these crucial variables, establishing a robust foundation for evaluating the impact of the intervention with a high degree of statistical confidence.