x

We are happy to announce that all trial registrations will now be issued DOIs (digital object identifiers). For more information, see here.
Why Do Skilled Immigrants Struggle in the Labor Market? A Field Experiment with Thirteen Thousand Resumes
Last registered on April 29, 2016

Pre-Trial

Trial Information
General Information
Title
Why Do Skilled Immigrants Struggle in the Labor Market? A Field Experiment with Thirteen Thousand Resumes
RCT ID
AEARCTR-0001149
Initial registration date
April 29, 2016
Last updated
April 29, 2016 12:21 PM EDT
Location(s)
Region
Primary Investigator
Affiliation
University of Toronto
Other Primary Investigator(s)
Additional Trial Information
Status
Completed
Start date
2008-04-01
End date
2009-09-30
Secondary IDs
Abstract
Thousands of randomly manipulated resumes were sent in response to online job postings in Toronto to investigate why immigrants, allowed in based on skill, struggle in the labor market. The study finds substantial discrimination across a variety of occupations towards applicants with foreign experience or those with Indian, Pakistani, Chinese, and Greek names compared with English names. Listing language fluency, multinational firm experience, education from highly selective schools, or active extracurricular activities had no diminishing effect. Recruiters justify this behavior based on language skill concerns but fail to fully account for offsetting features when listed.
External Link(s)
Registration Citation
Citation
Oreopoulos, Philip. 2016. "Why Do Skilled Immigrants Struggle in the Labor Market? A Field Experiment with Thirteen Thousand Resumes." AEA RCT Registry. April 29. https://doi.org/10.1257/rct.1149-1.0.
Former Citation
Oreopoulos, Philip. 2016. "Why Do Skilled Immigrants Struggle in the Labor Market? A Field Experiment with Thirteen Thousand Resumes." AEA RCT Registry. April 29. https://www.socialscienceregistry.org/trials/1149/history/8009.
Experimental Details
Interventions
Intervention(s)
I examined the degree to which employer discrimination is an explanation for why skilled immigrants struggle in the Canadian labor market. Thousands of resumes with randomly assigned characteristics were sent out to job postings in Toronto and Montreal that requested three to seven years of experience and an undergraduate degree. Thirty percent of resumes were randomly selected to have English-sounding names (such as Emily Wilson or John Martin) and the remaining resumes had Chinese, Indian, Pakistani, or Greek names. To study employers’ perceptions of education and experience, resumes with foreign names were randomly assigned to have either a Canadian or foreign graduate degree or Canadian or foreign work experience. Some resumes were randomly selected to include other information like language skills, an undergraduate degree from a prestigious university, experience at a respected multinational firm, or a graduate degree. I compared callback rates between different types of resumes to estimate how ethnicity and other resume characteristics affect employers’ decisions to contact applicants. Later, a random sample of employers from the study was contacted to investigate possible reasons for discrimination against ethnic-sounding names.
Intervention Start Date
2008-04-01
Intervention End Date
2009-09-30
Primary Outcomes
Primary Outcomes (end points)
Callback rates for job postings
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
Thousands of randomly created resumes were sent by email in response to job postings across multiple occupations in the Greater Toronto Area between April and November 2008. An additional set of resumes was sent across Toronto, and Montreal between February and September 2009 in order to improve precision and consider effects from adding other attributes. The resumes were designed to plausibly represent typical immigrants who arrived recently under the Canadian point system from China, India, and Pakistan (the current top three source countries) and Britain, as well as non-immigrants with and without ethnic-sounding names (including Greek names). They were constructed after consulting actual resumes of recent immigrants and online submissions. The sample of jobs I applied to represent all jobs posted during these periods that accepted applications via direct e-mail and generally required three to seven years of experience and an undergraduate degree. Positions that specifically required at least a graduate degree, North American experience, or certification were ignored.

With few exceptions, four resumes were sent to each employer over a two- to three-day period in random order. The first represented an applicant with an English sounding name, Canadian undergraduate education, and Canadian experience (Type 0). The second resume had instead a foreign sounding name (Chinese, Indian, Pakistani, or Greek), but still listed Canadian undergraduate education and Canadian experience (Type 1). The third resume included a foreign sounding name (Chinese, Indian, or Pakistani), corresponding foreign undergraduate degree, and Canadian experience (Type 2). The fourth included a foreign sounding name, foreign education, and some foreign experience (Type 3) or all foreign experience (Type 4). I also randomized applicants’ alma mater, whether the applicant listed being fluent in multiple languages (including French), whether they had additional Canadian education credentials, and whether their job experience was from well recognized multinational firms or large firms, or not. To address concerns whether employers shy away from resumes with foreign credentials because of additional costs in contacting references or concerns about legal working status, I also randomize a subset of resumes with foreign experience to list Canadian references (with a local telephone number) and explicitly list permanent residency status. The total sample size is 12,910 resumes, sent in response to 3,225 job postings. The trial was stratified on the candidate type (0-4) by job postings.

Work experiences were constructed from actual resumes accessible online. The descriptions were sufficiently altered to create distinct sets that would not be associated with actual people, but I also tried to maintain original overall content and form. Each resume listed the job title, job description, company name, and city location for an applicant’s three most recent jobs covering four to six years, with the first job beginning in the same year as the applicant’s undergraduate degree completion. The city listed was always the same (except for Type 3 resumes). Experience sets were constructed for 20 different occupation categories, almost all the same ones used by the online job site workopolis.com. In addition, company names were also independent of resume type for about half the sample. International companies were chosen wherever possible to keep the experience sets identical across immigrant and non-immigrant resumes except for location (for example ABC Inc., Toronto versus ABC Inc., Mumbai). In cases where no obvious international company was available, I picked closely related companies in size and industry. Alma mater was picked randomly from a list of about four universities in the same country as the applicant’s corresponding name and in the same proximity to the applicant’s location of experience. About half of the universities were listed in the 2008 QS World University Rankings’ Top 200. The other universities were less prestigious. Manipulation of this characteristic helps examine whether employers prefer applicants with degrees from Canada even in cases where, all else constant, other applicants have foreign degrees from arguably the most selective schools in China, India, or Pakistan. 20 percent of resumes, except those of Type 4, were randomly assigned Canadian master’s degrees.15 Master’s degrees were occupation specific and completed during the same three year period as the applicant’s most recent (Canadian) experience, so that it looked like the applicant was enrolled part-time while working full-time.

Language skills and extracurricular activities were also manipulated to help explore whether language or cultural concerns underlie callback differences. I randomly selected 20 percent of resumes in Toronto to list fluency in multiple languages and 60 percent of resumes in Montreal. Resumes with English or Greek sounding names listed fluency in English and French. The other resumes listed fluency in English, French, and the applicant’s mother tongue (Mandarin, Cantonese, Hindi, or Punjabi), depending on the applicant’s ethnic name origin. In addition, 60 percent of resumes listed active extracurricular activities. One of three possible sets was chosen listing characteristics such as volunteer initiative (e.g., Big Brother/Sister, Habitat for Humanity), social interests (e.g., competitive squash player, classical pianist) and proactive work skills (e.g., excellent common sense, judgment, and decision-making abilities).

Since resumes had to look different when sending to the same employer, I also randomized each applicant’s cover letter (i.e., a short, general message sent as a part of the e-mail text), and the e-mail subject line and the resume file name (resumes were saved as pdf files unless word documents were specifically requested). I randomized each resume’s layout, residential address and telephone number (all possibilities were within Toronto or Montreal. Each applicant listed three previous jobs, with earlier years of experience being over two, three, or four years for each particular job, and with the most recent job always being listed as starting from the year the bachelor’s degree was obtained. I randomized each applicant’s e-mail address and resume profile, which was listed near the top of the resume. Within each occupation, profiles were selected randomly from five sets. On average, all resumes are the same across description of job experience, years of schooling, style of resume, and cover e-mail.

Multiple telephone numbers and two e-mail accounts for each name were set up to collect employer responses. Employers who telephoned an applicant received the same automatically generated message mentioning the number dialed and a request to leave a message. Messages and e-mails were recorded and redirected to a single e-mail address. Responses were classified as callbacks, if the employer requested an applicant to contact them (not just for clarification). Responses were classified as requests for interviews, if one was specifically mentioned. Employers that contacted an applicant twice were contacted themselves during off-hours by e-mail or phone message and told that the applicant had accepted another position and was no longer looking for employment. I also recorded measures of language and social skills associated with each job using the Occupational Information Network (O*NET). The purpose was to examine whether callback differences across resume types differ by the extent to which jobs require language or social skills. For each job title, I recorded the O*NET’s corresponding skill measure for speaking, writing and, social perceptiveness.
Experimental Design Details
Randomization Method
A program by Johana N. Lahey and Ryan A. Beasley (2009) was used to randomly select the characteristic codes of each resume. Microsoft Office was then used to transform these choices into text and mail merge them into actual resume templates.
Randomization Unit
Job postings
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
12,910 resumes
Sample size: planned number of observations
12,910 resumes
Sample size (or number of clusters) by treatment arms
Treatment groups: 3,615 resumes in Type 1, 2,266 resumes in Type 2, 2,021 resumes in Type 3, 1,982 resumes in Type 4
Control group: 3,026 resumes in Type 0
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
University of Toronto Ethics Review Committee
IRB Approval Date
2007-12-14
IRB Approval Number
Protocol Reference 21501
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
Yes
Intervention Completion Date
September 30, 2009, 12:00 AM +00:00
Is data collection complete?
Yes
Data Collection Completion Date
September 30, 2009, 12:00 AM +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
12,910 resumes
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
12,910 resumes
Final Sample Size (or Number of Clusters) by Treatment Arms
Treatment groups: 3,615 resumes in Type 1, 2,266 resumes in Type 2, 2,021 resumes in Type 3, 1,982 resumes in Type 4 Control group: 3,026 resumes in Type 0
Data Publication
Data Publication
Is public data available?
Yes
Program Files
Program Files
Yes
Reports and Papers
Preliminary Reports
Relevant Papers
Abstract
Thousands of randomly manipulated resumes were sent in response to online job postings in Toronto to investigate why immigrants, allowed in based on skill, struggle in the labor market. The study finds substantial discrimination across a variety of occupations towards applicants with foreign experience or those with Indian, Pakistani, Chinese, and Greek names compared with English names. Listing language fluency, multinational firm experience, education from highly selective schools, or active extracurricular activities had no diminishing effect. Recruiters justify this behavior based on language skill concerns but fail to fully account for offsetting features when listed.
Citation
Oreopoulos, Philip. 2011. “Why Do Skilled Immigrants Struggle in the Labor Market? A Field Experiment with Thirteen Thousand Resumes.” American Economic Journal: Economic Policy 3(4): 148-171.