The Effect of Organizational Religiosity on Hiring: A Field Study

I will randomly assign (and send) recruitment emails to potential applicants for a paralegal position at Simms Showers LLP. The treatment will emphasize Simms Showers’ religious identity and the control will not.

2023-12-01

2023-12-31

I have two main endpoints in this experiment: 1) clicks on the links (from the emails sent); 2) applications (from the emails sent).

None of my primary outcomes are constructed.

I have one secondary outcome: pro-social motivation.

This measure will be constructed from the four statements sent out in a link in a follow-up email to both candidates that applied and did not apply. It will be measured on a 7-point likert-type scale with anchors of 1 (disagree strongly) to 7 (agree strongly) as outlined in the “Intervention (Hidden)” section above.

I will randomly assign (and send) recruiting emails from a Simms Showers LLP email account to potential applicants from ZipRecruiter.

I will assign individual religiosity to these applicants based on the zip code they provide in their resume. To assign individual religiosity, the primary source upon which I will rely is as follows: "National Neighborhood Data Archive (NaNDA): Religious, Civic, and Social Organizations by ZIP Code Tabulation Area, United States, 2003-2017". As a proxy for individual religiosity, I will use the variable "popden_8131", which is a calculation of the number of religious organizations per 1000 people in the zip code. I will use the measure from 2017, the latest date available. I also may incorporate the United State Religious Census data, which provides religious adherence by county (I can tie zip code to county) and use that as a measure of individual religiosity. In either case, in the event that that zip code is not available, I will use the first zip code that appears on zip-codes.com for the location that the individual lists on their resume (or Zip Recruiter profile).

I will assign gender to these applicants using gender-api.com. I will assign white/non-white to these applicants using namsor.app. In both cases, I the assignment will be made using the individual’s first and last names. I will determine quality of the candidate based on by using two AI Resume Screeners (I will choose two from the top 5 of: 1) SkillPool; 2) CVViZ; 3. Manatal; 4. Vervoe and 5. Trakstar Hire). As a robustness check, I will also employ a research assistant to check some sample of these resumes.

As outlined above, pro-social motivation will be constructed from the four statements sent out in a link in a follow-up email to both candidates that applied and did not apply.

My first hypothesis (H1) is that there will be no difference in the number of applicants to the position among the general population (i.e. not considering religion). To test this hypothesis, I will run the following regressions:

Clicks = Intercept + Treatment
Applications = Intercept + Treatment

My second hypothesis (H2) is that as individual religiosity (as measured by religious adherence from county associated with zip code) increases, applicants are more likely to apply to the religious condition. To test this hypothesis, I will run the following regressions:

Clicks = Intercept + (Treatment * Individual Religiosity)
Applications = Intercept + (Treatment * Individual Religiosity)

My third hypothesis (H3) is that more high-quality candidates will apply to the religious condition. To test this hypothesis, I will include two sets of regressions. First:
Clicks = Intercept + (Treatment * Candidate Quality)
Applications = Intercept + (Treatment * Candidate Quality)

Second:
Clicks = Intercept + (Treatment * Candidate Quality) + Individual Religiosity
Applications = Intercept + (Treatment * Candidate Quality) + Individual Religiosity

I include the second set to ensure that it is not just employee-organization fit driving more high-quality candidates in the religious condition.

My fourth hypothesis (H4) is that more women will apply to the religious condition; To test this hypothesis (as will with H3), I will include two sets of regressions:

First:
Clicks = Intercept + (Treatment * Sex)
Applications = Intercept + (Treatment * Sex)

Second:
Clicks = Intercept + (Treatment * Sex) + Individual Religiosity
Applications = Intercept + (Treatment * Sex) + Individual Religiosity

My fifth hypothesis (H5) is that more non-whites will apply to the religious condition; To test this hypothesis (as will with H3 and H4), I will include two sets of regressions:

First:
Clicks = Intercept + (Treatment * White)
Applications = Intercept + (Treatment * White)

Second:
Clicks = Intercept + (Treatment * White) + Individual Religiosity
Applications = Intercept + (Treatment * White) + Individual Religiosity

My sixth hypothesis (H6) is that more pro-socially motivated people will apply to the religious condition relative to the control condition. To test this hypothesis, I will run the following two regressions:

Pro-Social Motivation = Treatment
Pro-Social Motivation = Treatment + Individual Religiosity

Here, as in the regressions above, I include a control for individual religiosity to ensure that it is not just employee-organization identity alignment driving the relationship.

I may conduct additional exploratory analyses.

I will randomly assign (and send) recruiting emails from a Simms Showers LLP email account to potential applicants from ZipRecruiter.

I will assign individual religiosity to these applicants based on the zip code they provide in their resume. To assign individual religiosity, the primary source upon which I will rely is as follows: "National Neighborhood Data Archive (NaNDA): Religious, Civic, and Social Organizations by ZIP Code Tabulation Area, United States, 2003-2017". As a proxy for individual religiosity, I will use the variable "popden_8131", which is a calculation of the number of religious organizations per 1000 people in the zip code. I will use the measure from 2017, the latest date available. I also may incorporate the United State Religious Census data, which provides religious adherence by county (I can tie zip code to county) and use that as a measure of individual religiosity. In either case, in the event that that zip code is not available, I will use the first zip code that appears on zip-codes.com for the location that the individual lists on their resume (or Zip Recruiter profile).

I will assign gender to these applicants using gender-api.com. I will assign white/non-white to these applicants using namsor.app. In both cases, I the assignment will be made using the individual’s first and last names. I will determine quality of the candidate based on by using two AI Resume Screeners (I will choose two from the top 5 of: 1) SkillPool; 2) CVViZ; 3. Manatal; 4. Vervoe and 5. Trakstar Hire). As a robustness check, I will also employ a research assistant to check some sample of these resumes.

As outlined above, pro-social motivation will be constructed from the four statements sent out in a link in a follow-up email to both candidates that applied and did not apply.

My first hypothesis (H1) is that there will be no difference in the number of applicants to the position among the general population (i.e. not considering religion). To test this hypothesis, I will run the following regressions:

Clicks = Intercept + Treatment
Applications = Intercept + Treatment

My second hypothesis (H2) is that as individual religiosity (as measured by religious adherence from county associated with zip code) increases, applicants are more likely to apply to the religious condition. To test this hypothesis, I will run the following regressions:

Clicks = Intercept + (Treatment * Individual Religiosity)
Applications = Intercept + (Treatment * Individual Religiosity)

My third hypothesis (H3) is that more high-quality candidates will apply to the religious condition. To test this hypothesis, I will include two sets of regressions. First:
Clicks = Intercept + (Treatment * Candidate Quality)
Applications = Intercept + (Treatment * Candidate Quality)

Second:
Clicks = Intercept + (Treatment * Candidate Quality) + Individual Religiosity
Applications = Intercept + (Treatment * Candidate Quality) + Individual Religiosity

I include the second set to ensure that it is not just employee-organization fit driving more high-quality candidates in the religious condition.

My fourth hypothesis (H4) is that more women will apply to the religious condition; To test this hypothesis (as will with H3), I will include two sets of regressions:

First:
Clicks = Intercept + (Treatment * Sex)
Applications = Intercept + (Treatment * Sex)

Second:
Clicks = Intercept + (Treatment * Sex) + Individual Religiosity
Applications = Intercept + (Treatment * Sex) + Individual Religiosity

My fifth hypothesis (H5) is that more non-whites will apply to the religious condition; To test this hypothesis (as will with H3 and H4), I will include two sets of regressions:

First:
Clicks = Intercept + (Treatment * White)
Applications = Intercept + (Treatment * White)

Second:
Clicks = Intercept + (Treatment * White) + Individual Religiosity
Applications = Intercept + (Treatment * White) + Individual Religiosity

My sixth hypothesis (H6) is that more pro-socially motivated people will apply to the religious condition relative to the control condition. To test this hypothesis, I will run the following two regressions:

Pro-Social Motivation = Treatment
Pro-Social Motivation = Treatment + Individual Religiosity

Here, as in the regressions above, I include a control for individual religiosity to ensure that it is not just employee-organization identity alignment driving the relationship.

I may conduct additional exploratory analyses.

The randomization will be done in an office by a computer. In particular, I will use excel to randomize and generate 500 recipients of the control condition and 500 of the treatment.

My unit (clusters) of randomization will be the individual.

No

I will send the initial recruiting emails to 1000 individuals. For the pro-social motivation follow-up, I will send emails to all who applied to both conditions. I do not know that number now because I do not know how many will apply

I will send the initial recruiting emails to 1000 individuals. For the pro-social motivation follow-up, I will send emails to all who applied to both conditions. I do not know that number now because I do not know how many will apply

500 individuals control email ; 500 individuals religious email

I calculate a minimum detectable effect size for my main outcome 0.114. I note that in the pilot that I ran my main outcome variable (the unit) was whether or not the applicant submitted an application. In the actual study, this will be one outcome (along with whether or not they clicked the link in the email). The standard deviation for this variable was 0.2866. The minimum detectable effect size in this case is 11,4% (since the outcome is binary), which corresponds to a 11.4 percentage point change in the likelihood of applying for the position.