|
Field
Trial Title
|
Before
Do Online Service-pitchers' Words of Certainty and Rapport-building Enhance Their Hiring?
|
After
Pitch Attributes on Employer's Hiring Decisions
|
|
Field
Trial Start Date
|
Before
October 18, 2021
|
After
November 12, 2021
|
|
Field
Trial End Date
|
Before
October 25, 2021
|
After
November 15, 2021
|
|
Field
Last Published
|
Before
October 08, 2021 05:04 PM
|
After
November 10, 2021 12:38 AM
|
|
Field
Intervention (Public)
|
Before
In online labor marketplaces (e.g., Freelancer.com and Upwork.com), after an employee (i.e., buyer) post a job, workers (i.e., service providers or pitchers) who are interested in the job will bid by specifying the amount and time they need to complete the job. These workers normally send sale pitches (texts) to this employee.
The purpose of this research is to explore how certainty words and rapport-building words in worker's job-seeking pitches affect employer's hiring decisions.
I test the following 2 hypotheses:
Hypothesis 1: The relationship between online service-pitchers' words of certainty and their attractiveness to buyers is in the form of an inverted U-shaped curve, such that online service-pitchers' likelihood of being selected over competing sellers by buyers is greatest when their pitch-certainty is moderate rather than when it is low or high—hereafter referred to as the "TMGT-effect of pitch-certainty."
Hypothesis 2: The TMGT-effect of pitch-certainty is stronger for online service-pitchers whose pitches contain a lower (rather than higher)proportion of rapport-building words.
I have access to secondary dataset from one of largest online labor platform in the US. The initial results support our hypotheses. I will use an experiment to verify my findings.
This pre-registered experiment has a 3 (certainty words levels: High, medium, and low) by 2 (rapport-building words: Absent VS. present) by 2 (geographical (dis)similarity: Same VS Different) between-subject design.
a. Definitions
• Certainty words are words that show assertiveness of language used by authors, words such as always, absolutely, and assure (Fast & Funder, 2008). Certainty words variable are qualified into three levels based on the percentage of certainty words in the pitches. Since this experiment is used to verify findings obtained using archival data, we use percentile values of certainty words in our archival data as base to define certainty levels. Specifically, we define pitches with no less than 6 percent of certainty words (value of 75 percentile of certainty words percentage in our archival data) as high certainty, pitches with certainty words percentage values between no less than 1 percent (value of 26 percentile of certainty words percentage in our archival data) and 6 percent (value of 74 percentile of certainty words percentage in our archival data) as medium certainty, and pitches with at most 1 percent of certainty words (value of 25 percentile of certainty words percentage in our archival data) as low certainty.
• Rapport-building words are words that build bond between communicators (Gremler & Gwinner, 2008). Rapport-building words include four dimensions: 1. Greeting words (e.g., “hello” and “how are you”). 2. Mentioning communication recipient's names. 3. Politeness words (e.g., “thanks” and “regards”). And 4. Worker’s positive emotion about himself (e.g., “excellent” and “successful”). We define high and low rapport building using the similar approach to quantify certainty levels. Specifically, we define pitches with at least 6 percent of rapport-building word (value of 75 percentile of rapport-building words percentage in archival data) as high rapport-building and at most 2 percent of rapport-building words (value of 25 percentile of rapport-building words percent in archival data) as low rapport-building
• Geographical (dis) similarity measure whether employers and job-seeking workers are from the same geographical regions (Lin, Liu, & Viswanathan, 2018).
Thus, the treatments are the different levels of certainty words, different levels of rapport-building words, and whether employer and job-seeking workers come from same regions.
References
Fast, L. A., & Funder, D. C. (2008). Personality as manifest in word use: correlations with self-report, acquaintance report, and behavior. Journal of Personality and Social Psychology, 94(2), 334.
Gremler, D. D., Gwinner, K. P. (2008). Rapport-building behaviors used by retail employees. Journal of Retailing, 84(3), 308-324
Lin, M., Liu, Y., & Viswanathan, S. (2018). Effectiveness of reputation in contracting for customized production: Evidence from online labor markets. Management Science, 64(1), 345-359.
|
After
In online labor marketplaces (e.g., Freelancer.com and Upwork.com), after an employee (i.e., buyer) post a job, workers (i.e., service providers or pitchers) who are interested in the job will bid by specifying the amount and time they need to complete the job. These workers normally send sale pitches (texts) to this employee.
The purpose of this research is to explore how certainty words and rapport-building words in worker's job-seeking pitches affect employer's hiring decisions.
I test the following 2 hypotheses:
Hypothesis 1: The relationship between online service-pitchers' words of certainty and their attractiveness to buyers is in the form of an inverted U-shaped curve, such that online service-pitchers' likelihood of being selected over competing sellers by buyers is greatest when their pitch-certainty is moderate rather than when it is low or high—hereafter referred to as the "TMGT-effect of pitch-certainty."
Hypothesis 2: The TMGT-effect of pitch-certainty is stronger for online service-pitchers whose pitches contain a lower (rather than higher)proportion of rapport-building words.
I have access to secondary dataset from one of largest online labor platform in the US. The initial results support our hypotheses. I will use an experiment to verify my findings and a boundary condition (i.e., job complexity).
This pre-registered study has a 3 (high certainty word ratio (rapport-building words: Absent VS. present), medium certainty word ratio (rapport-building words: Absent VS. present), and low certainty word ratio (rapport-building words: Absent VS. Present)) by 2 (project complexity: Absent VS. Present) between-subject design.
a. Definitions
• Certainty words are words that show assertiveness of language used by authors, words such as always, absolutely, and assure (Fast & Funder, 2008).
• Rapport-building words are words that build bonds between communicators (Gremler & Gwinner, 2008). We measure rapport-building in two dimensions (Tickle-Degnen & Rosenthal, 1990):
1) Mutual attentiveness. This dimension shows the expression of mutual attention and involvement with the other, including greeting words (e.g., “hello” and “how are you”) and mentioning communication recipients' names.
2) Positivity, feelings of mutual friendliness and warmth. This dimension includes politeness words (e.g., “thanks” and “regards”) and small talk components (e.g., emoji and exclamation marks).
• Project complexity
We use the presence or absence of the following note to create project complexity:
Website design-jobs often experience delays due to complexities associated with the programmer, client’s clarity about needs, and communications between them.
Thus, the treatments are the different levels of certainty words, different levels of rapport-building words, and different level of job complexity.
References
Fast, L. A., & Funder, D. C. (2008). Personality as manifest in word use: correlations with self-report, acquaintance report, and behavior. Journal of Personality and Social Psychology, 94(2), 334.
Gremler, D. D., Gwinner, K. P. (2008). Rapport-building behaviors used by retail employees. Journal of Retailing, 84(3), 308-324
Tickle-Degnen, L., & Rosenthal, R. (1990). The nature of rapport and its nonverbal correlates. Psychological inquiry, 1(4), 285-293
|
|
Field
Intervention Start Date
|
Before
October 18, 2021
|
After
November 12, 2021
|
|
Field
Intervention End Date
|
Before
October 25, 2021
|
After
November 15, 2021
|
|
Field
Primary Outcomes (End Points)
|
Before
Hiring decision measured using Likert Scale 1-7. The hiring tendency increases from 1 to 7. 1 is least likely to hire and 7 means most likely to hire.
|
After
The likelihood of hiring is measured using Likert Scale 1-5. The hiring tendency increases from 1 to 5. 1 is Not at all and 5 means Extremely much.
|
|
Field
Primary Outcomes (Explanation)
|
Before
Hiring decision measured using Likert Scale 1-7. The hiring tendency increases from 1 to 7. 1 is least likely to hire and 7 means most likely to hire.
|
After
The likelihood of hiring is measured using Likert Scale 1-5. The hiring tendency increases from 1 to 5. 1 is Not at all and 5 means Extremely much.
|
|
Field
Experimental Design (Public)
|
Before
This pre-registered study will have a 3 (certainty words levels: High VS Medium VS Low) by 2 (rapport-building words levels: High VS Low) by 2 (geographical (dis)similarity: Same VS Different) subjects design.
The detailed explanations about certainty words, rapport-building words, and geographical (dis)similarity are provided in Intervention section.
The experiment will be conducted through Amazon Mechanical Turk platform.
|
After
This pre-registered study has a 3 (high certainty word ratio (rapport-building words: Absent VS. present), medium certainty word ratio (rapport-building words: Absent VS. present), and low certainty word ratio (rapport-building words: Absent VS. Present)) by 2 (project complexity: Absent VS. Present) between-subject design.
The detailed explanations about certainty words, rapport-building words, and job complexity are provided in Intervention section.
The experiment will be conducted through Amazon Mechanical Turk platform.
|
|
Field
Randomization Method
|
Before
After a worker from Amazon Mechanical Turk platform wants to participate. This worker can click the provided link and go to Qualtrics to do the survey.
Each participant will be randomly assigned one pitch. Each participant can only read one pitch and answer questions.
|
After
After a worker from Amazon Mechanical Turk platform wants to participate. This worker can click the provided link and go to Qualtrics to do the survey.
Each participant will be randomly assigned one set of pitches (two different pitches from two different service providers) and answer some questions. Each participant can only participate once.
|
|
Field
Randomization Unit
|
Before
There are 12 different versions of a sale pitch. Each version is a randomization unit.
|
After
There are 6 different sets of sale pitches (each set has two different pitches). Each set is a randomization unit.
|
|
Field
Planned Number of Clusters
|
Before
800 participants.
|
After
1000 participants.
|
|
Field
Planned Number of Observations
|
Before
800 participants.
|
After
1000 participants.
|
|
Field
Sample size (or number of clusters) by treatment arms
|
Before
800 participants.
|
After
1000 participants.
|
|
Field
Intervention (Hidden)
|
Before
This pre-registered study will have a 3 (certainty words levels: High VS Medium VS Low) by 2 (rapport-building words levels: High VS Low) by 2 (geographical (dis)similarity: Same VS Different) subjects design.
• Certainty words are words that show assertiveness of language used by authors, words such as always, absolutely, and assure (Fast & Funder, 2008). Certainty words variable are qualified into three levels based on the percentage of certainty words in the pitches. Since this experiment is used to verify findings obtained using archival data, we use percentile values of certainty words in our archival data as base to define certainty levels. Specifically, we define pitches with no less than 6 percent of certainty words (value of 75 percentile of certainty words percentage in our archival data) as high certainty, pitches with certainty words percentage values between no less than 1 percent (value of 26 percentile of certainty words percentage in our archival data) and 6 percent (value of 74 percentile of certainty words percentage in our archival data) as medium certainty, and pitches with at most 1 percent of certainty words (value of 25 percentile of certainty words percentage in our archival data) as low certainty.
• Rapport-building words are words that build bond between communicators (Gremler & Gwinner, 2008), such as greeting words (e.g., “hello” and “how are you”), mentioning communication-recipients’ names, politeness-words (e.g., “thanks” and “regards”), and positive emotion words (e.g., “agree” and “easy”). We define high and low rapport building using the similar approach as certainty levels. Specifically, we define pitches with at least 4.6 percent of rapport-building word (value of 75 percentile of rapport-building words percentage in archival data) as high rapport-building and at most 1.3 percent of rapport-building words (value of 25 percentile of rapport-building words percent in archival data) as low rapport-building
• Geographical (dis)similarity measure whether employers and workers are from the same geographical regions (Lin, Liu, & Viswanathan, 2018). We use Amazon Mechanical Turk for the experiment. Since around 60% workers are from US and around 30% of workers are from India at this platform (MTurk Tracker, 2021), we use either US or Indian common names to show geographical (dis)similarity.
Thus, the treatments are the different levels of certainty words, different levels of rapport-building words, and whether employer and job-seeking workers come from same regions.
b. Pitches Creation
The pitches are created using following procedure:
1) One pitch with medium levels of certainty is randomly selected from our archival data.
2) The selected pitch is revised into 12 different versions (Table 1):
(1) Pitch contains high-level certainty and rapport-building words and is written by a worker with a gender-neutral American name.
(2) Pitch contains high-level certainty and rapport-building words and is written by a worker with a gender-neutral Indian name.
(3) Pitch contains medium-level certainty and high-level rapport-building words and is written by a worker with a gender-neutral American name.
(4) Pitch contains medium-level certainty and high-level rapport-building words and is written by a worker with a gender-neutral Indian name.
(5) Pitch contains low-level certainty and high-level rapport-building words and is written by a worker with a gender-neutral American name.
(6) Pitch contains low-level certainty and high-level rapport-building words and is written by a worker with a gender-neutral Indian name.
(7) Pitch contains high-level certainty and low-level rapport-building words and is written by a worker with a gender-neutral American name.
(8) Pitch contains high-level certainty and low-level rapport-building words and is written by a worker with a gender-neutral Indian name.
(9) Pitch contains medium-level certainty and low-level rapport-building words and is written by a worker with a gender-neutral American name.
(10) Pitch contains medium-level certainty and low-level rapport-building words and is written by a worker with a gender-neutral Indian name.
(11) Pitch contains low-level certainty and low-level rapport-building words and is written by a worker with a gender-neutral American name.
(12) Pitch contains low-level certainty and low-level rapport-building words and is written by a worker with a gender-neutral Indian name.
References
Fast, L. A., & Funder, D. C. (2008). Personality as manifest in word use: correlations with self-report, acquaintance report, and behavior. Journal of Personality and Social Psychology, 94(2), 334.
Gremler, D. D., Gwinner, K. P. (2008). Rapport-building behaviors used by retail employees. Journal of Retailing, 84(3), 308-324
Lin, M., Liu, Y., & Viswanathan, S. (2018). Effectiveness of reputation in contracting for customized production: Evidence from online labor markets. Management Science, 64(1), 345-359.
MTurk Tracker. demographics.mturk-tracker.com. Retrieved May 2021.
|
After
This pre-registered study has a 3 (high certainty word ratio (rapport-building words: Absent VS. present), medium certainty word ratio (rapport-building words: Absent VS. present), and low certainty word ratio (rapport-building words: Absent VS. Present)) by 2 (project complexity: Absent VS. Present) between-subject design.
• Certainty words are words that show assertiveness of language used by authors, words such as always, absolutely, and assure (Fast & Funder, 2008).
• Rapport-building words are words that build bonds between communicators (Gremler & Gwinner, 2008). We measure rapport-building in two dimensions (Tickle-Degnen & Rosenthal, 1990):
1) Mutual attentiveness. This dimension shows the expression of mutual attention and involvement with the other, including greeting words (e.g., “hello” and “how are you”) and mentioning communication recipients' names.
2) Positivity, feelings of mutual friendliness and warmth. This dimension includes politeness words (e.g., “thanks” and “regards”) and small talk components (e.g., emoji and exclamation marks).
• Project complexity
We use the presence or absence of the following note to create project complexity:
Website design-jobs often experience delays due to complexities associated with the programmer, client’s clarity about needs, and communications between them.
Thus, the treatments are the different levels of certainty words, different levels of rapport-building words, and different level of job complexity.
b. Pitches Creation
The pitches are created using the following procedure:
1) One pitch with medium levels of certainty is randomly selected from our archival data.
2) The selected pitch is revised into three sets of different versions (Table 1):
a. Set 1: There are two pitches in this set. Both pitches contain high-level certainty. One pitch has rapport-building words, and another does not contain any.
b. Set 2: There are two pitches in this set. Both pitches contain medium-level certainty. One pitch has rapport-building words, and another does not contain any.
c. Set 3: There are two pitches in this set. Both pitches contain low-level certainty. One pitch has rapport-building words, and another does not contain any.
There are six different cases when we combine the sets of pitches with the presence or absence of project complexity.
References
Fast, L. A., & Funder, D. C. (2008). Personality as manifest in word use: correlations with self-report, acquaintance report, and behavior. Journal of Personality and Social Psychology, 94(2), 334.
Gremler, D. D., Gwinner, K. P. (2008). Rapport-building behaviors used by retail employees. Journal of Retailing, 84(3), 308-324
Tickle-Degnen, L., & Rosenthal, R. (1990). The nature of rapport and its nonverbal correlates. Psychological inquiry, 1(4), 285-293
|
|
Field
Did you obtain IRB approval for this study?
|
Before
No
|
After
Yes
|