Back to History Current Version

Pitch Attributes on Employer's Hiring Decisions

Last registered on October 08, 2021

Pre-Trial

Trial Information

General Information

Title
Do Online Service-pitchers' Words of Certainty and Rapport-building Enhance Their Hiring?
RCT ID
AEARCTR-0007847
Initial registration date
October 08, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
October 08, 2021, 5:04 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
Baruch College, City University of New York

Other Primary Investigator(s)

Additional Trial Information

Status
In development
Start date
2021-10-18
End date
2021-10-25
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Online labor market (e.g., Freelancer.com, Upworker.com, Fivre.com) provides opportunities for service-seeking buyers (i.e., employers) to find service-providers (i.e., workers) to work on jobs and for workers to find jobs. In this experiment, we examine how communication qualities (the proportion of certainty- and rapport building-words) in competing workers’ pitches (i.e., texts) influenced employer’ hiring decisions in online labor marketplaces. We already have findings about these questions using real-world data from one of largest online labor market places. We found online service-pitchers’ proportion of certainty-words to be positively associated with their getting selected by service-seeking buyers up to a threshold after which words of certainty harmed sellers’ contract-acquisitions; and this inverted U-shaped relationship, which matches the “too much of a good thing (TMGT) effect,” was stronger when sellers’ pitches had fewer (rather than more) rapport-building words. This experiment is used to verify our findings. We additionally examine effects of geographical (dis)similarity on above relationships.
External Link(s)

Registration Citation

Citation
Gao, Qiang. 2021. "Do Online Service-pitchers' Words of Certainty and Rapport-building Enhance Their Hiring?." AEA RCT Registry. October 08. https://doi.org/10.1257/rct.7847-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
In online labor marketplaces (e.g., Freelancer.com and Upwork.com), after an employee (i.e., buyer) post a job, workers (i.e., service providers or pitchers) who are interested in the job will bid by specifying the amount and time they need to complete the job. These workers normally send sale pitches (texts) to this employee.

The purpose of this research is to explore how certainty words and rapport-building words in worker's job-seeking pitches affect employer's hiring decisions.

I test the following 2 hypotheses:
Hypothesis 1: The relationship between online service-pitchers' words of certainty and their attractiveness to buyers is in the form of an inverted U-shaped curve, such that online service-pitchers' likelihood of being selected over competing sellers by buyers is greatest when their pitch-certainty is moderate rather than when it is low or high—hereafter referred to as the "TMGT-effect of pitch-certainty."
Hypothesis 2: The TMGT-effect of pitch-certainty is stronger for online service-pitchers whose pitches contain a lower (rather than higher)proportion of rapport-building words.

I have access to secondary dataset from one of largest online labor platform in the US. The initial results support our hypotheses. I will use an experiment to verify my findings.

This pre-registered experiment has a 3 (certainty words levels: High, medium, and low) by 2 (rapport-building words: Absent VS. present) by 2 (geographical (dis)similarity: Same VS Different) between-subject design.

a. Definitions

• Certainty words are words that show assertiveness of language used by authors, words such as always, absolutely, and assure (Fast & Funder, 2008). Certainty words variable are qualified into three levels based on the percentage of certainty words in the pitches. Since this experiment is used to verify findings obtained using archival data, we use percentile values of certainty words in our archival data as base to define certainty levels. Specifically, we define pitches with no less than 6 percent of certainty words (value of 75 percentile of certainty words percentage in our archival data) as high certainty, pitches with certainty words percentage values between no less than 1 percent (value of 26 percentile of certainty words percentage in our archival data) and 6 percent (value of 74 percentile of certainty words percentage in our archival data) as medium certainty, and pitches with at most 1 percent of certainty words (value of 25 percentile of certainty words percentage in our archival data) as low certainty.

• Rapport-building words are words that build bond between communicators (Gremler & Gwinner, 2008). Rapport-building words include four dimensions: 1. Greeting words (e.g., “hello” and “how are you”). 2. Mentioning communication recipient's names. 3. Politeness words (e.g., “thanks” and “regards”). And 4. Worker’s positive emotion about himself (e.g., “excellent” and “successful”). We define high and low rapport building using the similar approach to quantify certainty levels. Specifically, we define pitches with at least 6 percent of rapport-building word (value of 75 percentile of rapport-building words percentage in archival data) as high rapport-building and at most 2 percent of rapport-building words (value of 25 percentile of rapport-building words percent in archival data) as low rapport-building

• Geographical (dis) similarity measure whether employers and job-seeking workers are from the same geographical regions (Lin, Liu, & Viswanathan, 2018).

Thus, the treatments are the different levels of certainty words, different levels of rapport-building words, and whether employer and job-seeking workers come from same regions.

References
Fast, L. A., & Funder, D. C. (2008). Personality as manifest in word use: correlations with self-report, acquaintance report, and behavior. Journal of Personality and Social Psychology, 94(2), 334.
Gremler, D. D., Gwinner, K. P. (2008). Rapport-building behaviors used by retail employees. Journal of Retailing, 84(3), 308-324
Lin, M., Liu, Y., & Viswanathan, S. (2018). Effectiveness of reputation in contracting for customized production: Evidence from online labor markets. Management Science, 64(1), 345-359.
Intervention Start Date
2021-10-18
Intervention End Date
2021-10-25

Primary Outcomes

Primary Outcomes (end points)
Hiring decision measured using Likert Scale 1-7. The hiring tendency increases from 1 to 7. 1 is least likely to hire and 7 means most likely to hire.
Primary Outcomes (explanation)
Hiring decision measured using Likert Scale 1-7. The hiring tendency increases from 1 to 7. 1 is least likely to hire and 7 means most likely to hire.

Secondary Outcomes

Secondary Outcomes (end points)
No secondary outcomes
Secondary Outcomes (explanation)
No secondary outcomes

Experimental Design

Experimental Design
This pre-registered study will have a 3 (certainty words levels: High VS Medium VS Low) by 2 (rapport-building words levels: High VS Low) by 2 (geographical (dis)similarity: Same VS Different) subjects design.

The detailed explanations about certainty words, rapport-building words, and geographical (dis)similarity are provided in Intervention section.

The experiment will be conducted through Amazon Mechanical Turk platform.

Experimental Design Details
This pre-registered study will have a 3 (certainty words levels: High VS Medium VS Low) by 2 (rapport-building words levels: High VS Low) by 2 (geographical (dis)similarity: Same VS Different) subjects design.

The detailed explanations about certainty words, rapport-building words, and geographical (dis)similarity are provided in Intervention section.

a. Worker Selections
The experiment will be conducted on the Amazon Mechanical Turk platform (https://requester.mturk.com/). The participants are hired workers from this platform.
Only workers who satisfy the following conditions are able to participate:
• Having approved rate over 98% (making sure workers are qualified)
• Having approved coding over 1000 (making sure workers are qualified)
• Subjects must come from the US (controlling buyer location)
• Subjects must be older than 25 years old (younger people may not be good at making hiring decisions)
• We hire 800 different workers from AMTurk platform.

b. Experiment Procedure:
1) I will post a project link to Qualtrics site at Amazon Mechanical Turk platform.
2) A participant from Amazon Mechanical Turk who satisfied our pre-defined screening criteria can click the provided link
3) After coming to Qualtrics site, participants who want to participate must agree with information on consent form before proceeding to next step.
4) After giving consent, the participant is given a brief introduction to the online labor market and then asked to follow a scenario in which the participant is assumed to be the employer who posts a job.
5) The participant is randomly given one version of sale pitches from one worker who is interested in the posted job.
6) After completing the scenario, the participant first answered two questions:
• One question asks the participant to evaluate the relative competence they perceive from the given pitch: “This pitch shows job applicant is competent” (1 = Completely disagree 1, 7 = Completely agree).
• Another question asks the participant to give hiring likelihood: “What is your HIRING-likelihood in response to the pitch you just saw above? Indicate this by using the scales below” (1 = completely unlikely, 7 = completely likely).
7) Finally, the participant is given three more questions for manipulation check:
a. The first question asks the participant to evaluate how much the pitch expresses certainty about the worker’s ability to meet job-related needs: “Pitch expresses much certainty about the worker’s ability to meet your job-related needs” (1 = completely disagree, 7 = completely agree).
b. The second question asks the participant to evaluate how much the pitch expresses rapport-building: “Pitch contains substance that seems oriented toward building rapport, or good and warm feelings between you and the worker” (1 = completely disagree, 7 = completely agree).
c. The last question asks the participant to evaluate how much the pitch shows geographical similarity: “Pitch contains substance that make you feel homophily by geographic location between you and the worker” (1 = completely disagree, 7 = completely agree).


Each participant can only code one pitch.
Randomization Method
After a worker from Amazon Mechanical Turk platform wants to participate. This worker can click the provided link and go to Qualtrics to do the survey.
Each participant will be randomly assigned one pitch. Each participant can only read one pitch and answer questions.
Randomization Unit
There are 12 different versions of a sale pitch. Each version is a randomization unit.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
800 participants.
Sample size: planned number of observations
800 participants.
Sample size (or number of clusters) by treatment arms
800 participants.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
the unit is answers to survey questions by each participant.
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials