Pitch Attributes on Employer's Hiring Decisions

Last registered on November 18, 2021

Pre-Trial

Trial Information

General Information

Title
Pitch Attributes on Employer's Hiring Decisions
RCT ID
AEARCTR-0007847
Initial registration date
October 08, 2021
Last updated
November 18, 2021, 8:55 AM EST

Locations

Region

Primary Investigator

Affiliation
Baruch College, City University of New York

Other Primary Investigator(s)

Additional Trial Information

Status
In development
Start date
2021-11-18
End date
2021-11-25
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Online labor market (e.g., Freelancer.com, Upworker.com, Fivre.com) provides opportunities for service-seeking buyers (i.e., employers) to find service-providers (i.e., workers) to work on jobs and for workers to find jobs. In this experiment, we examine how communication qualities (the proportion of certainty- and rapport building-words) in competing workers’ pitches (i.e., texts) influenced employer’ hiring decisions in online labor marketplaces. We already have findings about these questions using real-world data from one of largest online labor market places. We found online service-pitchers’ proportion of certainty-words to be positively associated with their getting selected by service-seeking buyers up to a threshold after which words of certainty harmed sellers’ contract-acquisitions; and this inverted U-shaped relationship, which matches the “too much of a good thing (TMGT) effect,” was stronger when sellers’ pitches had fewer (rather than more) rapport-building words. This experiment is used to verify our findings. We additionally examine effects of geographical (dis)similarity on above relationships.
External Link(s)

Registration Citation

Citation
Gao, Qiang. 2021. "Pitch Attributes on Employer's Hiring Decisions." AEA RCT Registry. November 18. https://doi.org/10.1257/rct.7847-3.0
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details

Interventions

Intervention(s)
In online labor marketplaces (e.g., Freelancer.com and Upwork.com), after an employee (i.e., buyer) post a job, workers (i.e., service providers or pitchers) who are interested in the job will bid by specifying the amount and time they need to complete the job. These workers normally send sale pitches (texts) to this employee.

The purpose of this research is to explore how certainty words and rapport-building words in worker's job-seeking pitches affect employer's hiring decisions.

I test the following 2 hypotheses:
Hypothesis 1: The relationship between online service-pitchers' words of certainty and their attractiveness to buyers is in the form of an inverted U-shaped curve, such that online service-pitchers' likelihood of being selected over competing sellers by buyers is greatest when their pitch-certainty is moderate rather than when it is low or high—hereafter referred to as the "TMGT-effect of pitch-certainty."
Hypothesis 2: The TMGT-effect of pitch-certainty is stronger for online service-pitchers whose pitches contain a lower (rather than higher)proportion of rapport-building words.

I have access to secondary dataset from one of largest online labor platform in the US. The initial results support our hypotheses. I will use an experiment to verify my findings and a boundary condition (i.e., job complexity).

This pre-registered study has a 3 (high certainty word ratio (rapport-building words: Absent VS. present), medium certainty word ratio (rapport-building words: Absent VS. present), and low certainty word ratio (rapport-building words: Absent VS. Present)) by 2 (project complexity: Absent VS. Present) between-subject design.

a. Definitions

• Certainty words are words that show assertiveness of language used by authors, words such as always, absolutely, and assure (Fast & Funder, 2008).

• Rapport-building words are words that build bonds between communicators (Gremler & Gwinner, 2008). We measure rapport-building in two dimensions (Tickle-Degnen & Rosenthal, 1990):
1) Mutual attentiveness. This dimension shows the expression of mutual attention and involvement with the other, including greeting words (e.g., “hello” and “how are you”) and mentioning communication recipients' names.
2) Positivity, feelings of mutual friendliness and warmth. This dimension includes politeness words (e.g., “thanks” and “regards”) and small talk components (e.g., emoji and exclamation marks).

• Project complexity
We use the presence or absence of the following note to create project complexity:
Website design-jobs often experience delays due to complexities associated with the programmer, client’s clarity about needs, and communications between them.

Thus, the treatments are the different levels of certainty words, different levels of rapport-building words, and different level of job complexity.

References
Fast, L. A., & Funder, D. C. (2008). Personality as manifest in word use: correlations with self-report, acquaintance report, and behavior. Journal of Personality and Social Psychology, 94(2), 334.
Gremler, D. D., Gwinner, K. P. (2008). Rapport-building behaviors used by retail employees. Journal of Retailing, 84(3), 308-324
Tickle-Degnen, L., & Rosenthal, R. (1990). The nature of rapport and its nonverbal correlates. Psychological inquiry, 1(4), 285-293
Intervention Start Date
2021-11-18
Intervention End Date
2021-11-25

Primary Outcomes

Primary Outcomes (end points)
The likelihood of hiring is measured using Likert Scale 1-5. The hiring tendency increases from 1 to 5. 1 is Not at all and 5 means Extremely much.
Primary Outcomes (explanation)
The likelihood of hiring is measured using Likert Scale 1-5. The hiring tendency increases from 1 to 5. 1 is Not at all and 5 means Extremely much.

Secondary Outcomes

Secondary Outcomes (end points)
No secondary outcomes
Secondary Outcomes (explanation)
No secondary outcomes

Experimental Design

Experimental Design
This pre-registered study has a 3 (high certainty word ratio (rapport-building words: Absent VS. present), medium certainty word ratio (rapport-building words: Absent VS. present), and low certainty word ratio (rapport-building words: Absent VS. Present)) by 2 (project complexity: Absent VS. Present) between-subject design.

The detailed explanations about certainty words, rapport-building words, and job complexity are provided in Intervention section.

The experiment will be conducted through Amazon Mechanical Turk platform.
Experimental Design Details
This pre-registered study has a 3 (high certainty word ratio (rapport-building words: Absent VS. present), medium certainty word ratio (rapport-building words: Absent VS. present), and low certainty word ratio (rapport-building words: Absent VS. Present)) by 2 (project complexity: Absent VS. Present) between-subject design.

The detailed explanations about certainty words, rapport-building words, and job complexity are provided in Intervention section.

a. Worker Selections
The experiment will be conducted on the Amazon Mechanical Turk platform (https://requester.mturk.com/). The participants are hired workers from this platform.
Only workers who satisfy the following conditions are able to participate:
• Having approved rate over 98% (making sure workers are qualified)
• Having approved coding over 100 (making sure workers are qualified)
• Subjects must come from the US (controlling buyer location)
• Being over 18 years old.
• We hire 1000 different workers from AMTurk platform.

b. Experiment Procedure:
The data will be collected from a survey hosted on the Baruch College Qualtrics site.
1) I will post a project link to the Baruch College Qualtrics site on the Amazon Mechanical Turk platform.
2) A participant from Amazon Mechanical Turk who satisfied our pre-defined screening criteria can click the provided link
3) After coming to the Baruch Qualtrics site, participants who want to participate must agree with the information on the consent form before proceeding to the next step.
4) After giving consent, the participant is asked to follow a scenario in which the participant is assumed to be the employer who posts a job.
“You have a family friend who runs a popular hotdog shop. He is not very good with technology and he asks you for help. You suggested that he could try and hire freelancers online. You posted the following request on his behalf:

I need someone to build a website with an online ordering system for my small hotdog shop. The website should allow customers to view some basic info about my shop (history, location, hours etc. that I’d provide to you), the menu of items that I currently offer, and to also place an order online. The website should be mobile-friendly as well. Thanks.”

5) The participant is randomly assigned to one set of pitches explained in Intervention section. The participant is asked to read two different sales pitches written by two different service providers who are interested in the job.
6) After reading and comparing the given two pitches, the participant needs to answer several questions:
• One question asks the likelihood of the participant to hire the writer who gives each pitch: “How likely are you to hire the writer of each pitch?” (1 = Not at all, 5 = Extremely much).
• Eleven questions question ask the participant to evaluate each service provider from their pitches in terms of certainty, rapport-building, warmth, competence, credibility, trustworthiness, politeness, passion, confidence, friendliness, and similarity with the participant. For example, the question to evaluate the competence of the service provider: “How competent do you think of the writer of each pitch?” (1 = Not at all, 5 = Extremely much). These questions are used to check the manipulation of pitches and examine some controls.
• One question asks how the participant consider the complexity of web design job: “In general, how complex are web design jobs?” (1 = Not at all, 5 = Extremely much). This question is used to check the manipulation of project complexity.
7) Finally, a set of questions is given to obtain the participant’s demographics and his Positive and Negative Affect (PANAS):
a. The first question asks the participant’s experience in ordering food online, designing websites, and hiring someone online.
b. The second ten questions ask the participant to indicate how strong he feels right now in terms of ten emotional words (determined, attentive, alert, inspired, active, afraid, nervous, upset, ashamed, and hostile) (Thompson, 2007).
c. The last four questions ask the participant’s gender, age, ethnicity, and education level.

Each participant can only participate once.
Randomization Method
After a worker from Amazon Mechanical Turk platform wants to participate. This worker can click the provided link and go to Qualtrics to do the survey.
Each participant will be randomly assigned one set of pitches (two different pitches from two different service providers) and answer some questions. Each participant can only participate once.
Randomization Unit
There are 6 different sets of sale pitches (each set has two different pitches). Each set is a randomization unit.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
1000 participants.
Sample size: planned number of observations
1000 participants.
Sample size (or number of clusters) by treatment arms
1000 participants.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
the unit is answers to survey questions by each participant.
IRB

Institutional Review Boards (IRBs)

IRB Name
The CUNY Human Research Protection Program (HRPP)
IRB Approval Date
2021-11-09
IRB Approval Number
2021-2082

Post-Trial

Post Trial Information

Study Withdrawal

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials