Experimental Design Details
This pre-registered study has a 3 (high certainty word ratio (rapport-building words: Absent VS. present), medium certainty word ratio (rapport-building words: Absent VS. present), and low certainty word ratio (rapport-building words: Absent VS. Present)) by 2 (project complexity: Absent VS. Present) between-subject design.
The detailed explanations about certainty words, rapport-building words, and job complexity are provided in Intervention section.
a. Worker Selections
The experiment will be conducted on the Amazon Mechanical Turk platform (https://requester.mturk.com/). The participants are hired workers from this platform.
Only workers who satisfy the following conditions are able to participate:
• Having approved rate over 98% (making sure workers are qualified)
• Having approved coding over 100 (making sure workers are qualified)
• Subjects must come from the US (controlling buyer location)
• Being over 18 years old.
• We hire 1000 different workers from AMTurk platform.
b. Experiment Procedure:
The data will be collected from a survey hosted on the Baruch College Qualtrics site.
1) I will post a project link to the Baruch College Qualtrics site on the Amazon Mechanical Turk platform.
2) A participant from Amazon Mechanical Turk who satisfied our pre-defined screening criteria can click the provided link
3) After coming to the Baruch Qualtrics site, participants who want to participate must agree with the information on the consent form before proceeding to the next step.
4) After giving consent, the participant is asked to follow a scenario in which the participant is assumed to be the employer who posts a job.
“You have a family friend who runs a popular hotdog shop. He is not very good with technology and he asks you for help. You suggested that he could try and hire freelancers online. You posted the following request on his behalf:
I need someone to build a website with an online ordering system for my small hotdog shop. The website should allow customers to view some basic info about my shop (history, location, hours etc. that I’d provide to you), the menu of items that I currently offer, and to also place an order online. The website should be mobile-friendly as well. Thanks.”
5) The participant is randomly assigned to one set of pitches explained in Intervention section. The participant is asked to read two different sales pitches written by two different service providers who are interested in the job.
6) After reading and comparing the given two pitches, the participant needs to answer several questions:
• One question asks the likelihood of the participant to hire the writer who gives each pitch: “How likely are you to hire the writer of each pitch?” (1 = Not at all, 5 = Extremely much).
• Eleven questions question ask the participant to evaluate each service provider from their pitches in terms of certainty, rapport-building, warmth, competence, credibility, trustworthiness, politeness, passion, confidence, friendliness, and similarity with the participant. For example, the question to evaluate the competence of the service provider: “How competent do you think of the writer of each pitch?” (1 = Not at all, 5 = Extremely much). These questions are used to check the manipulation of pitches and examine some controls.
• One question asks how the participant consider the complexity of web design job: “In general, how complex are web design jobs?” (1 = Not at all, 5 = Extremely much). This question is used to check the manipulation of project complexity.
7) Finally, a set of questions is given to obtain the participant’s demographics and his Positive and Negative Affect (PANAS):
a. The first question asks the participant’s experience in ordering food online, designing websites, and hiring someone online.
b. The second ten questions ask the participant to indicate how strong he feels right now in terms of ten emotional words (determined, attentive, alert, inspired, active, afraid, nervous, upset, ashamed, and hostile) (Thompson, 2007).
c. The last four questions ask the participant’s gender, age, ethnicity, and education level.
Each participant can only participate once.