Why the Referential Treatment?: Evidence from Field Experiments on Referrals

Last registered on September 13, 2016

Pre-Trial

Trial Information

General Information

Title
Why the Referential Treatment?: Evidence from Field Experiments on Referrals
RCT ID
AEARCTR-0001492
Initial registration date
September 13, 2016

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
September 13, 2016, 9:55 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Primary Investigator

Affiliation
Harvard University

Other Primary Investigator(s)

PI Affiliation
Coursera Inc.

Additional Trial Information

Status
Completed
Start date
2013-01-01
End date
2013-06-30
Secondary IDs
Abstract
Referred workers are more likely than non-referred workers to be hired, all else equal. In three field experiments in an online labor market, we examine why. We find that referrals contain positive information about worker performance and persistence that is not contained in workers' observable characteristics. We also find that referrals performed particularly well when working directly with their referrers. However, we do not find evidence that referrals exert more effort because they believe their performance will affect their relationship with their referrer or their referrer's position at the firm.
External Link(s)

Registration Citation

Citation
Pallais, Amanda and Emily Glassberg Sands. 2016. "Why the Referential Treatment?: Evidence from Field Experiments on Referrals." AEA RCT Registry. September 13. https://doi.org/10.1257/rct.1492-1.0
Former Citation
Pallais, Amanda and Emily Glassberg Sands. 2016. "Why the Referential Treatment?: Evidence from Field Experiments on Referrals." AEA RCT Registry. September 13. https://www.socialscienceregistry.org/trials/1492/history/10610
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2013-01-01
Intervention End Date
2013-06-30

Primary Outcomes

Primary Outcomes (end points)
Dependent variables for each experiment are listed below:


Selection Experiment (measuring job performance and turnover)

- Accepted job offer

- Submitted work

- Submitted work on-time

- Accuracy

- Re-apply to continue in job;


Peer Influence Experiment (measuring job performance and turnover)

- Submitted work

- Submitted work on-time

- Accuracy

- Re-apply to continue in job;

Team Experiment (measure individual diligence and team performance)

- Logged in to task site

- Submitted work

- Answered individual question correctly

- Used own criterion in slogan

- Both partners submitted work

- Partners answered team question the same way

- Partners submitted the same slogan

- Partners submitted the same slogan which used both criteria




Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
This project analyzed a set of field experiments in an online labor market to answer two open questions about referrals: first do referrals contain information about worker productivity? Second, do referred workers work harder or more effectively because they are referred? Three experiments were conducted for this project: the peer influence experiment, the team experiment, and the selection experiment.

The peer influence experiment was designed primarily to determine whether referrals work harder as a result of being referred because they think their performance and persistence will affect either their referrer's position at the firm or their relationship with their referrer. In the peer influence experiment, referred and non-referred workers tested an airline flight website by answering questions about the site every other day for 12 days. Referrals in this experiment were randomized into two treatments. The non-monitoring treatment was designed to minimize peer influence. Referrals in this treatment were told their referrers would never know their performance and (after referring) referrers were told they would not be judged on the performance of their referrals. The monitoring treatment was designed to maximize peer influence. Each referrer in this treatment received an update on their referral's performance after each day of work. It was implied to each referrer that their referral's performance and willingness to continue working would affect whether the referrer was promoted.

The team experiment was designed to determine whether directly working with their referrers leads referrals to perform better. In the team experiment, the task was to work with an assigned partner to create a single, shared slogan for a public service announcement (PSA). Each of the two partners was given a different information sheet containing a distinct criterion for the slogan. Partners were asked to discuss the task and then to each submit the same slogan, which should have satisfied both criteria. Workers completed three such PSA tasks, each with a different partner. Each referral completed one task with their referrer and one task with another randomly-chosen referrer.

The selection experiment was designed explicitly to determine whether referrals contain information about worker quality. Four months after the peer influence experiment, the performance and persistence of referred and non-referred workers was measured in a job to which the referred workers had not been referred. Job offers were made to all referred and non-referred workers from the peer influence experiment. Similar to the peer influence experiment, workers who accepted the job offers were given a task that measured individual diligence over time. Workers were asked to visit the Twitter pages of three successful musicians and to answer a ten-question survey about those accounts every day for five consecutive days.
Experimental Design Details
Randomization Method
Randomization in office via computer
Randomization Unit
Selection experiment: individual workers
Peer influence experiment: individual workers (referrers were randomized; referrals were put into same treatment as referrer)
Team experiment: referrer-referral pairs -- In the team experiment, each referrer participated in all three team types. Referrer-referral pairs were randomly placed into blocking groups and only partnered with other people in the same blocking group. Within a blocking group, the ordering of the type of team workers participated in was random. Within team type, when relevant, workers' assigned partners were also randomly assigned.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
See below on number of observations
Sample size: planned number of observations
Selection Experiment: 265 workers; Peer Influence Experiment: 608 workers; Team Experiment: 658 workers
Sample size (or number of clusters) by treatment arms
Selection Experiment: (no treatment arms)
Peer Influence Experiment:
- Monitoring treatment: 213 referred workers
- Non-monitoring treatment: 215 referred workers
Team Experiment: (no treatment arms)
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Harvard University
IRB Approval Date
2012-11-02
IRB Approval Number
F22403-102

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
June 30, 2013, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
June 30, 2013, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
Same as planned
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
Same as planned
Final Sample Size (or Number of Clusters) by Treatment Arms
Same as planned
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
No
Reports, Papers & Other Materials

Relevant Paper(s)

Abstract
Referred workers are more likely than non-referred workers to be hired, all else equal. In three field experiments in an online labor market, we examine why. We find that referrals contain positive information about worker performance and persistence that is not contained in workers' observable characteristics. We also find that referrals performed particularly well when working directly with their referrers. However, we do not find evidence that referrals exert more effort because they believe their performance will affect their relationship with their referrer or their referrer's position at the firm.
Citation
Pallais, Amanda and Emily Sands. "Why the Referential Treatment: Evidence from Field Experiments on Referrals." Journal of Political Economy, forthcoming.

Reports & Other Materials