Can a skill-based peer-referral system to recruit enumerators improve data quality in the Philippines?

Last registered on September 15, 2023

Pre-Trial

Trial Information

General Information

Title
Can a skill-based peer-referral system to recruit enumerators improve data quality in the Philippines?
RCT ID
AEARCTR-0011999
Initial registration date
September 07, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
September 15, 2023, 8:46 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
Northwestern University

Other Primary Investigator(s)

PI Affiliation
PI Affiliation

Additional Trial Information

Status
In development
Start date
2023-09-11
End date
2023-10-13
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Credible research relies significantly on the quality of data enumerators collect from households. However, evidence suggests that this data can vary based on enumerators’ demographic characteristics, and their skills and behaviors when conducting interviews. Poor data quality can lead to bias and limited validity across contexts. Enumerator skills, especially non-cognitive skills, can be difficult to recognize in a typical recruitment process. However, one’s peers have additional information about their skills, so recruiting through referrals holds the potential to diversify the skills profile of the applicant pool. This study aims to measure the impact of recruiting enumerators through skills-based referrals on data quality. Specifically, it asks if referrals change the skill profile in an enumerator applicant pool. It also asks what the return of two specific skills, active listening and conscientiousness, on data quality is. In the first phase, trusted enumerators identified by the IPA Philippines team will be randomized into referring individuals in their network on the basis of one of the two skills. In the second phase, referred enumerators will in turn be asked to refer individuals on the basis of the skill they were referred on. This will be a supplementary study to the Philippines Socioeconomic Panel Survey (PSPS) which is being implemented by IPA Philippines.
External Link(s)

Registration Citation

Citation
Dillon, Andrew, Dean Karlan and Christopher Udry. 2023. "Can a skill-based peer-referral system to recruit enumerators improve data quality in the Philippines?." AEA RCT Registry. September 15. https://doi.org/10.1257/rct.11999-1.0
Experimental Details

Interventions

Intervention(s)
The intervention tested in this study is skill-based peer referrals. Trusted enumerators who have previously worked with IPA are requested to refer candidates who they consider to have strong active listening skills or are conscientious. Then we conduct a second round of 'friends of friends' referrals, where previously referred candidates similarly refer other candidates in their network who they consider to have strong active listening skills or are conscientious.
Intervention Start Date
2023-09-11
Intervention End Date
2023-09-29

Primary Outcomes

Primary Outcomes (end points)
Skill profile among recruited enumerators; survey and backcheck responses from households interviewed.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The intervention will be implemented in three phases:


1. Phase 1 – Business as Usual Comparison group: In this phase, recruitment will be conducted as per usual, with an open application, without any skill-based recruitment. Advertising of the job will occur through social media, email announcements, etc. The job application will include language to encourage all eligible applicants to apply.


2. a. Phase 2 – Conscientiousness treatment group: Nine trusted enumerators (identified by the IPA Philippines team) will be invited to a skill-based recruitment meeting. The meaning of the specific-skill, conscientiousness, will be discussed and the trusted enumerators will be invited to recruit six enumerators from their network who have not yet applied for the job to apply. Trusted enumerators will be compensated for every individual they have a referral conversation with.

2. b. Phase 2 – Active listening treatment group: Nine trusted enumerators (identified by the IPA Philippines team) will be invited to a skill- based recruitment meeting. The specific-skill, active listening, will be discussed and the trusted enumerators will be invited to recruit six enumerators from their network who have not yet applied for the job to apply. Trusted enumerators will be compensated for every individual they have a referral conversation with.

The recruitment team will be blind to the referral status of a candidate (that is, if they were referred or not) to ensure that the outcome of their application remains unbiased. Applications by ‘business as usual’ applicants who apply without referrals will continue to be accepted and reviewed during Phase 2.


3. Phase 3 — 'Friends of Friends’ Recruitment: In Phase 3, applicants recruited from trusted enumerators during Phase 2 will be invited to a skill-specific (conscientiousness or active listening) ‘friends of friends’ recruitment meeting. One meeting for each specific skill group will be organized. During the ‘friends of friends’ recruitment meeting, the definition of the specific skill will be described to the group and they will be invited to recruit two additional enumerators from their network to apply. Those making ‘friends of friends’ referrals will be compensated for every individual they have a referral conversation with.

The recruitment team will be blind to the referral status of a candidate (that is, if they were referred or not) to ensure that the outcome of their application remains unbiased. Applications by ‘business as usual’ applicants who apply without referrals will continue to be accepted and reviewed during Phase 3.
Experimental Design Details
Randomization Method
In the first phase of referrals, individual enumerators will be randomly assigned through Stata to refer individuals who are either active listeners or conscientious. In the second phase of referrals, referred enumerators will in turn refer on the basis of the skill on which they themselves were referred on.
Randomization Unit
Individual enumerators.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
We expect the sample size to be dynamic, because the size of the comparison group will be determined by the number of candidates who apply without referrals, and can not be predicted ahead of time with certainty. Each of the two treatment groups will have at most 162 referred applicants, for a maximum total of 324 referred applicants in both treatment arms.

The sample size for this study is dynamic, determined by the number of candidates that choose to apply for the job in each treatment arm. To manage this uncertainty, we introduce a stopping rule to determine when each phase ends and the next begins.
Stopping rule for Phase 1 (Business as usual): Phase 1 ends once the job application has been live for 7 days, or once 80 candidates have submitted their application, whichever is first.
Stopping rule for Phase 2 (Referrals by trusted enumerators): Phase 2 ends when it has been 7 days since the start of this phase or once 30 candidates have submitted their application, whichever is first. The end of this phase will be evaluated separately for each treatment arm.
Stopping rule for Phase 3 (Friends of friends referrals): This phase ends when sufficient qualified candidates have been recruited for the host project. We expect this phase to last approximately 7 days based on the host project's planned timeline for recruitment.
Sample size: planned number of observations
324+ applicants
Sample size (or number of clusters) by treatment arms
Comparison group: The size of the control group will be determined by the number of candidates who apply without referrals in the first week after posting the job application.
Treatment groups 1 and 2: In the first phase of referrals, we expect 9 enumerators to make at most 6 referrals in each of the two treatment arms (each associated with a specific skill on the basis of which referrals are made). In the second phase of referrals, we expect at most 54 enumerators to make at most 2 referrals in each of the two treatment arms. Accordingly, both the treatment arms will have at most 162 enumerators each.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials