Can a skill-based peer-referral system to recruit enumerators improve data quality in the Philippines?

The intervention tested in this study is skill-based peer referrals. Trusted enumerators who have previously worked with IPA are requested to refer candidates who they consider to have strong active listening skills or are conscientious. Then we conduct a second round of 'friends of friends' referrals, where previously referred candidates similarly refer other candidates in their network who they consider to have strong active listening skills or are conscientious.

2023-09-11

2023-09-29

Skill profile among recruited enumerators; survey and backcheck responses from households interviewed.

The intervention will be implemented in three phases:


1. Phase 1 – Business as Usual Comparison group: In this phase, recruitment will be conducted as per usual, with an open application, without any skill-based recruitment. Advertising of the job will occur through social media, email announcements, etc. The job application will include language to encourage all eligible applicants to apply.


2. a. Phase 2 – Conscientiousness treatment group: Nine trusted enumerators (identified by the IPA Philippines team) will be invited to a skill-based recruitment meeting. The meaning of the specific-skill, conscientiousness, will be discussed and the trusted enumerators will be invited to recruit six enumerators from their network who have not yet applied for the job to apply. Trusted enumerators will be compensated for every individual they have a referral conversation with.

2. b. Phase 2 – Active listening treatment group: Nine trusted enumerators (identified by the IPA Philippines team) will be invited to a skill- based recruitment meeting. The specific-skill, active listening, will be discussed and the trusted enumerators will be invited to recruit six enumerators from their network who have not yet applied for the job to apply. Trusted enumerators will be compensated for every individual they have a referral conversation with.

The recruitment team will be blind to the referral status of a candidate (that is, if they were referred or not) to ensure that the outcome of their application remains unbiased. Applications by ‘business as usual’ applicants who apply without referrals will continue to be accepted and reviewed during Phase 2.


3. Phase 3 — 'Friends of Friends’ Recruitment: In Phase 3, applicants recruited from trusted enumerators during Phase 2 will be invited to a skill-specific (conscientiousness or active listening) ‘friends of friends’ recruitment meeting. One meeting for each specific skill group will be organized. During the ‘friends of friends’ recruitment meeting, the definition of the specific skill will be described to the group and they will be invited to recruit two additional enumerators from their network to apply. Those making ‘friends of friends’ referrals will be compensated for every individual they have a referral conversation with.

The recruitment team will be blind to the referral status of a candidate (that is, if they were referred or not) to ensure that the outcome of their application remains unbiased. Applications by ‘business as usual’ applicants who apply without referrals will continue to be accepted and reviewed during Phase 3.

In the first phase of referrals, individual enumerators will be randomly assigned through Stata to refer individuals who are either active listeners or conscientious. In the second phase of referrals, referred enumerators will in turn refer on the basis of the skill on which they themselves were referred on.

Individual enumerators.

No

We expect the sample size to be dynamic, because the size of the comparison group will be determined by the number of candidates who apply without referrals, and can not be predicted ahead of time with certainty. Each of the two treatment groups will have at most 162 referred applicants, for a maximum total of 324 referred applicants in both treatment arms.

The sample size for this study is dynamic, determined by the number of candidates that choose to apply for the job in each treatment arm. To manage this uncertainty, we introduce a stopping rule to determine when each phase ends and the next begins.
Stopping rule for Phase 1 (Business as usual): Phase 1 ends once the job application has been live for 7 days, or once 80 candidates have submitted their application, whichever is first.
Stopping rule for Phase 2 (Referrals by trusted enumerators): Phase 2 ends when it has been 7 days since the start of this phase or once 30 candidates have submitted their application, whichever is first. The end of this phase will be evaluated separately for each treatment arm.
Stopping rule for Phase 3 (Friends of friends referrals): This phase ends when sufficient qualified candidates have been recruited for the host project. We expect this phase to last approximately 7 days based on the host project's planned timeline for recruitment.

324+ applicants

Comparison group: The size of the control group will be determined by the number of candidates who apply without referrals in the first week after posting the job application.
Treatment groups 1 and 2: In the first phase of referrals, we expect 9 enumerators to make at most 6 referrals in each of the two treatment arms (each associated with a specific skill on the basis of which referrals are made). In the second phase of referrals, we expect at most 54 enumerators to make at most 2 referrals in each of the two treatment arms. Accordingly, both the treatment arms will have at most 162 enumerators each.