|
Field
Abstract
|
Before
Substantial literature on survey response rates focuses on framing and the appeal to altruism as a motivation for participating, with methods like pre-survey post-cards and letters to incentivize cooperation, with evidence coming primarily from the U.S. and Europe. More recently, there has been interest in mobile phone surveys in low and middle income countries, where the efficacy of methods for improving response rates is not as well known. This study randomizes the use of pre-survey text messages, whether to send them and which type of appeal to make. The study also randomizes the messaging used in the consent script, appealing alternatively to “researcher” or “government” as the motivating authority. The experiment is conducted in random-digit dial (RDD) surveys in up to 12 countries in Latin America, Africa, and Asia.
|
After
Substantial literature on survey response rates focuses on framing and the appeal to altruism as a motivation for participating, with methods like pre-survey post-cards and letters to incentivize cooperation, with evidence coming primarily from the U.S. and Europe. More recently, there has been interest in mobile phone surveys in low and middle income countries, where the efficacy of methods for improving response rates is not as well known. This study randomizes the use of pre-survey text messages, whether to send them and which type of appeal to make. The study also randomizes the messaging used in the consent script, appealing alternatively to “researcher” or “government” as the motivating authority, in the first round of experiments, and appeals to efficacy of participation and to self-interest with reminders about monetary compensation. The experiment is conducted in 11 random-digit dial (RDD) surveys in 10 countries, with followup surveys in 5 of those surveys.
|
|
Field
Trial End Date
|
Before
September 30, 2020
|
After
June 30, 2021
|
|
Field
JEL Code(s)
|
Before
|
After
C83
|
|
Field
Last Published
|
Before
July 13, 2020 03:51 PM
|
After
December 10, 2020 11:50 AM
|
|
Field
Intervention (Public)
|
Before
The experiment varied two factors
Factor 1 is SMS text message sent to respondent prior to CATI interview call, with 3 possible levels:
S0 = No SMS
SG= SMS, appeal to "government"
SR= SMS, appeal to "researcher"
Factor 2 is appeal in the consent script, with 3 levels:
G = consent appeals to "government"
R = consent appeals to "researcher"
P = consent appeals to "policymaker"
In Colombia and Mexico, it was a 2x2 (no cases assigned to S0 or P)
In other countries it was a 3x1 (S, G, or P)
"Mixed message" cells (SG-R, SG-P, SR-G, SR-P, etc.) are not populated.
In Spanish-speaking countries, "policymaker" appeals are omitted because terminology is hard to distinguish from government.
|
After
The experiment varied two factors
Factor 1 is SMS text message sent to respondent prior to CATI interview call, with 3 possible levels:
S0 = No SMS
SG= SMS, appeal to "government"
SR= SMS, appeal to "researcher"
Factor 2 is appeal in the consent script, with 3 levels:
G = consent appeals to "government"
R = consent appeals to "researcher"
P = consent appeals to "policymaker"
In Colombia and Mexico, it was a 2x2 (no cases assigned to S0 or P)
In the other wave 1 countries it was a 3x1 (S, G, or P)
"Mixed message" cells (SG-R, SG-P, SR-G, SR-P, etc.) are not populated.
In Spanish-speaking countries, "policymaker" appeals are omitted because terminology is hard to distinguish from government.
In Wave 2 surveys, the treatment factor structure is altered and different messaging contrasts are planned. The pre-survey SMS message has the following appeal type for each treatment arm:
A1 Placebo/short message -- no specific appeal
A2 General Learning (Food access), says that the first wave of the survey was informative about food access
A3 General Learning (Household finances), says the first wave of the survey related to household finances
A4 Specific Learning (Food access), same as above but shared a statistic about food access
A5 Specific Learning (Household finances), same as above, but shared a statistic about household finances
B1 Self Interest, reminded respondents about the monetary incentive
B2 General Learning (Food access) + Self interest, same as A2, but included message about monetary incentive
B3 General Learning (Household finances) + Self interest, same as A3 but included message about monetary incentive
B4 Specific Learning (Food access) + Self interest = A4 + monetary incentive message
B5 Specific Learning (Household finances) + Self interest = A5 + monetary incentive message
|
|
Field
Intervention End Date
|
Before
September 30, 2020
|
After
December 31, 2020
|
|
Field
Sample size (or number of clusters) by treatment arms
|
Before
Varies by country/survey, but approximately 12000 cases in each treatment arm
|
After
Varies by country/survey, but approximately 12000 cases in each treatment arm for the Wave 1 experiment.
The Wave 2 experiment will be conducted with followups from the RDD survey. This sample will be substantially smaller, approximately 6,000 total spread over 10 treatment arms, with unequal cell sizes ranging from 8.3% (~500 cases) to 16.7% (~1,000 cases).
|
|
Field
Power calculation: Minimum Detectable Effect Size for Main Outcomes
|
Before
Binary outcome (response rate): approximately 2.7 percentage points with Dunn-Bonferroni adjustment, assuming 4 tests
|
After
Binary outcome (response rate): approximately 2.7 percentage points with Dunn-Bonferroni adjustment, assuming 4 tests for Wave 1 experiment.
For the Wave 2 experiment, we estimate the minimum detectable effect size for the contrast with the smallest sample will be 0.104, assuming a binary outcome with control group mean of 0.50 and using a Dunn-Bonferroni correction for multiple hypothesis tests.
|
|
Field
Additional Keyword(s)
|
Before
Methods
|
After
Survey Methods
|