Testing Experimental Robustness across Survey Invitation Strategies

Last registered on March 25, 2021
View Trial History

Pre-Trial

Trial Information

General Information

Title
Testing Experimental Robustness across Survey Invitation Strategies
RCT ID
AEARCTR-0007408
Initial registration date
March 25, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
March 25, 2021, 6:27 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
Denmark
Region

Primary Investigator

Name
Paw Hansen
Affiliation
University of Copenhagen, Department of Political Science
Contact Primary Investigator

Other Primary Investigator(s)

Additional Trial Information

Status
In development
Start date
2021-04-06
End date
2021-04-27
Keywords
Other
Additional Keywords
JEL code(s)
Secondary IDs
Abstract
We randomly assign three distinct invitations to participate in a survey. Each invitation is hypothesized to trigger a particular form of motivation with the potential respondents. The overall aim is to test (a) whether these subtle differences in the invitation to participate (a) affect response rates, and (b) shape the sample due to self-selection and. Moreover, we embed a separate survey experiment in the survey and test whether the three different invitations lead to treatment heterogeneity.
External Link(s)

Registration Citation

Citation
Hansen, Paw. 2021. "Testing Experimental Robustness across Survey Invitation Strategies." AEA RCT Registry. March 25. https://doi.org/10.1257/rct.7408-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
We randomly assign participants to receive one of three distinct invitations to participate in a survey. Each invitation appeals to a specific form of motivation. Thus, the intervention consists of subtle text cues in the invitations.
Intervention Start Date
2021-04-06
Intervention End Date
2021-04-27

Primary Outcomes

Primary Outcomes (end points)
We have three outcomes of interest. (1) Response rates across the three groups, (2) respondents' levels of three types of motivation (Public Service Motivation, user orientation, and self-efficacy), and (3) heterogeneity in how respondents reply to a survey experiment embedded in the issued survey
Primary Outcomes (explanation)
Outcome number 2 is measured using three widely-used scales, each consisting three survey items.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We randomly assign participants to receive one of three distinct invitations to participate in a survey. Each invitation appeals to a specific form of motivation. Thus, the intervention consists of subtle text cues in the invitations. Within the survey, we embed a survey experiment. Our goal is to examine differences between the three groups in terms of (a) response rates (b) levels of motivation, and (c) heterogeneity in how respondents react to the embedded survey experiment. The embedded survey experiment is preregistered separately.
Experimental Design Details
Randomization Method
Randomization is done with computer.
Randomization Unit
We conduct randomization at the individual level.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
Approx. 9,000 public employees.
Sample size: planned number of observations
Approx. 9,000 public employees.
Sample size (or number of clusters) by treatment arms
Approx. 3,000 public employees per treatment arm.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials