Priming Public Managers to Participate in Research: Evidence from a Low-Intensity Survey Experiment

Last registered on September 25, 2020

Pre-Trial

Trial Information

General Information

Title
Priming Public Managers to Participate in Research: Evidence from a Low-Intensity Survey Experiment
RCT ID
AEARCTR-0006526
Initial registration date
September 25, 2020

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
September 25, 2020, 1:54 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
University of Copenhagen

Other Primary Investigator(s)

PI Affiliation
VIVE – The Danish Center for Social Science Research
PI Affiliation
VIVE – The Danish Center for Social Science Research

Additional Trial Information

Status
On going
Start date
2020-09-23
End date
2020-10-07
Secondary IDs
Abstract
Much public management research uses data involving voluntary participation from public managers in surveys or interviews. To what extent may low-intensity text appeals promote public managers’ willingness to volunteer as respondents in research studies? Are particular types of text appeal more effective than others are? Using a between-subjects survey experimental design, this study examines the effects of various text appeals on public managers’ willingness to participate in a qualitative research interview. At the end of an electronic survey sent to about 10,000 public managers in Denmark, respondents are invited to participate in a follow-up interview study. Each respondent is randomly assigned to either a control group (basic text) or one of two treatment groups, TG1 or TG2. Respondents in TG1 are exposed to brief text serving “controlled extrinsic” forms of motivation, whereas respondents in TG2 are exposed to brief text appealing to prosocial forms of motivation. We examine and compare the effects of the treatments on the respondents’ survey-reported willingness to participate in the follow-up interview study.
External Link(s)

Registration Citation

Citation
Pedersen, Rasmus Tue, Mogens Jin Pedersen and Niels Bjørn Grund Petersen. 2020. "Priming Public Managers to Participate in Research: Evidence from a Low-Intensity Survey Experiment." AEA RCT Registry. September 25. https://doi.org/10.1257/rct.6526
Experimental Details

Interventions

Intervention(s)
The interventions consist of low-intensity text appeals appearing at the end of an electronic survey. All survey respondents are notified that a set of qualitative interviews are to be conducted following the survey examination. By random assignment, respondents are then exposed to one of the following five text appeals:

(1) “We want to hear if you are willing to participate.” [in Danish: “I den forbindelse vil vi høre, om du vil deltage.”]
(2) “With your participation, you will contribute with your special knowledge and unique insights and experience” [in Danish: “Med din deltagelse vil du bidrage med din særlige viden og unikke oplevelser og erfaringer.”]
(3) “With your participation, you will have the chance to further reflect upon you own leadership style and development” [in Danish: “Med din deltagelse vil du få mulighed for at reflektere nærmere over din egen ledelsesstil og lederudvikling.”]
(4) “With your participation, you will help establish new and important knowledge about management training programmes in Denmark – which ultimately will be of benefit to citizens and society.” [in Danish: “Med din deltagelse vil du bidrage til at skabe ny og vigtig viden om lederuddannelser i Danmark – hvilket i sidste ende vil være til stor gavn for borgere og samfundet.”]
(5) “With your participation, you will help establish new and important knowledge about management training programmes in Denmark – which ultimately will be of benefit to other managers.” [in Danish: “Med din deltagelse vil du bidrage til at skabe ny og vigtig viden om lederuddannelser i Danmark – hvilket i sidste ende vil være til stor gavn for andre ledere.”]
Intervention Start Date
2020-09-23
Intervention End Date
2020-10-07

Primary Outcomes

Primary Outcomes (end points)
Willingness to participate in a follow-up interview study (1) or not (0).
Primary Outcomes (explanation)
As our outcome measure, all respondents are asked to report whether they may be contacted for an interview. Response categories are “yes” and “no.” Respondents who agree to participate are asked to report their contact email address (open textbox). Answering "yes" and reporting a valid email adress will be considered as "willingness to participate in a follow-up interview study."

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
About 10,000 public managers in Denmark receive an invitation to participate in an electronic survey.
At the end of the survey, respondents are invited to participate in a follow-up interview study. In particular, all respondents are exposed to a survey item stating that a set of qualitative interviews are to be conducted following the survey examination. Our experimental design involves manipulation of the exact survey item text.

Each respondent is randomly assigned to either a control group (CG) or one of two treatment groups, TG1 or TG2. Respondents are assigned to CG, TG1, and TG2 with equal probability (i.e., 33/33/33 spilt).

CG respondents receive the following text: “We want to hear if you are willing to participate.”

TG1 respondents receive one of the following two texts (assigned at random) serving “controlled extrinsic” forms of motivation:

(a) “With your participation, you will contribute with your special knowledge and unique insights and experience”
(b) “With your participation, you will have the chance to further reflect upon you own leadership style and development”

TG2 respondents receive one of the following two texts (assigned at random) appealing to prosocial forms of motivation:

(a) “With your participation, you will help establish new and important knowledge about management training programmes in Denmark – which ultimately will be of benefit to citizens and society”
(b) “With your participation, you will help establish new and important knowledge about management training programmes in Denmark – which ultimately will be of benefit to other managers.”

As our outcome measure, all respondents are asked to report whether they may be contacted for an interview. Response categories are “yes” and “no.” Respondents who agree to participate are asked to report their contact email address (open textbox).

We test the following hypotheses:

H1: Text appeals serving “controlled extrinsic” forms of motivation increase willingness to participate in research (relative to a basic text)
H2: Text appeals serving prosocial forms of motivation increase willingness to participate in research (relative to a basic text)
H3: Text appeals serving prosocial forms of motivation increase willingness to participate in research (relative to text appeals serving “controlled extrinsic” forms of motivation)
Experimental Design Details
About 10,000 public managers in Denmark receive an invitation to participate in an electronic survey.
At the end of the survey, respondents are invited to participate in a follow-up interview study. In particular, all respondents are exposed to a survey item stating that a set of qualitative interviews are to be conducted following the survey examination. Our experimental design involves manipulation of the exact survey item text.

Each respondent is randomly assigned to either a control group (CG) or one of two treatment groups, TG1 or TG2. Respondents are assigned to CG, TG1, and TG2 with equal probability (i.e., 33/33/33 spilt).

CG respondents receive the following text: “We want to hear if you are willing to participate.”

TG1 respondents receive one of the following two texts (assigned at random) serving “controlled extrinsic” forms of motivation:

(a) “With your participation, you will contribute with your special knowledge and unique insights and experience”
(b) “With your participation, you will have the chance to further reflect upon you own leadership style and development”

TG2 respondents receive one of the following two texts (assigned at random) appealing to prosocial forms of motivation:

(a) “With your participation, you will help establish new and important knowledge about management training programmes in Denmark – which ultimately will be of benefit to citizens and society”
(b) “With your participation, you will help establish new and important knowledge about management training programmes in Denmark – which ultimately will be of benefit to other managers.”

As our outcome measure, all respondents are asked to report whether they may be contacted for an interview. Response categories are “yes” and “no.” Respondents who agree to participate are asked to report their contact email address (open textbox).

We test the following hypotheses:

H1: Text appeals serving “controlled extrinsic” forms of motivation increase willingness to participate in research (relative to a basic text)
H2: Text appeals serving prosocial forms of motivation increase willingness to participate in research (relative to a basic text)
H3: Text appeals serving prosocial forms of motivation increase willingness to participate in research (relative to text appeals serving “controlled extrinsic” forms of motivation)
Randomization Method
Randomization is carried out by simple randomization by computer (randomization based on a single sequence of random assignments).
Randomization Unit
The individual survey respondent.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
Providing an exact estimate is difficult. Our observations will be those out of the 10,000 public managers who are invited to participate in the survey who (a) chose to participate and (b) reach the end of the survey (where are survey item interventions occur).

Given uncertainty about the survey response rates and survey dropout attrition, our conservative estimate is around 1,800 observations
Sample size: planned number of observations
Providing an exact estimate is difficult. Our observations will be those out of the 10,000 public managers who are invited to participate in the survey who (a) chose to participate and (b) reach the end of the survey (where are survey item interventions occur). Given uncertainty about the survey response rates and survey dropout attrition, our conservative estimate is around 1,800 observations
Sample size (or number of clusters) by treatment arms
SAMPLE SIZE (OR NUMBER OF CLUSTERS) BY TREATMENT ARMS
CG: 600 public managers
T1: 600 public managers (300 to text TG1a; 300 to text TG1b)
T2: 600 public managers (300 to text TG2a; 300 to text TG2b)
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan

Analysis Plan Documents

Analysis Plan

MD5: da7e5b9b0b500ee2cfb68ddf8f4a6171

SHA1: cb650c31711ebb4b9352e4ec46d4aca46ca71980

Uploaded At: September 25, 2020

Post-Trial

Post Trial Information

Study Withdrawal

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials