Priming Public Managers to Participate in Research: Evidence from a Low-Intensity Survey Experiment

The interventions consist of low-intensity text appeals appearing at the end of an electronic survey. All survey respondents are notified that a set of qualitative interviews are to be conducted following the survey examination. By random assignment, respondents are then exposed to one of the following five text appeals:

(1) “We want to hear if you are willing to participate.” [in Danish: “I den forbindelse vil vi høre, om du vil deltage.”]
(2) “With your participation, you will contribute with your special knowledge and unique insights and experience” [in Danish: “Med din deltagelse vil du bidrage med din særlige viden og unikke oplevelser og erfaringer.”]
(3) “With your participation, you will have the chance to further reflect upon you own leadership style and development” [in Danish: “Med din deltagelse vil du få mulighed for at reflektere nærmere over din egen ledelsesstil og lederudvikling.”]
(4) “With your participation, you will help establish new and important knowledge about management training programmes in Denmark – which ultimately will be of benefit to citizens and society.” [in Danish: “Med din deltagelse vil du bidrage til at skabe ny og vigtig viden om lederuddannelser i Danmark – hvilket i sidste ende vil være til stor gavn for borgere og samfundet.”]
(5) “With your participation, you will help establish new and important knowledge about management training programmes in Denmark – which ultimately will be of benefit to other managers.” [in Danish: “Med din deltagelse vil du bidrage til at skabe ny og vigtig viden om lederuddannelser i Danmark – hvilket i sidste ende vil være til stor gavn for andre ledere.”]

2020-09-23

2020-10-07

Willingness to participate in a follow-up interview study (1) or not (0).

As our outcome measure, all respondents are asked to report whether they may be contacted for an interview. Response categories are “yes” and “no.” Respondents who agree to participate are asked to report their contact email address (open textbox). Answering "yes" and reporting a valid email adress will be considered as "willingness to participate in a follow-up interview study."

About 10,000 public managers in Denmark receive an invitation to participate in an electronic survey.
At the end of the survey, respondents are invited to participate in a follow-up interview study. In particular, all respondents are exposed to a survey item stating that a set of qualitative interviews are to be conducted following the survey examination. Our experimental design involves manipulation of the exact survey item text.

Each respondent is randomly assigned to either a control group (CG) or one of two treatment groups, TG1 or TG2. Respondents are assigned to CG, TG1, and TG2 with equal probability (i.e., 33/33/33 spilt).

CG respondents receive the following text: “We want to hear if you are willing to participate.”

TG1 respondents receive one of the following two texts (assigned at random) serving “controlled extrinsic” forms of motivation:

(a) “With your participation, you will contribute with your special knowledge and unique insights and experience”
(b) “With your participation, you will have the chance to further reflect upon you own leadership style and development”

TG2 respondents receive one of the following two texts (assigned at random) appealing to prosocial forms of motivation:

(a) “With your participation, you will help establish new and important knowledge about management training programmes in Denmark – which ultimately will be of benefit to citizens and society”
(b) “With your participation, you will help establish new and important knowledge about management training programmes in Denmark – which ultimately will be of benefit to other managers.”

As our outcome measure, all respondents are asked to report whether they may be contacted for an interview. Response categories are “yes” and “no.” Respondents who agree to participate are asked to report their contact email address (open textbox).

We test the following hypotheses:

H1: Text appeals serving “controlled extrinsic” forms of motivation increase willingness to participate in research (relative to a basic text)
H2: Text appeals serving prosocial forms of motivation increase willingness to participate in research (relative to a basic text)
H3: Text appeals serving prosocial forms of motivation increase willingness to participate in research (relative to text appeals serving “controlled extrinsic” forms of motivation)

About 10,000 public managers in Denmark receive an invitation to participate in an electronic survey.
At the end of the survey, respondents are invited to participate in a follow-up interview study. In particular, all respondents are exposed to a survey item stating that a set of qualitative interviews are to be conducted following the survey examination. Our experimental design involves manipulation of the exact survey item text.

Each respondent is randomly assigned to either a control group (CG) or one of two treatment groups, TG1 or TG2. Respondents are assigned to CG, TG1, and TG2 with equal probability (i.e., 33/33/33 spilt).

CG respondents receive the following text: “We want to hear if you are willing to participate.”

TG1 respondents receive one of the following two texts (assigned at random) serving “controlled extrinsic” forms of motivation:

(a) “With your participation, you will contribute with your special knowledge and unique insights and experience”
(b) “With your participation, you will have the chance to further reflect upon you own leadership style and development”

TG2 respondents receive one of the following two texts (assigned at random) appealing to prosocial forms of motivation:

(a) “With your participation, you will help establish new and important knowledge about management training programmes in Denmark – which ultimately will be of benefit to citizens and society”
(b) “With your participation, you will help establish new and important knowledge about management training programmes in Denmark – which ultimately will be of benefit to other managers.”

As our outcome measure, all respondents are asked to report whether they may be contacted for an interview. Response categories are “yes” and “no.” Respondents who agree to participate are asked to report their contact email address (open textbox).

We test the following hypotheses:

H1: Text appeals serving “controlled extrinsic” forms of motivation increase willingness to participate in research (relative to a basic text)
H2: Text appeals serving prosocial forms of motivation increase willingness to participate in research (relative to a basic text)
H3: Text appeals serving prosocial forms of motivation increase willingness to participate in research (relative to text appeals serving “controlled extrinsic” forms of motivation)

Randomization is carried out by simple randomization by computer (randomization based on a single sequence of random assignments).

The individual survey respondent.

No

Providing an exact estimate is difficult. Our observations will be those out of the 10,000 public managers who are invited to participate in the survey who (a) chose to participate and (b) reach the end of the survey (where are survey item interventions occur).

Given uncertainty about the survey response rates and survey dropout attrition, our conservative estimate is around 1,800 observations

Providing an exact estimate is difficult. Our observations will be those out of the 10,000 public managers who are invited to participate in the survey who (a) chose to participate and (b) reach the end of the survey (where are survey item interventions occur). Given uncertainty about the survey response rates and survey dropout attrition, our conservative estimate is around 1,800 observations

SAMPLE SIZE (OR NUMBER OF CLUSTERS) BY TREATMENT ARMS
CG: 600 public managers
T1: 600 public managers (300 to text TG1a; 300 to text TG1b)
T2: 600 public managers (300 to text TG2a; 300 to text TG2b)

Analysis Plan