Survey experiments on citizens' engagement

The Innovation Growth Lab and the Centre for Collective Intelligence Design at Nesta are conducting an online survey of European citizens’ attitudes to social challenges and their willingness to participate in activities aimed at addressing these challenges. The survey will be carried out with 4800 citizens in six countries of the European Union, on behalf of the European Commission. In the course of the survey, 13 randomised experiments will be used to test 13 hypotheses about how the importance they place on social challenges, their willingness to participate in activities aimed at addressing those challenges, and their attitudes to experimentation are affected by adjusting the framing of the questions or the details of the proposed activities.

External Link(s)

Registration Citation

Citation

Baeck, Peter et al. 2023. "Survey experiments on citizens' engagement." AEA RCT Registry. June 23. https://doi.org/10.1257/rct.11487-1.0

Sponsors & Partners

Interventions

Intervention(s)

The interventions will involve adjustments to the wording of several different questions in the survey. Further details are in the attached document, “Details of specific experiments”.

Intervention Start Date

2023-06-15

Intervention End Date

2023-06-30

Primary Outcomes

Primary Outcomes (end points)

Primary outcomes for each of the experiments will be one of the following:
-- Importance placed by the respondent on the five social challenges
-- Willingness to participate in a specific activity aimed at addressing one of the social challenges
-- Perception of the acceptability of experimentation in public policy.
Further details are in the attached document, “Details of specific experiments”.

Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)

None

Secondary Outcomes (explanation)

Experimental Design

Survey respondents will be randomly allocated to between two and four arms, as appropriate for each of the experiments. Further details are in the attached document, “Details of specific experiments”.

Experimental Design Details

Randomization Method

Randomisation will be carried out within the Qualtrics survey interface, with stratification by respondent's country of residence. Randomisation for each of the 13 experiments will be carried out independently.

Randomization Unit

Individual

Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters

4800 individual survey respondents

Sample size: planned number of observations

4800 individual survey respondents

Sample size (or number of clusters) by treatment arms

There will be 1200, 1600 or 2400 respondents in each treatment arm, depending on the specific experiments. Further details are in the attached document, “Details of specific experiments”.

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

Minimum detectable effect sizes to five 80% power for comparisons between each of the treatment arms in each experiment are: -- 0.12 standard deviations, for experiments with four arms -- 0.14 standard deviations, for experiments with three arms -- 0.16 standard deviations, for experiments with two arms These calculations are thought to be conservative because (a) they assume that Bonferroni correction will be used for multiple outcomes, whereas we plan to use Benjamini-Krieger-Yekutieli correction, and (b) they do not take account of explanatory power of the covariates (for which we do not currently have estimates).

Supporting Documents and Materials

Documents

Document Name

Details of specific experiments

Document Type

other

Document Description

Table containing details of the hypotheses being tested in each of the individual experiments, along with the definitions of the dependent variables, conditions, analyses and expected sample sizes for each

File

Details of specific experiments

MD5: e89239b2929b4a81967366a3e90344e2

SHA1: e57a5db287ddb597b9fce9bbd20fcae36ef46af2

Uploaded At: June 15, 2023

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

IRB