Experimental Design
The experiment consists of three parts and is conducted prior to the first and second panel waves of a probability-based, bimonthly online panel in 2026. The online panel studies individual attitudes and preferences relevant to political and economic decision-making processes.
For each survey wave, an email invitation is sent to all panelists on the first day of every odd month. The survey remains open for one month. Panelists receive an incentive based on the length of the questionnaire (about 20-25 minutes; €0.25 euros per minute), which is credited to their study account. In addition, participants receive a conditional yearly bonus of €10 if they complete all surveys, or €5 if they miss only one. Incentives can be cashed out, paid as an Amazon voucher, or donated to charity twice a year.
In addition to the standard recruitment interview questions (sociodemographic characteristics, Big Five personality traits, need for cognition), we ask participants about their motivations to participate (monetary incentive/ interest in study content/ desire to express opinion/ sample representativeness/ support for social science research/ support for the research team/ other). Participants first select all motivations that apply and then indicate their primary motivation among these. We plan to use these interview questions, and particularly, the motivation question to explore treatment effect heterogeneity.
The study follows a between-subject design with one factor comprising four levels.
Between September and November 2025, new panel members are recruited based on two different sampling frames (population register vs. commercial address provider). After recruitment, participants are invited to participate in their first panel wave starting on 1 November 2025.
In Experiment 1, we randomly assign the newly recruited panelists to one of four equally sized treatment groups within each stratum. Participants in the first treatment group T1.1 receive a “Thank-you” postcard with printed text, while those in the second group T1.2 receive the same postcard but with handwritten text. Participants in the third group T1.3 receive the “Thank-you” postcard with printed text along with an unconditional monetary incentive of €5. The fourth group T1.4 serves as a control group and therefore receives no treatment.
The “Thank-you” postcards for treatment groups T1.1 - T1.3. are sent on 15 December 2025 and their second panel wave opens on 1 January 2026.
After completion of the second panel wave in January, we use the resulting data to determine the optimal treatment allocation based on participants’ available characteristics. To do so, we implement a policy tree that maximizes survey response conditional on treatment-specific costs, subject to an overall budget constraint equal to the cost of random assignment across the four treatments.
In Experiment 2, we split the control group T1.4 into two equally sized subgroups in order to evaluate the learned optimal policy. In treatment group T2.1, we implement the optimal policy derived from wave 2. In treatment group T2.2, we repeat Experiment 1 and randomly assign participants to one of the four equally sized treatment groups. The treatment allocation in T2.1 and T2.2 is constrained to ensure equal overall costs. The “Thank-you” postcards are sent on 15 February 2026 and the third panel wave opens on 1 March 2026. Panel members in treatment groups T1.1 - T1.3 do not receive an additional intervention.
We use data from the third panel wave in March to evaluate whether the optimal treatment allocation yields higher response rates than random assignment.
For additional evidence on the efficiency of the optimal policy, we replicate Experiment 2 with non-treated panel members from an earlier recruitment in Experiment 3 (T3.1 and T3.2).