Personalized Targeting with Self-Selection: An Online Real-Effort Experiment (Wave 1)

In the main experiment part of Wave 1, participants will be randomly assigned to one of the following four groups (incentive designs):

1. Control Group: No additional reward (To ensure fairness, a fixed compensation of 50 JPY will be paid at the end of the experiment).
2. Pay for Performance (PfP) Group: An additional reward of 5 JPY for each correct answer.
3. Bonus Loss Group: Participants are provisionally granted an additional bonus of 50 JPY but will lose this right if they fail to answer at least 10 questions correctly.
4. Relative Performance Pay Group: An additional reward is given based on comparison with the performance of other participants assigned to the same Relative Performance Pay group. Specifically, participants in this group will be ranked based on the number of correct answers within this group, and rewards will be assigned according to the following rule: 100 JPY for the top 20%, 75 JPY for 21%-40%, 50 JPY for 41%-60%, 25 JPY for 61%-80%, and 0 JPY for the bottom 20%.

2025-12-05

2025-12-12

The number of correct answers on the encryption task performed in the main experiment (second session) of Wave 1

The number of correct answers within the 2-minute time limit will be used directly as the variable indicating productivity.

The primary objective of Wave 1 is to use this experimental dataset (covariates, intervention dummies, outcomes) to construct the CATE estimation model for use in Wave 2. To achieve this objective, Wave 1 will implement a two-part experiment targeting Japanese subjects from an online survey panel.

1. Baseline Survey: All participants uniformly perform the real-effort task once under a "no additional reward" condition to measure baseline productivity (2-minutes time limit). Participants will also answer surveys before and after this task. The surveys will measure demographic variables (e.g., age, gender) and behavioral traits (e.g., Big Five, risk preference, altruism, loss aversion, trust in AI).

2. Main Experiment: About one week after the baseline survey, participants will be randomly assigned to one of four groups. Before the task, participants will take a comprehension check regarding the performance and reward structure of their assigned intervention. They will then perform the real-effort task a second time under their respective intervention conditions (2-minutes time limit). After the task, they will answer questions regarding their preferences for reward incentives and their willingness to accept machine learning personalization.

The real-effort task used will be an encryption task (Erkal, Gangadharan, & Nikiforakis, 2011). This is a task where participants convert three-letter English words into numbers based on a provided conversion table. The task has 30 questions, so the maximum number of correct answers is 30.

*Erkal, N., Gangadharan, L., & Nikiforakis, N. (2011). Relative earnings and giving in a real-effort experiment. American Economic Review, 101(7), 3330-3348.

Not available

Stratified randomization based on age and gender

Individual

No

N/A

10,000

2,500

Results from a pilot study showed an effect size of Cohen’s d = 0.236 for the Pay for Performance group relative to the control group. Using a smaller, more conservative effect size of Cohen's d = 0.2 (a benchmark for small effects), the sample size required to estimate the ATE with 5% significance and 80% power is 393 participants per group, totaling 1,572. The primary objective of Wave 1 is to construct a machine learning model for personalization (CATE estimation) in Wave 2, which generally requires a larger sample than ATE detection. In light of this, and to ensure a size that sufficiently exceeds the ATE detection requirement and secures model-building precision, we set the target sample size at 10,000 participants in total (2,500 per group).