Simplicity is the ultimate sophistication? Giving information on time inconsistency to increase sophistication and commitment demand

I test two different interventions which aim to increase sophistication about time inconsistency in physical activity (PA) and in turn increase demand for a PA commitment device. The expectation of an increase in sophistication having a positive effect on Demand for Commitment is based on the prediction of several of the most popular economic models of self-control that increasing sophistication increases demand for commitment. The first intervention gives subjects information about their past time inconsistent preferences in PA, while the second gives information on their past time inconsistent behaviour. Further details are available in the hidden Intervention details section which will be visible after the completion of the study.

2021-03-22

2021-06-14

- Sophistication about PA TI
- Demand for a PA commitment device

Primary outcome 1 - Sophistication about PA TI: Measured as P/A, where P = Predicted PA for a given two-week period and A = Actual PA for the same two-week period. I will have this measure of sophistication at two time points - pre-intervention and post-intervention. The two variables used to construct the sophistication measure are measured as follows:

Predicted PA:
"Own Prediction"
For approx. 66% of subjects (randomly chosen) Predicted PA is measured using a self-report survey question asking the subject to state their predicted hours of their own physical activity for the next two weeks. See actual formulation of question in attached pdf "Supplemental material".
"Prediction for a similar other"
For approx. 33% of subjects (randomly chosen), Predicted PA is measured using a survey question asking the subject to state their prediction of how many hours of physical activity another subject with whom they've been matched (their "partner subject") will do in the next two weeks. A subject is matched with another subject who has given the same or similar responses to the Ideal PA (see Secondary Outcomes section) and Actual PA (see below) questions in a given questionnaire. The prediction for the partner subject is then used as a proxy for the the subject's prediction of their own behaviour. See actual formulation of question in attached pdf "Supplemental material".
Which of these two questions a participant receives is determined by the randomization of the participant to an Incentive type (See Experimental Design section below). Participants are randomised to different incentive types to facilitate another separate experiment on incentives being run in parallel with the experiment described here (see more detail in the Experimental design section below).
The Predicted PA measure is measured in each of questionnaires 1 and 2.

Actual PA
Measured using a self-report survey question asking the subject to state their actual hours of physical activity for the previous two weeks. See formulation of question in attached pdf "Supplemental material". Measured in each of the three questionnaires.


Primary outcome 2 - Demand for a PA commitment device: Summary index of four measures obtained from four survey questions which elicit a subject's demand for a hypothetical monetary PA commitment device. The four questions are as follows:
1. "House Money" (HM) commitment demand - Binary: Subjects are asked whether they would prefer to receive €25 per week for the next 4 weeks unconditionally, or to receive it each week conditional on doing at least 2.5 hours of PA in that week.
2. HM commitment demand - Continuous: Subjects are presented with the above two options except that €25 in the unconditional option is replaced by €X. They are asked what is the minimum value of €X for which they would choose the unconditional option.
3. Self-funded commitment demand - Binary: Subjects are asked if they want to take up a commitment device where they pay a deposit of €100 now and are repaid €25 at the end of the week for each of the next 4 weeks if they do at least 2.5 hours of PA in that week. If they fail to meet that target in a given week, they lose the €25 for that week.
4. Self-funded commitment demand - Continuous: Subjects are presented with the commitment device in (3) above except that the deposit amount of €100 is replaced by €X. They are asked what is the maximum value of €X at which they would be willing to use the commitment device.
These four measures will be used to construct a Demand for Commitment summary index as described by Anderson (2008). This is calculated by standardizing the four measures and then getting a weighted average, where the measures are weighted by their correlation with other measures and the number of missing values (a measure that is less correlated with other measures in the index will be given greater weight as it provides “new info”). Additionally, a measure with less missing values is given more weight. This summary index will then be used as the primary outcome measure. This summary index is calculated for each of questionnaire 1, 2 and 3, so I will have measures of commitment demand at three different time points. Two consistency check questions are also asked in questionnaire 1. One question presents the subject with the same options as in question 1 above except that the conditional option states that they must do "no more than" 2.5 hours of physical activity to be paid the monetary reward of €25. The second question presents the commitment device in question 3 above altered in a similar manner.
See the formulation of these questions in the attached pdf "Supplemental Material".

References:
Anderson, M. L. (2008). Multiple inference and gender differences in the effects of early intervention: A re-evaluation of the Abecedarian, Perry Preschool, and Early Training Projects. Journal of the American Statistical Association, 103(484), 1481-1495.

- Ideal PA
- Predicted PA
- Actual PA

- Ideal PA: Measured using a self-report survey question asking the subject to state their ideal hours of physical activity for the next two weeks. See actual formulation of question in attached pdf "Supplemental material". Measured in each of the three questionnaires.

- Predicted PA: See explanation in Primary outcomes section above

- Actual PA: See explanation in Primary outcomes section above

I randomly assign respondents to a control arm or to one of two different treatment arms - the TIP treatment or the TIB treatment. Subjects in the TIP treatment arm receive the TIP intervention described in the "Intervention" section above, and subjects in the TIB treatment arm receive the TIB intervention described above. Further details are available in the hidden Experimental Design details section which will be visible after the completion of the study.

The experiment is conducted through a longitudinal three-questionnaire online survey with a general population cohort (the "Lifelines" cohort based in the north of the Netherlands). Cohort members are invited to join the survey, which is programmed in Qualtrics, via an email invitation issued by the Lifelines organisation.

Respondents are randomly assigned to a control arm or to one of two different treatment arms - the TIP treatment or the TIB treatment. Subjects in the TIP treatment arm receive the TIP intervention described in the "Intervention" section above, and subjects in the TIB treatment arm receive the TIB intervention described above. Subjects are blinded to the experiment as far as is possible in this situation - subjects are not informed that they are taking part in an experiment but it is possible that they could discover that they are part of an experiment if they communicate with subjects from other arms.

Incentives - There are three different types of incentives. Participants are randomized to one of these incentive types to facilitate another separate experiment on incentives being run in parallel with the experiment described here. The randomization occurs as follows:

A.33% of subjects, measures are incentivised monetarily using Choice-matching (Cvitanić et al. 2019), a method for eliciting honest responses to survey questions. These participants receive the "Own Prediction" Predicted PA question (see Primary outcomes section above), which is incentivised with Choice-matching. See the attached PDF "Supplemental material" to see how this is explained to subjects.
B. 33% of subjects will have all measures incentivized with Choice-Matching, except for the Predicted PA question. These participants receive the "Prediction for a similar other" Predicted PA question (see Primary outcomes section above) and they receive monetary incentives for accurate predictions in this question.
C. 33% of subjects will receive no incentives. These participants receive the "Own Prediction" Predicted PA question (see Primary outcomes section above), which unincentivised.

After randomizing subjects to an incentive type, stratified randomization is used to randomize participants to one of the the three arms in this study (TIP, TIB, Control) using a single stratification factor - incentive type. This gives three strata - incentive types A, B, and C. Participants are randomized with equal probability to one of the the three arms (Control, TIP, TIB) within strata A and C. Within stratum B, participants are randomized with 66% probability to the TIP arm, 33% probability to the control arm, and zero probability to the TIB arm. This is because the TIB intervention is not possible to implement for individuals in stratum B, as they all receive the "Prediction for a similar other" prediction question, as described in the primary outcomes section above.

When analysing the data, adjustments are made to ensure that a balance is kept in incentive type between the two arms being compared (for instance, only subjects from strata A and C are included in the TIP arm when comparing to the TIB arm, as the TIB arm contains no subjects from stratum B).

Primary analysis
Linear regression will be used to estimate treatment effects on our primary outcomes. The outcomes analysed will be:
- Sophistication for the two weeks following questionnaire 2
- Demand for commitment summary index in each of questionnaire 2 and questionnaire 3
As there are two primary outcomes, False Discovery Rate-adjusted p-values ((Benjamini and Hochberg, 1995; Benjamini et al. 2006) will be presented alongside unadjusted p-values to account for multiple hypothesis testing.
I hypothesise that the two interventions will each have a positive treatment effect on each of the primary outcomes. To test this, the following comparisons will be made for each primary outcome:
- TIB Treatment v Control
- TIP Treatment v Control
- Pooled Treatment (TIB and TIP arms combined) vs Control.
I also test whether one intervention is more effective than the other, for which I have no prior expectation, using the following comparison:
TIP treatment v TIB treatment
The following variables will be used as controls: baseline values of the outcome variable being analysed, PA level, and TIP (calculated as (Ideal-Actual)/Actual for the two week period after questionnaire 1); the psychological variables trait self-control, optimism, and information avoidance; the sociodemographic variables age, gender, marital status, household composition, education; and I will also control for medical reasons that restricts the individual's ability to do PA, whether the person likes PA or not, and whether the person likes to be told when to do PA. In the regressions on Demand for Commitment, baseline sophistication will also be controlled for.
Available case analysis will be used. This means that in each analysis, any participant with missing data for the dependent variable in that analysis will be omitted from that analysis. Control variables will be encoded as categorical variables and each will have a “missing” category for participants who have missing data for that control variable.

Secondary analysis
Secondary analysis comprises firstly of analysis of secondary outcomes which is carried out in the same manner as that of primary outcomes outlined above.
Also, as part of secondary analysis, two subgroup analyses of primary outcomes will be carried out. The first analysis will be by time inconsistency categorisation in the 2 weeks preceding the intervention in questionnaire 2. That means that I have three subgroups in this analysis: 1. Time inconsistents (i.e. Ideal[Predicted PA] as measured in questionnaire 1 > Actual PA as measured in questionnaire 2); 2. Time consistents (i.e. Ideal[Predicted PA] = Actual PA); 3. Reverse time Inconsistents (i.e. Ideal[Predicted PA] < Actual PA). The second subgroup analysis will be by baseline PA levels, with three subgroups in the analysis - low, medium, high. The thresholds will be 33rd and 66th percentile.
Thirdly, a number of qualitative questions will be asked of treatment subjects at the end of questionnaire 3 in order to get some additional qualitative information on the mechanisms through which the treatment worked (or did not work). See attached pdf "Supplemental Material" for further details.
Finally, robustness checks on primary outcome findings will also be carried out. The outcome Sophistication will be disaggregated into two separate weekly measures (rather than one fortnightly measure) and analysed. For the outcome Demand for commitment, subjects who are deemed to be inconsistent in their commitment choices will be excluded in a robustness analysis. This is determined using the consistency check questions described in the outcome measures section above. A subgroup analysis by incentive type will also be carried out for robustness.

References
Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological), 57(1), 289-300.
Benjamini, Y., Krieger, A. M., & Yekutieli, D. (2006). Adaptive linear step-up procedures that control the false discovery rate. Biometrika, 93(3), 491-507.
Cvitanić, J., Prelec, D., Riley, B., & Tereick, B. (2019). Honesty via choice-matching. American Economic Review: Insights, 1(2), 179-92.

Participants are first randomized to one of three incentive types with equal probability. This is done for the purposes of another separate experiment run in parallel with the experiment described here. Then, stratified randomisation of subjects to one of the three arms in this study occurs (Control, TIP, TIB). One stratification factor is used - incentive type received by subject. Qualtrics randomizer elements are used to carry out all randomisation. See further details in the Experimental Design section above.

Individual

No

N/A

Invitations to take part in this survey will be sent out to approximately 40,000 members of the Lifelines cohort study, which should give an estimated sample size of 4,800 based on a 12% response rate observed in a pilot study with this cohort.

Estimated:
TIP arm: 2,133
TIB arm: 1,067
Control arm: 1,600
Reason for unequal distribution between arms is explained in the experimental design section above.