Promoting Physical Activity Through Relative Performance Feedback: A Double-Edged Sword

Last registered on June 27, 2022

Pre-Trial

Trial Information

General Information

Title
Promoting Physical Activity Through Relative Performance Feedback: A Double-Edged Sword
RCT ID
AEARCTR-0009664
Initial registration date
June 27, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
June 27, 2022, 5:01 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
University of Vechta

Other Primary Investigator(s)

PI Affiliation
University of Vechta
PI Affiliation
University of Vechta
PI Affiliation
University of Vechta

Additional Trial Information

Status
In development
Start date
2022-06-27
End date
2022-07-24
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This paper studies the effects of relative performance feedback on physical activity in a field experiment. Previous relative performance feedback interventions show highly heterogeneous effects. We first aim to clarify which individuals benefit and which may suffer from relative performance feedback using a highly diverse subject pool for which we have a large amount of individual data. Previous findings suggest that participants increase performance if they receive above average feedback. Therefore, we test a variant (treatment PRPF) in which we tell each subject the best rank they achieved during the observation period. This variant thus contains the most positive feedback that can be given to a person without lying. It can also be interpreted as showing the subject what abilities they have.
We compare this variant with the one in which the most frequently occupied rank is communicated to the subject as his rank (treatment RPF), and with a control group without feedback (treatment control).

Since evidence on the driving forces behind the mechanisms is still scarce, we also want to reveal the underlying behavioral mechanisms behind performance changes after relative performance feedback. To do this, we not only systematically vary the rule by which the communicated rank is selected, but we also vary whether the rule is communicated to subjects. Thus, we compare treatments RPF/PRPF with RPFno/PRPFno (with _no indicating that the rule is not disclosed).

Additionally, we test how performance after RPF changes if the comparison group is systematically varied. Previous findings suggest that the choice of a suitable comparison group. In half of the feedbacks, the comparison group is the entire study group (with subjects aged 18-74), called RPF_all/PRPF_all, and in the other half, each subject is compared to the 50 subjects most similar to their own age (RPF_age/PRPF_age). We expect more significant differences when subjects get feedback in relation to more similar peers.

We run the experiment within an ongoing, 365-day long study with about 900 participants who all aim at improving their physical activity. All those subjects have been positively health screened, are using a smartphone app (ActiVAtE Behavior) to transmit their steps (main performance measure) in a timely manner and have provided extensive individual survey data including socio-demographics, body measures, health goals, motivation, etc.. Furthermore, we have gathered their economic preferences (e.g. competitiveness, social preferences, risk preferences, cheating) using survey-based, incentivized experimental games (e.g. Ultimatum Game, Public Goods Games, Holt-Laury-Lottery, Dictator Game, Coin Toss Game). Besides, we use an incentivized belief elicitation about their previous relative performance. Most of the participants are also equipped with a fitness tracker (medisana ViFit Run) to collect data without carrying their smartphones. The consent forms and data protection concept have been approved by the University of Vechta’s data protection officer. Since participants have been recruited via television and radio within a region with about 1 million inhabitants and there was never an in-person individual or group meeting, people usually do not know each other.
External Link(s)

Registration Citation

Citation
Hiller, Maximilian et al. 2022. "Promoting Physical Activity Through Relative Performance Feedback: A Double-Edged Sword." AEA RCT Registry. June 27. https://doi.org/10.1257/rct.9664
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2022-06-27
Intervention End Date
2022-07-24

Primary Outcomes

Primary Outcomes (end points)
Performance (measured steps), performance changes
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We randomize all participants into treatment groups with varying relative performance feedback with respect to the number of steps they walked over the last few weeks. Relative performance feedback is determined as follows:

1. We look at the last seven days and add up the corresponding step counts, thus we avoid weekday effects. With this rolling window approach, we then get a smoothed version for each day compared to just looking at that day alone. We explain this to people with, "Every day we create a ranking based on the steps of the last 7 days".

2. If we do this over 14 days and rank each day, then there is a ranking value for each subject every day. In the unadorned version (RTF), we give people the value they had most often during the 14 days ("Your most frequent ranking during the last 14 days was..."), in the embellished one they get the best of these 14 values ("Your top rating in the last 14 days was...").

3. Feedback is provided in form of a table with ten percentage clusters and a classification of one’s own actual relative performance. Afterwards, the performance (daily number of steps) is further surveyed and compared with previous performance.

The relative performance feedback is varied in three ways:
• We vary between subjects whether their relative performance feedback is based on their most frequent ranking score over the last 2 weeks (RPF) or on their highest achievement (PRPF). Obviously, PRPF gives participants more positive feedback at the outcome level than RPF does in many cases. At the procedural level, PRPF tells people what they have already achieved at least once, that is, what they can accomplish. With RPF, on the other hand, the description that is most applicable to that person is sent.
• Furthermore, relative performance feedback is either given in comparison with all other participants (RPF-all/PRPF-all) or in comparison with the 50 participants closest in the age distribution (RPF-age/PRPF-age). With this approach, we generate a probably more relevant reference group for those participants.
• Last, we vary whether subjects are informed how their rank has been determined (RPF/PRPF) or not (RPFno/PRPFno). This allows us to isolate the cause of a possible effect: is it the ranking itself or does it also play a role in the subjects' evaluation and for a possible change in behavior how the ranking came about.

Participants are randomly distributed into different treatment groups (n ≈ 100 participants/group). We designed the following 9 groups, based on a mixed between-/within-subject design. The intervention phase takes 4 weeks with 2 interventions taking place at t1 (intervention start) and t2 (t1 + 2 weeks).

[Group 1] t1: no feedback; t2: no feedback

[Group 2] t1: RPF-all; t2: RPF-age

[Group 3] t1: RPF-age; t2: RPF-all

[Group 4] t1: PRPF-all; t2: PRPF-age

[Group 5] t1: PRPF-age; t2: PRPF-all

[Group 6] t1: RPFno-all; t2: RPFno-age

[Group 7] t1: RPFno-age; t2: RPFno-all

[Group 8] t1: PRPFno-all; t2: PRPFno-age

[Group 9] t1: PRPFno-age; t2: PRPFno-all

We expect e.g. that absolute performance increases most for participants who receive a positive feedback. By systematically shifting relative performance feedback in group 2 into a more positive direction (even for relatively low performers), we expect the highest performance increase in this group. We also think that comparing participants with those of similar age has a performance enhancing effect, because a relevant descriptive norm is created, which participants want to live up to. Participants know that the overall group of participants is very heterogeneous in terms of gender, age, prior experience, time restrictions, etc., and we hypothesize that there is a difference between providing relative performance feedback about all participants and providing feedback with respect to only those who are (approximately) the same age. To clearly disentangle whether a positive feedback per se drives the results or only in interaction with the allocation rule, we compare performance of group 2-5 with those of groups 6-9.

Our setting also allows us to take an exploratory approach, as we gathered an exceptional volume of additional behavioral data, which is seldom in this field of work. In our subgroup analysis, we expect insights on influences of gender, age, competitiveness, social attributes, etc. in this context while we take care of multiple testing biases.
Experimental Design Details
Randomization Method
Randomization into the different treatments is done by a computerized random draw.
Randomization Unit
Individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
Does not apply
Sample size: planned number of observations
about 900 individuals
Sample size (or number of clusters) by treatment arms
about 100
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
German Association for Experimental Economic Research e.V.
IRB Approval Date
2022-06-20
IRB Approval Number
q8KFWTq3

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials