Advice in Strategic Environments: Experimental Evidence on Recommendation Design

Subjects are confronted with five different two player, two action games. In a sequence of rounds, they will first be shown the payoff matrix for each new game. Then, they will be asked to design recommendation devices for that game. A recommendation device involves the specification of probabilities with which the two players 1, and 2 will play either of their two available actions, labeled Red or Blue. Probabilities are elicited via the specification of the colors of balls in a container of 24 balls. Each ball is split in two halves, with one half labeled 1 and the other half labeled 2. Players designate the color of each half of all balls in the container. This decision comprises the recommendation device. Robot players are then presented with this recommendation device and decide whether or not to follow the recommendations, based on a best response analysis from a simulation of playing the games a large number of periods.

If one or more of the 2 robot players does not follow the recommendation device, the subject’s payoff from the round of the game is 0. If, in a round, the recommendations are followed always (i.e. by all robot players, regardless which ball is taken from the container), subjects (potentially) get positive number of points, as specified below for different treatments.

Each game is played for 5 rounds and at the start of each round following the first, subjects get a feedback for the previous round. If all recommendations were followed, they learn their potential payoff and whether this is maximum possible in this game. If some recommendations were not followed, which player did not follow a recommendation and for which ball. They then have the opportunity to re-design the recommendation device or keep it unchanged. Their potential payoff from the game is for the round with their largest payoff for this game. After all games are played, one game is chosen randomly for payoff purposes.

In a prior study (AEARCTR-0009706) we had subjects complete a similar task and paid them according to the minimum average payoff earned by robot players playing each game, provided that recommendations were always followed (that is, that a correlated equilibrium was achieved). We will use that prior study’s treatment for comparison with two new treatments - the two treatments of the present study:

TREATMENT 1: If, in a round, the subject’s recommendations are always followed, the subject earns a default positive (flat) payoff. Thus, in this treatment, the subject’s goal is simply to construct a correlated equilibrium without regard to the payoffs actually earned by the robot players.

TREATMENT 2: If, in a round, the subject’s recommendations are always followed, the subject earns the average payoff of all robot players in the role of Player 1. Thus, in this treatment, the subject’s goal is additionally aligned with one of the players.

Our comparison across treatments will make use of the outcome variables described below.

2025-04-09

2025-12-31

1. Recommendation devices constructed by the subjects (describing the recommendations in the form of probabilities to play of each outcome of the game).

2. Multiple measures on how well each subject performs for each game (and for each round) and the average of those measures across subjects. Measures for a subject and a game are:
- CE: describes whether CE is found or not [index: 1 or 0];
- Selten: CE minus Volume, where the Volume is the volume of the set of all correlated equilibria in the game;
- Score: relative payoff of the best device (across 5 rounds) wrt the best CE [a real number between 0 and 1] - relevant only in Treatment 2, when there is an additional goal;
- TopScore: describes whether best CE is found or not [index: 1 or 0] - relevant only in Treatment 2, when there is an additional goal;
- N-RSME2: geometric distance between the best CE of the game and the best device (based on the Euclidean distance between the two distributions, normalized to a real number between 0 and 1). Relevant only in Treatment 2, when there is an additional goal.
- N-RSME1: geometric distance between the CE set of the game and the best device (normalized to a real number between 0 and 1) - relevant for those subjects who scored 0 for CE.

3. Payoffs earned and feedback on outcomes: whether recommendations were followed or not.

Answers to cognitive tests, financial literacy questions, and general demographic information will be collected through a questionnaire as part of the experimental design.

Subjects face five different 2 player, 2 action games represented by payoff matrices. For each game they are asked to design a recommendation device specifying for each game which action each player 1 or 2 should play in each round of the 2x2 game. This recommendation device involves the specification of probabilities with which each player will be recommended to choose either of their two actions. Robot players are then presented with this recommendation device. The robot players then decide whether or not to follow the recommendations, based on a best response analysis from a simulation of playing each game a large (infinite) number of periods under the given recommendation device.

If one or more of the 2 robot players does not follow the recommendation device, the subject’s payoff from the round of the game is 0. If, in a round, the recommendations are followed always (i.e. by all robot players, regardless which ball is taken from the container), subjects (potentially) get positive number of points, as specified below.

Each game is played for five rounds and at the start of each round following the first, subjects get a feedback for the previous round. If all recommendations were followed, they learn their potential payoff and whether this is maximum possible in this game. If some recommendations were not followed, which player did not follow a recommendation and for which ball. They then have the opportunity to re-design the recommendation device or keep it unchanged. Their potential payoff from the game is for the round with their largest payoff for this game. After all games are played, one game is chosen randomly for payoff purposes.

The individuals providing the recommendations (the subjects) receive rewards if their recommendations are followed by the robot player.

Additionally, in some treatments, their rewards vary depending on a specific goal related to the players’ payoffs in the game.

In the previous study (AEARCTR-0009706), we investigated the case when the subjects are incentivized to design the correlated equilibrium with the most efficient and fair payoff.

In this study we will run two additional treatments:

TREATMENT 1: If, in a round, the recommendations are followed always, the subject earns a default positive payoff. Thus, in this treatment, the subject’s goal is simply to construct a correlated equilibrium.

TREATMENT 2: If, in a round, the recommendations are followed always, the subject earns an average payoff of all robot players in the role of Player 1. Thus, in this treatment, the subject’s goal is additionally aligned with one of the players.

Randomization over the game order by computer program.

Order of experimental treatments, determination of game for payoff.

Yes

100 subjects for each treatment.

Each subject is an individual observation playing several different games. The order of the games is randomized, so each subject is an observation on the various games played. 100 subjects total for each treatment.

200 subjects total, 100 in each treatment.