Strategy vs. Direct Response Method

Last registered on September 20, 2019

Pre-Trial

Trial Information

General Information

Title
Strategy vs. Direct Response Method
RCT ID
AEARCTR-0004737
Initial registration date
September 20, 2019

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
September 20, 2019, 9:44 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
University of Nottingham Ningbo China

Other Primary Investigator(s)

PI Affiliation
Harbin Institute of Technology, Shenzhen

Additional Trial Information

Status
In development
Start date
2019-10-07
End date
2020-10-06
Secondary IDs
Shenzhen Peacock Plan
Abstract
This document outlines a plan for an experiment that investigates when behavior in experiments that employ the strategy method coincides with or differs from experiments that use the direct response method.
External Link(s)

Registration Citation

Citation
Chen, Zhuoqiong and Marcus Roel. 2019. "Strategy vs. Direct Response Method." AEA RCT Registry. September 20. https://doi.org/10.1257/rct.4737-1.0
Experimental Details

Interventions

Intervention(s)
n/a
Intervention (Hidden)
Theoretical Motivation:
Suppose people want to be nice to people who are nice to them and prefer to be nasty to those who are nasty to them. In a two-player sequential game, player 2’s initial beliefs about player 1’s action doesn’t matter when actions are elicited by the direct response method (assuming that her social-preferences are of fairly simple nature) as she will perfectly observe what player 1 does. Depending on player 1’s action, player 2 may reward or punish player 1, or simply take some selfish action. For a rational player 2, initial beliefs are also irrelevant when choices are elicited by the strategy method as she can simply make choices conditional on player 1’s hypothetical actions and her respective preferences.
We, however, conjecture that initial beliefs affect player 2’s actions, and that this effect differs across the two elicitation methods. There exist different underlying psychological mechanisms result in such possibilities.
One such possibility is that players may lack the ability to think conditionally. As a result, a player’s preferences over nice, nasty and selfish actions at a given node can be influenced by her preferences over these actions at alternative nodes. One way to model such lack of conditional thinking is via a belief-based model in which the second mover’s social concern is a combination of her true social-preferences at a given node and her preferences in response to the action she expects player 1 to do. For example, when player 2 initially expects player 1 to take a selfish action, she is more likely to respond with a selfish - or even nasty action - in response to all of player 1’s hypothetical actions – even if some of those actions are undoubtedly nice. The second mover doesn’t fully update how she views player 1, either in terms of player 1’s social-type or with regards to the nice or nastiness of his choice. In this model, beliefs do not directly affect a person’s social preferences, for instance through norms, but through altering the person’s thought processes. The motivation behind the lack-of-conditional-thinking theory is the casual observation that under the strategy method, we see fewer cases where a second mover rewards a first mover for his niceness.

Another possible psychological mechanism is that players prefer to signal their social types at the decision node that they think is rather unlikely. Assume a selfish player 2 who always takes the monetary payoff maximizing choice under the direct response method. When her choices are not payoff relevant, she would like to signal – either to herself or to outsiders – that she is nice to nice people and nasty to nasty people. For the strategy method, this has the implication that player 2 will respond to actions she thinks rather likely with her normal selfish responds but chooses a nice or nasty action in response to actions she views as rather unlikely.

We now proceed to outline the basic setup of our experiment and sketch out predictions of theories in which initial beliefs affect behavior in experiments that use the direct response method differently to those that use the strategy method.

Experiment:
To test the hypothesis that initial beliefs matter, we manipulate player 2’s beliefs about player 1’s behavior in a sequential prisoner’s dilemma (sPD) and a mini-ultimatum (UG) game. We employ a 2 by 2 design, varying beliefs and the elicitation method (strategy vs. direct response method) for both games.

Belief manipulation occurs through providing both subjects information about aggregate player 1 behavior from previous existing experiments.

In particular, we selected experiments that either featured a large fraction of selfish-behavior (a majority of defections in the sPD and unequal offers in the UG) or a large fraction of non-selfish-behavior (a majority of cooperations in the sPD and equal offers in the UG). The payoff structure in the cited experiments were either identical or very similar to the payoffs in our sPD and UG. We will refer to these treatments as selfish-belief treatment and non-selfish-belief treatment.

Null Hypothesis: Different expectations of player 1’s behavior can directly affect player 2’s (as well as player 1’s) behavior. By manipulating beliefs for both the direct response and the strategy method, we are able to check whether player 2’s preferences are directly affected by different beliefs about player 1, for example because social preferences are not only driven by preferences over outcomes but also by (1) whether the other player adheres to social norms, (2) by positive or negative surprises about the other player’s behavior, (3) by experimenter demands, etc. The null hypothesis is that the effect of beliefs is constant across the two elicitation methods.

When second movers suffer from a lack of conditional thinking, we derive the following predictions (holding constant the direct influence of beliefs on preferences)
sPD: for the strategy method, the second mover cooperates less in response to cooperation when she has more selfish-beliefs.
UG: for the strategy method, the second mover rejects an unequal offer less often for more non-selfish beliefs.

Note: such theory also predicts higher cooperation in response to defection for more non-selfish beliefs in the sPD and higher rejections of equal splits for more selfish-beliefs.
It is important to highlight, however, that the intensity for non-selfish actions is likely quite different for different nodes of the games. As such, we expect that these more extreme changes to behavior to occur relatively infrequent.


When second mover follows the alternative belief-based model of signaling their social type, the predictions are exactly the opposite of the previous model:
sPD: for the strategy method, the second mover cooperate more in response to cooperation when she has more selfish-beliefs
UG: for the strategy method, the second mover rejects an unequal offer more often for more non-selfish beliefs.
Intervention Start Date
2019-10-07
Intervention End Date
2020-10-06

Primary Outcomes

Primary Outcomes (end points)
Player 1 and Player 2 choices.
Primary Outcomes (explanation)
n/a

Secondary Outcomes

Secondary Outcomes (end points)
Player beliefs; Control questions with regards to experimental instructions; survey questions such as gender, age, location, field of study, type of degree, employment status, household income, and previous experience with experiments
Secondary Outcomes (explanation)
n/a

Experimental Design

Experimental Design
n/a
Experimental Design Details
Experimental setting:

The experiment is run on Amazon Turk with participant’s locations restricted to the USA.
Subjects play both games in random order and are randomly assigned to the role of player 1 or player 2 for the entire experiment. Subjects interact with a different participant for the second game.

If we assume that the dif-in-dif effect of beliefs is 10%, a power analysis suggests that we require around* 550 subjects at a given node to detect such difference in player 2’s behavior. (* the exact number varies slightly with the levels of player 2’s selfish/non-selfish behavior). As we only observe one response of player 2 under the direct response method, we increase the sample of the direct response treatment.

Overall, we plan to obtain 500 observations for each of the strategy method treatments and 1500 observations for each of the direct response method treatments. We triple observations for the direct response method as player 1 is unlikely to take each of his action with equal probability. For reasonable frequencies of selfish to non-selfish behavior by both player 1 and player 2, this results in a power of 80% at a 10% significance level.

Caveat: Since the order of games is random, this analysis assumes that there is no order effect present in the data. If we were to detect an order effect, we would double the observations in order to obtain the same power.
Randomization Method
randomization is automatically done by a computer at the time the individual starts the experiment
Randomization Unit
the unit (clusters) of randomization in our experiment is at the individual level
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
equals the total number of participants
Sample size: planned number of observations
4000 people
Sample size (or number of clusters) by treatment arms
500 for each of the two strategy method treatments, 1500 for each of the two direct response method treatments
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
At a power of 80% and a significance level of 10%, the minimum detectable effect size is a difference of 10 percentage points.
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials