Outcome bias: Information complexity and domain specificity

Principals and agents will interact under two treatments that affect the presentation of which states of the world the agent controls and which the agent does not. Principals' perceptions of agents across other domains will then be collected.

July 2025 addition:

Third parties will repeat the procedures of the principals in eliciting perceptions of agents.

There are two main areas of exploration:
1) How does information affect the interaction?
2) How does the outcome of the interaction affect principals' perceptions of the agent in other domains?

To answer the first question, the principal-agent interaction will occur under one of two treatments that change the way that the states of the world under the agent's control are displayed. Under Combined Information, all states of the world (those the agent controls and those he does not) are presented as part of the same random process. Under Separated Information, the states of the world that the agent controls are presented as part of a separate random process from the states of the world that the agent does not control.

To answer the second question, we will follow up the principal-agent interaction by eliciting principals' perceptions of agents across four other domains. For each elicitation, the intervention is that we will show principals one randomly-selected round in which the agent participated. This vignette will include the agent's effort and the outcome that their principal experienced that round (a win or a loss). The elicitations of perceptions will be incentivized in a way that makes accurate perceptions to be optimal for the principal.

July 2025 addition:

We will repeat the procedures of the second question using third parties in the roles of principals. We will elicit the third parties' perceptions of agents across the same four domains. For each elicitation, the intervention is that we will show third parties one randomly-selected round in which the agent participated. This vignette will include the agent's effort and the outcome that their principal experienced that round (a win or a loss). The elicitations of perceptions will be incentivized in a way that makes accurate perceptions to be optimal for the third party.

2023-09-05

2023-12-31

For the first area of exploration, the key outcomes we will measure are:

Investment by the agent under different information regimes.
Punishment (in general) by the principal under different information regimes.
How each information regime affects the following:
How sensitive the principal's punishment choice is to the agent's effort.
I will regress punishment on effort.
How sensitive the principal's punishment choice is to their outcome (win or lose).
I will regress punishment on outcome after controlling for effort.
How sensitive the principal's punishment choice is to their outcome when that outcome is within the agent's control.
I will regress punishment on outcome interacted with whether the region is within the agent's control after controlling for effort and whether the region is within the agent's control.
How sensitive the principal's punishment choice is to their outcome when that outcome is outside of the agent's control.
I will regress punishment on outcome interacted with whether the region is within the agent's control and whether the principal was assigned to the Split Information treatment after controlling for effort, whether the region is within the agent's control, the information treatment, and the interactions of these control variables.

For the second area of exploration, the key outcomes we will measure are:

How an agent's effort affects the likelihood that a principal will select them to be a dictator in a subsequent dictator game.
How an agent's effort affects the likelihood that a principal will select them to perform a real-effort task on their behalf in a subsequent game.

For the second area of exploration, the key outcomes will be the perceptions of the principal that we elicit in each of four domains:

First, the principals will guess the average effort that the agent invested across all rounds. Their reward will increase as their guesses are more accurate.
I will regress the principal's guess on the outcome observed after controlling for the effort observed.

Second, the principals will select an agent to play the role of dictator in a dictator game. The selected dictator will determine any additional payments given to the principal.
I will regress the principal's selection of dictator on the outcome observed after controlling for the effort observed.
NOTE: variables will be standardized within the set of dictators a principal observes since they must make a selection from a set of four possible dictators

Third, the principals will select an agent to perform a real-effort task on their behalf. The principal will receive greater earnings if the selected agent performs the real-effort task accurately.
I will regress the principal's selection of agent for the real-effort task on the outcome observed after controlling for the effort observed.
NOTE: variables will be standardized within the set of dictators a principal observes since they must make a selection from a set of four possible dictators

Fourth, the principals will guess demographic characteristics of the agents. They will be paid for their accuracy.
I will regress the principal's guesses about demographic information on the outcome observed and on the effort observed to measure stereotyping in both dimensions.


July 2025 addition:

To supplement the primary data collection between the counterparties (Principals and Agents), I will elicit behavior from third parties who are replicating the procedures of the second area of exploration. These third parties will feature identical key outcomes:

How an agent's effort affects the likelihood that the third party will select them to be a dictator in a subsequent dictator game.
How an agent's effort affects the likelihood that the third party will select them to perform a real-effort task on their behalf in a subsequent game.

For this replication/expansion, the key outcomes will be the perceptions of the third parties that we elicit in each of four identical domains. Like principals, third parties will either be assigned to observe information in a combined or split information treatment. The information treatment will be a key source of heterogeneity.

First, the third parties will guess the average effort that the agent invested across all rounds. Their reward will increase as their guesses are more accurate.
I will regress the guess on the outcome observed after controlling for the effort observed (heterogeneity by information presentation).

Second, the third parties will select an agent to play the role of dictator in a dictator game. The selected dictator will determine any additional payments given to the third parties.
I will regress the third parties' selection of dictator on the outcome observed after controlling for the effort observed (heterogeneity by information presentation).
NOTE: variables will be standardized within the set of dictators a third party observes since they must make a selection from a set of four possible dictators

Third, the third parties will select an agent to perform a real-effort task on their behalf. The third party will receive greater earnings if the selected agent performs the real-effort task accurately.
I will regress the third parties' selection of agent for the real-effort task on the outcome observed after controlling for the effort observed (heterogeneity by information presentation).
NOTE: variables will be standardized within the set of dictators a third party observes since they must make a selection from a set of four possible dictators

Fourth, the third parties will guess demographic characteristics of the agents. They will be paid for their accuracy.
I will regress the third parties' guesses about demographic information on the outcome observed and on the effort observed to measure stereotyping in both dimensions (heterogeneity by information presentation).

For the first area of exploration, we care about how sensitive principals are to the presentation of information because the Separated Information treatment makes clear the limitations on the agent's control. Thus, if punishment diminishes in these states of the world in the Separated Information treatment, the principal may have previously misattributed their loss to the agent's effort under Combined Information but no longer does because of the clarity of information. If, however, principals continue to punish based on outcomes outside of the agent's control despite the clarity of information, it is less likely to result from confusion and is more likely the result of negative affect causing the principal to "blame" the agent.

For the second area of exploration, we care about how much behavior in the principal-agent context affects the principal's perceptions of agents in other domains. In particular, we care about how the principal's perceptions are colored by the luck that the agent experienced in prior interactions. Under attribution bias, we may expect that a principal will think that a lucky agent 1) will exert more effort, on average; 2) will allocate more to them in a dictator game; or 3) will exert more effort in a real-effort task. Conditioning on luck in this way is sub-optimal behavior for the principal. Additionally, this misattribution may cause stereotyping, so we care about whether a principal may think that a lucky agent is more or less likely to be 1) male or female; 2) old or young; or 3) high or low GPA.

July 2025 addition:

We replicate the second area of exploration with third parties taking the place of principals to ensure that the phenomena observed cannot be attributable to anything related to confusion about reciprocity. If third parties display perceptions that are similarly colored by the luck that the agent experienced in prior interactions, then this confirms the misattribution of luck to the agent's underlying type. Unlike principals, third parties will not display behavior that is sensitive to an emotional state that is affected by the interaction nor will their behavior be sensitive to confusion about reciprocity to agents because the third parties have not interacted with these agents.

Under attribution bias, we may expect that a third party will think that a lucky agent 1) will exert more effort, on average; 2) will allocate more to them in a dictator game; or 3) will exert more effort in a real-effort task. Conditioning on luck in this way is sub-optimal behavior for the third party. Additionally, this misattribution may cause stereotyping, so we care about whether a third party may think that a lucky agent is more or less likely to be 1) male or female; 2) old or young; or 3) high or low GPA.

Principals and agents will interact under two treatments that affect the presentation of which states of the world the agent controls and which the agent does not. Principals' perceptions of agents across other domains will then be collected.

July 2025 addition:

Third party perceptions of agents across the same domains will also be collected.

The principal will win based on the roll of an 8-sided dice. The principal will always win with a 1 or 2 and will always lose with a 7 or 8. The agent will control whether the principal wins with a 3, 4, 5, or 6. The agent must sacrifice their own money to ensure that the principal wins with these numbers. The cost per number will vary within a 3-cent band around $0.25. The principals can then destroy up to $4 of the agent's money at a cost of $0.25 per dollar.

The Combined Information treatment will present all 8 numbers as part of one random process. In announcing the outcome, we will simply announce the number and outcome (win or loss).
The Separated Information treatment will present the 4 numbers under the agent's control on the left side and the 4 numbers outside of the agent's control on the right side. In announcing the outcome, we will first announce the side (left or right) and then the number and outcome (win or loss).

After 10 rounds of this interaction, we will elicit principals' perceptions of agents based on one randomly-selected round that they observe. Each principal will first guess the average effort that the agent exerted across all 10 of their rounds based only on the one observed round. Principals will then select agents to play the role of dictator and to serve as their delegate in a real-effort task. They will, again, only observe one randomly-selected round of the agent. For each selection, principals will be given 100 "Tokens" to allocate across 4 agents. The more Tokens, the more shares of that agent's outcome (either their dictator choice or their real-effort reward). Thus, in each case, we will evaluate the principals' allocations of tokens relative to the 4 agents they were presented with. Finally, we will ask principals to guess the demographics of the agents based on the one randomly-selected round that the principal is shown. They will be incentivized to guess the agent's gender, age, and GPA.

July 2025 addition:

We will also elicit third party perceptions of agents in the same way:

Each third party will first guess the average effort that the agent exerted across all 10 of their rounds based only on the one observed round. third parties will then select agents to play the role of dictator and to serve as their delegate in a real-effort task. They will, again, only observe one randomly-selected round of the agent. For each selection, third parties will be given 100 "Tokens" to allocate across 4 agents. The more Tokens, the more shares of that agent's outcome (either their dictator choice or their real-effort reward). Thus, in each case, we will evaluate the third parties' allocations of tokens relative to the 4 agents they were presented with. Finally, we will ask third parties to guess the demographics of the agents based on the one randomly-selected round that the third party is shown. They will be incentivized to guess the agent's gender, age, and GPA.

Principals and agents will be assigned to one information condition randomly with a 50% chance determined by the computer. They will stay in this condition throughout the study.

In each interaction, the agent's cost per number will vary randomly (again determined by the computer) within a 3-cent band around $0.25 (we will use this variation to estimate the local demand curve).

Next, within each interaction, the outcome of the principal will be (partly) randomly determined by the roll of the dice, which will be simulated by the computer.

Finally, each elicitation will feature randomly-selected observations of one round of different agents. Principals will only ever observe agents who are assigned to the same information treatment (Combined or Separated). But, otherwise, the computer will make these selections entirely at random among all rounds of all agents in the study session.

July 2025 addition:

Each elicitation from the third party will feature randomly-selected observations of one round of different agents. Third parties will only ever observe agents who are assigned to the same information treatment (Combined or Separated). But, otherwise, the computer will make these selections entirely at random among all rounds of all agents in the study session.

Each individual will experience the information randomization once. However, they will experience the cost randomization, the dice randomization, and the agent-selection randomization each round.

No

160 subjects: 80 principals and 80 agents
40 subjects per role and information treatment

July 2025 addition:

240 subjects: all third parties
120 subjects per information treatment

10 punishment decisions per principal --> 800 punishment decisions, 400 per information treatment 20 dictator choices and real-effort delegate choices per principal --> 1600 dictator and delegate choices, 800 per information treatment 5 guesses about average effort and guesses about demographics --> 400 guesses, 200 per treatment July 2025 addition: 240 subjects: all third parties 120 subjects per information treatment 40 dictator choices and real-effort delegate choices per principal --> 9600 dictator and delegate choices, 4800 per information treatment 10 guesses about average effort and guesses about demographics --> 2400 guesses, 1200 per treatment

160 subjects: 80 principals and 80 agents
40 subjects per role and information treatment

July 2025 addition:

240 subjects: all third parties
120 subjects per information treatment

From the pilot: The effect size for outcome bias in punishment choices (0.42 (SE=0.25)). The differential effect size for outcome bias in punishment choices across treatments (0.22 (SE=0.59)). The difference-in-differences in effect size for outcome bias in punishment choices across treatments and across agent-controlled and non-agent-controlled states (2.17 (SE=1.21)). I calculated a conservative 90% power using Stata's "Power" command. Testing the smallest effect size (differential punishment across treatments), the necessary sample is 806 punishment observations, corresponding to approximately 80 principals or 160 total subjects. This is more than sufficient for all other tests. The effect size for outcome bias in dictator selection (8.46 (SE=3.75)). The effect size for outcome bias in real-effort delegate selection (6.49 (SE=2.51)). The effect size for outcome bias in guesses about average contribution (0.07 (SE=0.09)). The effect size for outcome bias in guesses about demographics (Not available because of software error in pilot). Using the sample size from the first set of tests (N=160), I calculated power for each of the tests. Power for tests of outcome bias in dictator selection and real-effort agent selection approach 100%. Power is smallest for the test of outcome bias in guesses about the average contribution at 84%. Given the appropriateness of this sample size for all other tests, and that 80% is the typical benchmark for statistical power, I consider this sample size sufficient for these tests. July 2025 addition: Based on data from a June, 2025 pilot with 20 subjects: The effect size for outcome bias in dictator selection (3.65 (SE=1.30)). The effect size for outcome bias in real-effort delegate selection (4.83 (SE=1.25)). The effect size for outcome bias in guesses about average contribution (0.095 (SE=0.12)). The effect size for outcome bias in guesses about demographics (0.22 (SE=3.03)). Using a naive approach (Stata's "Power" command after residualizing the effect of observed effort), I calculated 80% power with a 5% significance level. The required sample sizes are (in order): 182 observations (i.e. 5 subjects), 120 observations (i.e. 3 subjects), 2330 observations (i.e. 233 subjects), 2326 observations (i.e. 233 subjects). To be conservative, I have rounded this up to 240 subjects.