Using Bayes Factor Functions to synthesize evidence from replications of experiments

Last registered on April 04, 2025
View Trial History

Pre-Trial

Trial Information

General Information

Title
Using Bayes Factor Functions to synthesize evidence from replications of experiments
RCT ID
AEARCTR-0015736
Initial registration date
April 03, 2025

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
April 04, 2025, 1:35 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
United States of America
Region
MA

Primary Investigator

Name
Jeffrey Livingston
Affiliation
Bentley University
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Ezgi Kurt
PI Affiliation
Bentley University
Contact Investigator
PI Name
Laura Young
PI Affiliation
Bentley University
Contact Investigator

Additional Trial Information

Status
On going
Start date
2025-04-03
End date
2025-09-19
Keywords
Other
Additional Keywords
Bayes Factor Functions, Replication
JEL code(s)
D90, C11, C12
Secondary IDs
Prior work
This trial is based on or builds upon one or more prior RCTs.
Abstract
Following replications of an experiment, we show how Bayes Factor Functions can be used to update our beliefs about a) whether an effect exists and b) the magnitude of the effect. We compare this to Butera et al. (2020), who show how to use a method introduced by Maniadis et al. (2015) to calculate Post-Study Probabilities that an effect exists, following a new replication of an experiment. We conduct several replications of the seminal Kahneman, Knetsch, and Thaler (1990) experiment as an illustrative example.
External Link(s)

Registration Citation

Citation
Kurt, Ezgi, Jeffrey Livingston and Laura Young. 2025. "Using Bayes Factor Functions to synthesize evidence from replications of experiments." AEA RCT Registry. April 04. https://doi.org/10.1257/rct.15736-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
We replicate the Kahneman, Knetsch, and Thaler (1990) experiment. The instructions are based on a replication and extension conducted by Gächter et al. (2022).

In the experiment, subjects are randomized (via random number generation by computer) into a role as a seller of a coffee mug or a buyer of a coffee mug.

Sellers are given a coffee mug. Each mug was purchased from the Bentley University Bookstore at a cost of $12.98 per mug. They are allowed to keep the mug with them. The experiment begins approximately 20 minutes later.

In the experiment, subjects first play two rounds for hypothetical payments. In these initial rounds:
Sellers are told they own a token. The value of the token to them is induced. The value is randomly selected from prices between $0.00 and $12.50 in 50 cent increments. They are then told that the price of the token will be randomly selected from this range. They are asked if they would be willing to sell the token at each of these prices. Their earnings are determined by whichever price is randomly selected and what they said they would do at that price. IF they do not sell their token at the randomly-chosen price, their payoff is the value of the token. If they do sell their token at the randomly chosen price, their payoff is the price. This is repeated twice, with new induced values and prices selected in each round.

In these initial rounds, buyers are told they can purchase a token if they wish. The value of the token to them is induced. The value is randomly selected from prices between $0.00 and $12.50 in 50 cent increments. They are then told that the price of the token will be randomly selected from this range. They are asked if they would be willing to buy a token at each of these prices. Their earnings are determined by whichever price is randomly selected and what they said they would do at that price. If they do not buy a token at the randomly chosen price, their payoff is $0. If they do buy a token at the randomly chosen price, their payoff is their value for the token minus the price. This is repeated twice, with new induced values and prices selected in each round.

Finally, the third round involves the purchase or sale of the mug. between $0.00 and $12.50 in 50 cent increments. Sellers are told that the price of the mug will be randomly selected from the range of $0.00 and $12.50. They are asked if they would be willing to sell their mug at each of these prices. The price is then randomly selected. If they said they would want to sell their mug at the randomly chosen price, they return the mug to us and are paid the price. If they said they would not want to sell their mug at the randomly chosen price, they keep their mug.

Buyers are told that the price of the mug will be randomly selected from the range of $0.00 and $12.50. They are asked if they would be willing to buy a mug at each of these prices. The price is then randomly selected. If they said they would want to buy a mug at the randomly chosen price, they are given a mug and pay us the randomly selected price. If they said they would not want to buy a mug at the randomly chosen price, they do not purchase a mug and no transaction occurs.

The prices chosen in each round and values induced in rounds 1 and 2 are drawn separately for each subject.


Thus far the experiment has been replicated six times.

We plan on repeating this endeavor, conducting another five replications.
Intervention (Hidden)
Intervention Start Date
2025-04-03
Intervention End Date
2025-09-19

Primary Outcomes

Primary Outcomes (end points)
Willingness to pay (WTP) (for subjects who are not initially endowed with a coffee mug)
Willingness to accept (WTA) (for subject who are initially endowed with a coffee mug)
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We replicate the Kahneman, Knetsch, and Thaler (1990) experiment. The instructions are based on a replication and extension conducted by Gächter et al. (2022).

In the experiment, subjects are randomized (via random number generation by computer) into a role as a seller of a coffee mug or a buyer of a coffee mug.

Sellers are given a coffee mug. Each mug was purchased from the Bentley University Bookstore at a cost of $12.98 per mug. They are allowed to keep the mug with them. The experiment begins approximately 20 minutes later.

In the experiment, subjects first play two rounds for hypothetical payments. In these initial rounds:
Sellers are told they own a token. The value of the token to them is induced. The value is randomly selected from prices between $0.00 and $12.50 in 50 cent increments. They are then told that the price of the token will be randomly selected from this range. They are asked if they would be willing to sell the token at each of these prices. Their earnings are determined by whichever price is randomly selected and what they said they would do at that price. IF they do not sell their token at the randomly-chosen price, their payoff is the value of the token. If they do sell their token at the randomly chosen price, their payoff is the price. This is repeated twice, with new induced values and prices selected in each round.

In these initial rounds, buyers are told they can purchase a token if they wish. The value of the token to them is induced. The value is randomly selected from prices between $0.00 and $12.50 in 50 cent increments. They are then told that the price of the token will be randomly selected from this range. They are asked if they would be willing to buy a token at each of these prices. Their earnings are determined by whichever price is randomly selected and what they said they would do at that price. If they do not buy a token at the randomly chosen price, their payoff is $0. If they do buy a token at the randomly chosen price, their payoff is their value for the token minus the price. This is repeated twice, with new induced values and prices selected in each round.

Finally, the third round involves the purchase or sale of the mug. between $0.00 and $12.50 in 50 cent increments. Sellers are told that the price of the mug will be randomly selected from the range of $0.00 and $12.50. They are asked if they would be willing to sell their mug at each of these prices. The price is then randomly selected. If they said they would want to sell their mug at the randomly chosen price, they return the mug to us and are paid the price. If they said they would not want to sell their mug at the randomly chosen price, they keep their mug.

Buyers are told that the price of the mug will be randomly selected from the range of $0.00 and $12.50. They are asked if they would be willing to buy a mug at each of these prices. The price is then randomly selected. If they said they would want to buy a mug at the randomly chosen price, they are given a mug and pay us the randomly selected price. If they said they would not want to buy a mug at the randomly chosen price, they do not purchase a mug and no transaction occurs.

The prices chosen in each round and values induced in rounds 1 and 2 are drawn separately for each subject.


Thus far the experiment has been replicated six times.

We plan on repeating this endeavor, conducting another five replications.
Experimental Design Details
Randomization Method
random number generation by computer
Randomization Unit
individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
n/a
Sample size: planned number of observations
20-35 per replication
Sample size (or number of clusters) by treatment arms
20-35 per replication
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
0.2 for a MDE of around 0.5 SD. The replications are intentionally severely underpowered.
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
Bentley University
IRB Approval Date
2024-03-22
IRB Approval Number
240322069
Analysis Plan

Analysis Plan Documents

PAP

MD5: 3c6d32afbd74d8738a230bbdcb3ffddc

SHA1: d3ae5d583caa954351be63adc69cd2c731cf1343

Uploaded At: April 03, 2025

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials