Counterfactual Reasoning and Algorithmic Descriptions

Last registered on May 03, 2023

Pre-Trial

Trial Information

General Information

Title
Counterfactual Reasoning and Algorithmic Descriptions
RCT ID
AEARCTR-0011349
Initial registration date
May 01, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
May 03, 2023, 4:33 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Affiliation
Southwestern University of Finance and Economics

Other Primary Investigator(s)

PI Affiliation
Southwestern University of Finance and Economics

Additional Trial Information

Status
In development
Start date
2023-05-03
End date
2024-12-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This project proposes a measure of complexity for mechanisms that is based on their algorithmic descriptions. We formulate the “mental” model followed by participants who are given an algorithmic description of a mechanism as an extensive game-form that reflects the steps of the algorithmic description. Players’ strategies in this extensive-form game reflect the information used from their (direct) strategies in the resulting mechanism in each step. In this framework, the standard direct mechanism can be interpreted as eliciting a strategy for said extensive game-form. We define a family of sequential mechanisms for a given algorithmic description as the set of nodes in which the continuation strategies are elicited. A mechanism A is deemed as less complex than B if during A’s execution, its strategies elicit the actions from a subset of the nodes of B. We test this theory in a controlled laboratory experiment, by studying whether subjects ability to best respond in direct and sequential versions of the Gale-Shapley DA and Boston mechanisms will improve when these are less complex with respect to our measure.
External Link(s)

Registration Citation

Citation
Bo, Inacio and Inacio Bo. 2023. "Counterfactual Reasoning and Algorithmic Descriptions." AEA RCT Registry. May 03. https://doi.org/10.1257/rct.11349-1.0
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2023-05-03
Intervention End Date
2024-12-31

Primary Outcomes

Primary Outcomes (end points)
The main outcomes of interest are the following:
1. In DA18, IIDA6 and IIBOS6, subjects rankings over schools submitted in each round.
2. In DA1, DA3, IIBOS1, and IIBOS2, the partial rankings over schools submitted every time the subject is asked for it, in each round.
3. The predicted assignments for each round.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We conduct a matching experiment based on 6 experimental treatments that differ in matching rule and the mechanism used to implement them. They can be described in two phases.

Experiment Phase 1: Perfect information Gale-Shapley DA
We will have three different treatments: DA1, DA3 and DA18. There will be 60 subjects per treatment. All treatments will share the following features:
• The experiment will consist of 20 rounds.
• In each round, a new market consisting of 18 students and 18 schools is created. Each school has a unit capacity. The experimental subject will take the role of student S1, while S2,…,S18 are robot players.
• Students’ payoffs over schools and schools’ priorities over students were created such that under truth-telling, student S1 will be matched to his/her 1st, 3rd, 5th, and 7th highest-paying alternative in five out of the twenty markets each.
• There are four types of schools and four types of students (other than S1). Students/schools of the same type share the same payoff/priority ranking. Student S1 has a payoff ranking different from the other students.
• Subjects will be shown the entire payoff and priority table and will be told that all robot players will rank/choose schools in decreasing order of payoff value.
• At the beginning of each round, subjects will be asked to provide two schools that represent their prediction about which school they will end up matched to.
• Their payoff will consist of the average of two randomly drawn rounds for their assigned schools (which pay between CNY 10 and CNY 180) and two randomly drawn rounds for their predicted matchings (which pay CNY 10).

For DA18, we will ask subjects to submit a full ranking over all 18 schools in each round. The outcome will be the result of the Gale-Shapley DA mechanism with the ranking submitted by the subject as the preference for S1, and the preferences created for students S2,…,S18 as their submissions.

For DA1 and DA3, in each round the subject will act as student S1 in two different instances of the Iterative Deferred Acceptance Mechanism (Bó and Hakimov, 2022) in which every time the subject is asked to act, he/she will submit a choice of a single school (DA1) or a ranking of 3 schools among the ones that did not yet reject him/her (DA3). No feedback other than the fact that the subject was rejected from all previously ranked schools will be given.

Experiment phase 2: Imperfect Information Boston
We will have three different treatments: IIBOS1, IIBOS2 and IIBOS6. There will be 60 subjects per treatment. Each treatment will share the following features:
• The experiment will consist of 20 rounds.
• In each round, a new market consisting of 5 students (S2, S3,…,S6) and 6 schools is randomly created, using a model in which a student s’s utility from being matched to school c is the sum of two values:
o A common value for school s, drawn from the uniform distribution in the range [1,1000].
o An idiosyncratic value, drawn for each student/school pair, drawn from the uniform distribution in the range [0,1000].
• In each round the preference of student S1 will be chosen so that the ranking of his/her outcome under truth-telling is equally distributed between 1,2,3,4 and 5. That is, there will be four markets where under truth-telling student S1 will be ranked to his/her [1,2,3,4,5]th option.
• Each school has a unit capacity. The experimental subject will take the role of student S1, while S2, S3,…,S6 are robot players.
• Subjects will be shown the priority table, will be told the way in which the other students’ preferences were generated, and will be told that all robot players will rank/choose schools in decreasing order of payoff value.
• At the beginning of each round, subjects will be asked to provide two schools that represent their prediction about which school they will end up matched to.
• Their payoff will consist of the average of two randomly drawn rounds for their assigned schools (which pay between CNY 30 and CNY 180) and two randomly drawn rounds for their predicted matchings (which pay CNY 20).

For IIBOS6, in each round we will ask subjects to submit a full ranking over all 6 schools. The outcome will be the result of the Boston mechanism with the ranking submitted by the subject as the preference for S1, and the preferences created for students S2, S3,…,S6 as their submissions.
For IIBOS1 and IIBOS2, in each round the subjects will act as student S1 in two different instances of a sequential version of the Boston Mechanism, as follows:

IIBOS1
• In the first step, every student is asked to choose a school.
• Schools that receive one or more application permanently accepts the highest-ranked one with respect to its priority, and reject the other students.
• Students who were rejected are asked to choose a school among those that did not yet reject him/her, and the procedure continues until every student is accepted at some school.

IIBOS2
• In the first step, every student is asked to submit a ranking with two schools.
• The Boston Mechanism is executed with the submitted rankings.
• Students who are matched to some school are permanently matched to those schools.
• If there are students left unmatched, these are asked to submit a new ranking of two among those that did not yet reject him/her. The Boston mechanism is executed with the preferences submitted by these students in this second step, and where the schools that were matched to some student in the previous step have zero capacity. Students who are matched to some school are permanently matched to those schools.
• If there are students left unmatched, they are asked to submit a new ranking of two among those that did not yet reject him/her. The Boston mechanism is executed with the preferences submitted by these students in this third step, and where the schools that were matched to some student in the previous steps have zero capacity. Students who are matched to some schools are permanently matched to those schools.

Subjects’ Belief elicitation
Before each round in every treatment, the subjects will be asked for a prediction about which school they will end up matched in that round. For DA1, DA3 and DA18, they will have the opportunity of providing two schools. For the other treatments, only one school. If the outcome in that round is one of the schools predicted, the subject will earn an additional CNY 10 (resp. 20 in the Boston) in that round. (Notice that, as the payoffs for the main task, the average of two randomly drawn rounds will be used for the prediction payoff).



Experimental Design Details
Not available
Randomization Method
Otree based randomisation.
Randomization Unit
Experimental subjects are randomly assigned to one experimental treatment.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
No clustering is necessary as each subject will be an independent observation.
Sample size: planned number of observations
20 rounds per subject, 60 subjects per treatment. 1200 (20 rounds x 60 subjects) observations per treatment. This results in 7200 observations (6 treatments) in total.
Sample size (or number of clusters) by treatment arms
We are recruiting 60 Subjects per treatment who will make decisions over 20 experimental rounds each. We are conducting 6 treatments, resulting in a total of 360 subjects (6 treatments x 60 subjects).
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Research ethics committee of the China Center for Behavioral Economics and Finance
IRB Approval Date
2023-05-01
IRB Approval Number
2023_004
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information