An Experiment on the Cultural Interpretation of Meritocracy

Last registered on November 30, 2022

Pre-Trial

Trial Information

General Information

Title
An Experiment on the Cultural Interpretation of Meritocracy
RCT ID
AEARCTR-0010274
Initial registration date
November 16, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
November 30, 2022, 2:28 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region
Region

Primary Investigator

Affiliation
Paris School of Economics

Other Primary Investigator(s)

PI Affiliation
Paris School of Economics
PI Affiliation
Warwick University

Additional Trial Information

Status
In development
Start date
2022-11-22
End date
2023-04-30
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
While in Western developed countries people establish a firm link between their beliefs in the importance of merit (as opposed to luck and privilege) in the income generating process and the desired level of redistribution, this significant correlation is not observed in East Asian countries despite a tradition of meritocratic civil service exams with competitive selection and a heavy focus on education. We argue that the concept of meritocracy takes different forms in Asia versus in Europe. We make a distinction between "consequentialist" meritocracy and "deontological" meritocracy, with the former adopting meritocracy as a means of maximising total surplus and the latter aiming at distributing resources proportional to intrinsic merits. We predict that these two types of meritocracy interpretations would generate different distributional preferences, and empirically test it with incentivised spectator games with students from highly selective colleges of different cultures. Spectators are asked to make redistribution choices in a series of scenarios provided to them. We randomize the status-quo income distribution described in the scenarios, by assigning half of the spectators to a highly unequal status-quo distribution (winner takes all) and the other half to an almost equal status-quo distribution. We compare both the sums redistributed and the Gini coefficients implemented by spectators of different cultures, in order to disentangle preference for inequality from status-quo bias. In order to test our predictions of which meritocracy considerations Western (deontological meritocrats) and East Asian (consequentialist meritocrats) respondents would privilege when faced with ethical trade-offs, we compare the redistribution choices made by spectators in a baseline scenario where the initial income distribution is determined by performance on a task, with their redistribution choices in scenarios where ethical trade-offs arise.
External Link(s)

Registration Citation

Citation
Belguise, Margot, Yuchen Huang and Zhexun Mo. 2022. "An Experiment on the Cultural Interpretation of Meritocracy." AEA RCT Registry. November 30. https://doi.org/10.1257/rct.10274-1.0
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2022-11-22
Intervention End Date
2022-12-31

Primary Outcomes

Primary Outcomes (end points)
Our dependent variables are the sums redistributed by the spectators and the Gini coefficients the spectators implement.
Primary Outcomes (explanation)
In order to compare different cultures’ revealed fairness preferences, the focus of our analysis will be double-difference outcomes, which is the cross-culture times cross-scenario differences in the choices made by our respondents. These will be obtained as double-difference estimators in regression frameworks, where the sum redistributed or the Gini coefficients implemented will be regressed on scenario dummies interacted with culture dummies (controlling for a series of demographic controls).

We distinguish the sum redistributed and the Gini coefficients implemented by spectators because we randomise the status-quo distribution to which spectators are assigned, in order to distinguish preferences for inequality from a strong status-quo bias (when the status-quo distribution is very unequal).

To assess “meritocratic preferences”, we will compare choices made in a scenario where task performance (“merit”) determines who is the status-quo winner (merit scenario) with choices made in a scenario where a random draw determines who is the status-quo winner.

To compare how spectators of different cultures deal differently when faced with a trade-off between proportionally rewarding personal merit (meritocratic preference) and other considerations (efficiency concerns, inequality of opportunity, etc), we will compare choices made in the merit scenario to choices made in a series of other scenarios which present those trade-offs.

In addition to this, we also randomise the framing of one of our scenarios (with one scenario framed as 20% probability that the worker reported to be the winner is the “wrong winner” versus 20% probability that the worker to be the loser is the “wrong loser” ), in order to compare how spectators of different cultures react differently to those two framings of the same scenario.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
In this project, we plan to experimentally test a series of predictions on the differences in the distributional choices which deontological and consequentialist meritocrats would make. Specifically, we will ask our respondents to indicate how they want us to distribute bonus payments between two workers. We will randomise the content, framing and order of the scenarios that they will choose to elicit their true distributional preferences. Respondents’ choices will be actually implemented with given probabilities, as we will hire Amazon MTurk workers to solve effort-paying tasks and re-enact the scenarios described in the survey and will randomly draw the bonus schedule to be paid to the workers from the respondents’ answers.
Experimental Design Details
WHY THIS IS A RANDOMISED CONTROLLED TRIAL::

- To test for the hypothesis of a greater status-quo bias among East Asian respondents compared to Western respondents, we randomize the status-quo distribution described to respondents. Regressing, for each scenario, the Gini coefficient implemented by the respondent on a dummy for the status-quo distribution to which the respondent was assigned to (either being very unequal or very equal), interacted with a culture dummy, would thus make it possible for us to disentangle culturally-varying preferences for inequality from culturally-varying status-quo bias in a double-difference estimation framework. This is one of the key predictions we aim to test.

- To test for the hypothesis that East Asian respondents have a greater aversion to rewarding the "wrong winner", we randomise the framing of one of our scenarios, mentioning for 50% of the sample that there is a 20% probability that the software used would pick the "wrong winner" (who actually performed fewer tasks) and for the remaining 50% of our sample that there is a 20% probability that the software used would pick the "wrong loser". We again aim to compare the coefficients on a treatment dummy (corresponding to the framing used) interacted with a culture dummy with double-difference estimation.

- Although this is more minor since it is not at the core of our main predictions, we also randomize the order of some of our scenarios to control for the fact that the order of the scenarios may prime respondents for subsequent scenarios (as a robustness check, we will thus verify that the coefficients obtained are not statistically different depending on the order of the scenarios to which respondents were exposed).

In more details, the experiment will be held in two parts: respondents making redistributive decisions and the Amazon MTurk workers solving effort-paying tasks.

The respondents' part: The respondents will fill up a questionnaire. This questionnaire first asks for informed consent and lets respondents choose not to participate, excluding respondents who are younger than 18 years old. It then proceeds to ask respondents a series of questions to elicit their beliefs on the role of the state in redistribution, the role of personal effort and luck/privilege in determining success, individualism versus collectivism, perception of own success, social desirability bias and perception of own merit.

The questionnaire then proceeds to describe scenarios opposing two workers, indicate the way those two workers will be paid in the absence of redistribution, and ask the respondent to indicate whether he/she wishes the researchers to distribute rewards differently between those two workers. Those scenarios include:

1) Whether the source of an unequal original distribution is random luck or the numbers of tasks completed
2) One of the workers is randomly selected to have a training before the task and completed more tasks.
3) One of the workers is randomly selected to have a hindrance during the task and completed less tasks.
4) A trade-off where redistributing to the worker who completed less (thus less deserving) has a benefit (it increases the total rewards to be divided between the two workers).
5) A “wrong” winner or loser (respectively completed fewer or more tasks) is picked with some probability ; 50% of the sample will be assigned to a framing making the "wrong winner" idea salient, while the rest will be assigned to a framing making the "wrong loser" salient.
6) One of the workers did better, but only marginally.

Respondents will be randomly separated into two different status-quo scenarios: .for 50% of respondents, the distribution of rewards between the two workers in the absence of redistribution will be a 12-0 split, while for the remaining 50% of the sample, it will be a 7-5 split to test for a culturally-varying status quo bias.

Workers' information:
The workers will be asked to complete some data entry task for which they will need to search some information on Wikipedia (they will be asked to collect the date of birth and of death of a series of famous individuals which has already been compiled by Gergaud et al (2016), making it possible to verify whether workers correctly completed the tasks, as well as additional information such as parental occupation of those famous individuals and the word count of the English language Wikipedia article on the famous person. After reading the consent form – for which they can take their time, and this will be ensured by a using a timer so that the button to submit answers to the consent form only appear after enough time has been given to them -, the workers will be given 30 minutes to complete as many tasks as possible.

Each worker will be paid a participation fee consistent with the US federal minimum wage given task length. In addition to this participation fee, the workers will be paid rewards up to three times as large as the participation fee, determined by the responses to the questionnaire. When making redistribution choices, survey respondents will thus be asked to indicate how they wish that those rewards be divided between the two workers described in the scenarios.

The workers will be told that the bonus might depend on how many tasks they correctly perform, but not precisely how: an initial payment is decided given their performance, or given the outcome of a random lottery, and then the final amount they will receive is decided by a third-person spectator who can choose to redistribute between them. They will also be told that they could be randomly selected to receive a short tutoring before the tasks or required to complete a small irrelevant task during the 30 minutes allocated to the assessed tasks, which might affect their final numbers of tasks completed (either positively or negatively), and the third-person spectator will know about their circumstances. The tasks to be performed will be the same for all workers apart from this.

The tasks will be posted to Amazon MTurk and individual Amazon MTurk workers will be able to sign up for the tasks. They will be informed explicitly before the task about their payment: they will first receive a base compensation corresponding to the minimum wage requirement given the task's length, and then will receive a bonus that is decided by a third-person spectator.

No personal information will be collected from the workers at all apart from their publicly available amazon worker ID, which we will use only for the purpose of paying bonuses.
Randomization Method
The randomisation is done by the inbuilt function of QUALTRICS system for the French sample and by a Chinese survey company for the Chinese sample.
Randomization Unit
Treatments (status-quo income distribution, framing of the wrong winner / wrong loser scenario, as well as the order of scenarios) will be randomised at the individual level.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
There are three sample groups: undergraduate students in France who specialised in Europeans studies in a given university campus, undergraduate students in France who specialised in Euro-Asian studies in another separate university campus, and undergraduate students in China from a randomly drawn online sample.
Sample size: planned number of observations
The French sample is expected to be around 300 to 500 students (depending on the take-up rate of the survey as we will be distributing our surveys among undergraduate college students across two campuses of the same university). The Chinese college student sample is fixed to be 400.
Sample size (or number of clusters) by treatment arms
Our main randomization strategies, such as the randomization into different status-quo income distributions and the randomization into different framings of the scenarios on winning/losing, are at the same level (not a two-stage randomization), as such essentially we will be splitting our sample into two halves, with 200 individuals per treatment arm for the Chinese sample. Treatment arm sizes for the French sample will depend on the take-up rate of our survey, but we expect to arrive at similar treatment group sizes there as the Chinese sample.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Following Almås et al. (2020), we assume that the French sample will implement a Gini coefficient of 0.25 with a standard deviation of about .998 when the difference in income is entirely induced by luck. Assuming all the students participating, we would reach 80% power if the size of the detectable effect is above 0.3 (in Gini coefficient) for Asian students and 0.26 (in Gini coefficient) for American students.
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
Paris School of Economics Institutional Review Board
IRB Approval Date
2022-11-15
IRB Approval Number
2022-26
IRB Name
University of Warwick - Economics Research Ethics Panel
IRB Approval Date
2022-11-17
IRB Approval Number
N/A

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials