The labor market implications of grade inflation - manager experiment

2025-07-18

2025-07-31

Incentivized scores and wages

The experiment will be conducted online (using Connect or Prolific) and will consist of two components, each lasting about 15 minutes, run concurrently (i.e., within a week of each other). In both components, participants are told to imagine themselves in the role of a manager looking to hire workers for their firm where underlying ability on the job is measured by the math portion of the SAT exam. Their task is to evaluate the profiles of and set wages for a group of candidates who participated in a separate study.

The manager experiment consists of participants first reporting their priors about how scores on the math SAT are distributed in the candidate population. Managers learn that the candidates have taken a math test and that, consistent with college transcripts, they will see the letter grades candidates received on the test, but not the raw scores. After seeing three sample questions from the test, managers are asked for their expectations about how hard the test was (i.e., how well candidates have done).

Managers then learn that they will be randomly assigned to one of three grading treatments: a control in which a college professor scores the tests and assigns letter grades from C- to A+ (i.e., 9 grading bins), compress where a professor scores the tests but only assigns grades from B- to B+ (3 bins) and inflate where a professor only assigns grades from A- to A+ (also 3 bins).

As a signal extraction exercise, managers are first asked to predict the grading scheme used by the professor in the treatment to which they are assigned. Specifically, they are asked to report the lowest numerical score (from 0 to 100 percent) needed to receive each letter grade. Second, they are randomly shown 9 of the 18 possible candidate profiles (which include the letter grades received on the test) and asked to assign a numerical score between 0 and 100 to each. Managers are incentivized to report scores that match the true test performance of the candidates. These scores are considered ability \emph{signals} in the analysis described below.

In the last part of the experiment, managers see expanded profiles for the same 9 candidates and are incentivized to assign wages that match their ability (i.e., math SAT scores). Here, the profiles include the math SAT priors the candidates reported first and the letter grades each candidate received on the test to provide managers with the information a Bayesian would require to set wages optimally.

Managers then answer a few demographic questions before being redirected back to the platform for payment. They receive a fixed payment for completing the experiment and can earn a bonus for reporting correct priors about the average math SAT scores of the candidates, their estimates of the grading schemes and performance on the math test, assigning scores that match the actual performance of candidates on the test and assigning wages that match individual candidate math SAT scores and their variance estimates.

Managers in both components of the experiment know that all the candidates are between 21 and 25 years old, have recently graduated from a four-year college/university (or are enrolled in one) and have taken the SAT exam. The only difference between the first component of the experiment and the second is that candidate gender is also revealed in the first component, and it is not revealed in the second component.

In office by computer

Experimental participant

No

between 800 and 2100 participants

Each participant evaluates nine candidate profiles each so between 7200 and 18900.

The sample will be divided equally among the three treatment conditions.