Salary Disclosure in Job Ads

I run an experiment on an online job search platform in which treated ads are induced to post salary ranges in job ads, while control ads have a choice over whether to disclose salary ranges (as is status quo).

2024-06-10

2024-11-11

Firms may respond to treatment by adjusting reported salaries, while workers may respond by adjusting applications. Thus, key outcomes at the job ad level are:

i. Log min salary of ads,
ii. Log max salary of ads,
iii. Size of the salary range (difference between min and max salary divided by median salary),
iv. Number of male applications,
v. Number of female applications

I am especially interested in testing two main hypotheses:

1. In response to treatment, workers may reallocate search towards jobs for which learning about salaries was previously harder, but that are ex-post revealed to pay well. A key example of this are jobs with larger firms – administrative data reveals that they hide salary more, but that they also pay better than smaller firms for similar jobs. I will mainly explore heterogeneity by firm size, and use data driven approaches on baseline administrative data for defining other relevant job and firm characteristics that may plausibly drive substitution patterns in the specific way described here.

2. The effects of treatment may be especially pronounced for women who tend to have weaker professional networks and labor market exposure. Thus, they may be less able to extract wage signals from job descriptions. Alternatively, they may prefer knowing the bargaining space before entering a negotiation process. Thus, primary and secondary job search outcomes are separated by gender

To test for these hypotheses, I will run two specifications:
y_{jt}=\alpha_1 +\beta_1 T_j+X_jt\gamma+\epsilon_{jt}
for job j and time t where T_j is a treatment indicator, and X_j are controls for job characteristics and \beta_1 is the coefficient of interest
y_{jt}=\alpha_2 + \beta_2 T_jH_j + \lambda H_j + X_{jt}\gamma + \epsilon_{jt}
where H_j is the margin of relevant heterogeneity.

Secondary outcomes:
i. Unique number of views by men on the ad
ii. Unique number of views by women on the ad
iii. Share of male applicants with current salary above max salary of the ad
iv. Share of male applicants with expected salary above max salary of the ad
v. Share of male applicants with current salary below min salary of the ad
vi. Share of male applicants with expected salary below min salary of the ad
vii. Share of female applicants with current salary above max salary of the ad
viii. Share of female applicants with expected salary above max salary of the ad
ix. Share of female applicants with current salary below min salary of the ad
x. Share of female applicants with expected salary below min salary of the ad
Cluster level outcomes xi-xiii (to be regressed on whether the cluster is high or low treatment intensity); definition of clusters is described in the “experimental design” section
xi. Average (log) reported min salaries of job ads in the cluster
xii. Average (log) reported max salaries of job ads in the cluster
xiii. Average width of salary ranges in clusters (average at cluster level of [ max salary - min salary ] / median salary)
xiv. Share of female applications in the applicant pool of a job
xv. An index of: Share of applicants that meet experience requirement of the ad; Share of applicants that meet skills requirement of the ad; Share of applicants that meet education requirement of the ad; Share of applicants that match the career level of the job
xvi. Share of applicants who meet all requirements of the ad OR Average share of requirements met by applicants to the ad – I will choose whichever of these two variables allow for more variation, depending on the distribution. I will not need to use this outcome if requirements don't extend beyond the ones covered in xv.

Because male outcomes (iii) and (iv), and (v) and (vi) are related, and so are the corresponding female outcomes vii-viii and ix-x, I will do corrections for multiple hypothesis testing across them

I will additionally conduct the following exploratory analysis:

1. Assessing the extent of spillovers

Control ads may respond to the behavior of treatment ads, rather than just their own treatment status. Applicants may also redirect search from control to treatment ads. Thus, measuring potential spillovers is important.

i. Spillovers may increase with the share of other ads treated. Let Z be a dummy that indicates that the job was in a high saturation unit. I can then estimate the following equation:
y_jt= \alpha_1+\beta T_j + \phi Z_j + X_jt \gamma + u_jt
where \beta gives the treatment effect on job j, holding share of other jobs treated constant, and errors are clustered at the cluster level (details on how this is constructed are in the experimental design section).

ii. Treatment may interact with the treatment status of other ads - e.g., treatment effects may be lower where more ads reveal salary
y_jt= \alpha_1+\beta_h Z_j T_j + \beta_l T_j+ \phi Z_j + X_jt \gamma + u_jt
then \beta_l measures the effect of being a treatment ad in a low saturation market. \phi gives the effect of being a control ad in a high (h) saturation market relative to a control ad in a low (l) saturation market. \beta_h measures the effect of being assigned to treatment in a high saturation market relative to being in the control group in the same market or being a treatment ad in a low (l) saturation market. Errors will be clustered at the labor market cluster level

Test of spillovers:
\phi=0 i.e., no effect on control ads in high saturation group relative to low saturation group
\beta_h=0 i.e., treatment effect does not vary with share of ads treated

iii. Spillovers may extend beyond clusters of the labor market I generated. I will generate a continuous measure of distance d_{-j} between job j and all other job ads (-j)​. To do this, I will compare job titles and construct a matrix containing pairwise distances between job title j and -j using text analysis tools. Job titles that are more similar are more likely to compete. I will then construct donuts with all ads in radii of distance (d-x) around job j to estimate:
y_jt = \alpha + \beta T_j + \sum_{d=x}^D \lambda_d \frac{T_{-j}}{N} + X_jt \gamma + u_jt
where \beta reveals own treatment effect, holding constant the share of ads around job j that are treated in a given radius. To estimate a relevant D, I will estimate a series of nested models where d increases in increments of x and then select the one that minimized the Schwarz Bayesian Information Criterion (BIC) as in Egger et al. (2022)

2. Heterogeneous treatment effects on:
i. More senior career level of the job
ii. More complex job (as characterized by the number of skills required for the job)
iii. Higher education requirements of the job
iv. Higher experience requirements of the job
v. Firm’s history of salary disclosure (for firms that exist in baseline administrative data)
vi. Baseline firm market share of vacancies and applications (for firms that exist in baseline administrative data)
vii. Job title of firm representative who posts the ad (split into the categories: CEO/general management, HR personnel, Department-specific personnel)
viii. Whether the employer expressed a gender preference for candidates in the ad

3. Data source: I will request URLs to LinkedIn profiles from workers to capture their hiring outcomes and social connections by web scraping. I hope to use employment information as hiring outcomes, and social networks information as a margin of worker heterogeneity, provided the response rate is high and that a sufficient number of profiles are public.

I account for potential spillovers by implementing a two-step randomization process: the national labor market is partitioned into mostly self-contained segments (”clusters”), and these segments are randomly assigned to high (75%) vs. low (25%) treatment intensity. Within clusters, randomly chosen ads are then proportionally assigned to treatment or control. Labor market segments are defined on the basis of their occupation-industry classification, and these occupation-industry cells are further collapsed if they – based on historical data from the portal – shared at least 10% of their applicant pool with each other. Assignment of clusters to high and low intensity groups is stratified by cluster size (number of job ads they contained in historical data) such that about 50% of ads on average should be treated (assuming distribution of ads across clusters during the experiment is similar to that of baseline administrative data).

The experiment is initiated when employers start filling out a form to post a job on the platform. Currently, it is mandatory for firms to report the minimum and maximum salary for a job to the job platform but may choose to hide it from the job seekers by checking a box to “Hide the salary range from appearing on your job post” - an option which is available next to the salary range fields. This box is unchecked by default, i.e., salaries are visible by default.

In the experiment, once firms report their industry, and the occupation category of the job, a random number from 0 to 100 will be generated at the backend and firms will be assigned to treatment if the random number is less than or equal to X where X equals 25 for low treatment intensity clusters and 75 for high treatment intensity clusters.

Treatment ads will not be shown the option to hide the salary range. i.e., for them, the option “Hide the salary range from appearing on your job post” will never appear.
Instead, under the salary range fields, they will be displayed the following text:


“To find you the best match, the salary for this job will be displayed in the ad, as part of a reform. Click [here] to learn more.”

Clicking “learn more” generates a pop-up that describes the objectives of the study in more detail, and explains that salary ranges for select positions will be randomly chosen to be disclosed to jobseekers for the duration of the study. If firms have any concerns or questions, they are provided the email and phone number of the support staff of the platform as well as the PI of the study for getting in touch.

When treated the firms enter the salary range in the salary fields, a “Confirmation” pop-up will also appear to remind firms:


“Please be advised that the salary for your posted job will be displayed in the job details for potential candidates. [ Confirm ] ”


where they can only proceed if they press the “Confirm” button to confirm that they have understood that their salary will be posted publicly.

Firms that have queries or reservations may get in touch with the platform or the PI via the contact details provided in the pop-up described above. Upon contact (which they’re more likely to establish with the platform staff that they’re more familiar with than the PI), they will be informed about the objectives of the study as described in the “learn more” pop-up above. Treated firms who wish to keep their salaries hidden at this stage may ask to be excluded from the experiment. They will be allowed to override the experiment and hide the salary but face a higher time cost to doing so than in status quo, i.e., they call a support staff member at the platform, hear the pitch for the experiment, inform the support staff member that they understand the objectives but still prefer not to post salaries, and wait before they can post the ad for the support staff to notify the platform’s development team, and in turn for the development team to implement this exception. Firms will be asked if they wish to be granted the exception for this job post alone, or for all future posts, and their request will be implemented accordingly. These firms or ads will be considered non-compliers in the treatment group in my analysis.

Not available

Clusters are randomly pre-assigned a treatment intensity using STATA. Ads are randomly assigned to treatment or control by the job platform using an automated algorithm that takes as inputs the industry and occupation of the new job ad being created, and maps it to the relevant cluster and corresponding treatment intensity. It then generates a random number between 0 and 100 and assigns the add to treatment if the random number is less than X where X is either 25 or 75, depending on whether the ad is in a high or low treatment intensity cluster

Job ads

Yes

Up to 475

A minimum of 14,000 ads expected to be collected over 5 months

Due to the saturation design, all clusters have treatment and control ads within them. Clusters are randomly split across high and low intensity treatment such that on average about half of the total number of job ads are treated (i.e., induced to reveal salary). This resulted in up to 168 clusters in low and 307 clusters in high treatment intensity.