The Impact of GREs on admission decisions for graduate programs in public health

Last registered on February 08, 2021

Pre-Trial

Trial Information

General Information

Title
The Impact of GREs on admission decisions for graduate programs in public health
RCT ID
AEARCTR-0007139
Initial registration date
February 08, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
February 08, 2021, 11:58 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
CUNY School of Public Health

Other Primary Investigator(s)

PI Affiliation
CUNY School of Public Health

Additional Trial Information

Status
On going
Start date
2021-02-01
End date
2021-06-01
Secondary IDs
Abstract
The impact of requiring the GREs on student admissions decisions has long been a point of discussion in the CUNY Graduate School of Public Health. Most graduate programs in the school have decided not to require the GREs any longer. However, GREs are still required for applicants to the MPH in Epidemiology and Biostatistics program as well as for the PhD Program in all degree concentrations. These programs that are still requiring the GREs have decided to assess what impact the GREs have on admissions decisions. For this study, we are conducting a randomized controlled trial, with randomization scheme and review methods varying by program (MPH versus doctoral). For the MPH program, faculty members who volunteer to review program applications will be randomly assigned to one of two groups, one group will have applicant GRE scores visible for review (Group A)) and one group will not be able to see the GRE scores (Group B). Applications will be assigned to reviewers in batches, with each applicant being randomly assigned to one reviewer in each group, so each application will
be reviewed by two faculty members, one in group A (with GREs) and one in group B (without GREs). Faculty members will be re-randomized to group A or B with each new batch of applications ready for review. For Doctoral Program applications, all applications will be randomly assigned to two reviewers, one with GREs visible and one without. The order of review will also be randomized to avoid bias if, for example, a reviewer decided to first review all applications with the GREs and then all those without. We will then use risk ratio regression, stratified on program (MPH or Doctoral) with GEE to adjust for clustering (same applicant reviewed by two people) to determine what impact viewing the GREs has on admission decisions (admit or reject in the case of the MPH Program and decision to interview or not in the case of the PhD Program). In order to ensure fairness in the application process, for MPH program applications, in cases in which the two reviewers disagree on the admissions decision, the application will be reviewed by a third reviewer with the GREs visible (the current admissions standard) and the final admissions decision will be based on the two reviewers who viewed the GRE scores. If they disagree based on their application
reviews, they will discuss their decisions and come to a consensus decision, as is standard. For doctoral program applications, when the two reviewers disagree the entire Department Doctoral Program Admissions Committee will review the application with the GRE score visible and come to a consensus, which is the traditional procedure outside of the GRE evaluation. These additional reviews for application with discrepant admissions decisions will be conducted outside of the experiment (i.e. their decisions will not be part of the analysis). For both study populations (MPH in Epidemiology and Biostatistics and PhD Program), we will also explore applicant characteristics such as
race/ethnicity, gender and age as possible modifiers of the impact viewing the GREs has on admissions decisions.
External Link(s)

Registration Citation

Citation
Jones, Heidi and Elizabeth Kelvin. 2021. "The Impact of GREs on admission decisions for graduate programs in public health." AEA RCT Registry. February 08. https://doi.org/10.1257/rct.7139
Experimental Details

Interventions

Intervention(s)
The intervention is blinding admissions reviewers to GRE scores for the MPH program in Epidemiology & Biostatistics and the PhD program in Public Health (Epidemiology, Community Health and Health Policy, and Environmental and Planetary Health Sciences).
Intervention Start Date
2021-02-01
Intervention End Date
2021-06-01

Primary Outcomes

Primary Outcomes (end points)
Admissions decision (entry/no entry for MPH program; interview/no interview for PhD program)
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Time taken per each applicant review
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Reviewers are randomized to receive applications with GRE scores visible and those without GRE scores as part of the admissions process. Randomization will be done differently for the MPH program, which uses rolling admissions, so that reviewers will be randomized in batches to be blinded/unblinded throughoug the rolling admissions process. For the doctoral program, with a finite set of applicants, reviewers will receive approximately half of their applications with GRE scores and half without (randomly assigned so that each application is reviewed by one faculty member in a blinded and one in an un-blinded fashion). Additionally, in the doctoral program, the order of the review will be randomized to avoid batch effects.
Experimental Design Details
Randomization Method
Randomization in office by a computer
Randomization Unit
Unit of randomization is the reviewer for each applicant.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
1 MPH program, 1 PhD program (three Departments) - total number of reviewers=47 faculty.
Sample size: planned number of observations
approximately 600 applications across the programs/degrees
Sample size (or number of clusters) by treatment arms
approximately 150 in the doctoral program, 450 in the MPH program
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Sample size is determined by the number of applicants to the programs. Statistical analysis will include calculation of 95% confidence intervals to put the precision of the findings into context.
IRB

Institutional Review Boards (IRBs)

IRB Name
City University of New York IRB
IRB Approval Date
2021-02-01
IRB Approval Number
2020-0764

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials