Understanding the Effect of Test-Optional Admissions Policies on Admissions

Last registered on August 25, 2022
View Trial History

Pre-Trial

Trial Information

General Information

Title
Understanding the Effect of Test-Optional Admissions Policies on Admissions
RCT ID
AEARCTR-0009777
Initial registration date
August 24, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
August 25, 2022, 2:22 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
United States of America
Region
Northeast

Primary Investigator

Name
Zachary Bleemer
Affiliation
Yale University
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
John Friedman
PI Affiliation
Brown University
Contact Investigator

Additional Trial Information

Status
In development
Start date
2022-08-29
End date
2024-07-01
Keywords
Education
Additional Keywords
JEL code(s)
I23, I24
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
In recent years, many colleges and universities are reconsidering their use of standardized test scores in their admissions processes. The debate has gained new urgency in recent years after COVID-19 forced most schools to adopt "test-optional" admissions policies. This study implements an experiment in which a selective northeast university's admissions officers re-evaluate real applications from past years with altered or omitted standardized test scores to determine how the presence or absence of test scores affects admissions decisions. The results from this experiment will answer the following questions: (1) How does a test-optional policy affect the way that admissions officers evaluate applications? (2) How do these effects vary across students, both by level of academic preparation and student background? In combination with descriptive results from other sources, these findings will characterize the winners and losers from test-optional policies and reveal the functional role of standardized tests in selective university admissions.
External Link(s)

Registration Citation

Citation
Bleemer, Zachary and John Friedman. 2022. "Understanding the Effect of Test-Optional Admissions Policies on Admissions." AEA RCT Registry. August 25. https://doi.org/10.1257/rct.9777-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
The admissions office of a highly-selective private university in the northeast that currently implements a test-optional admissions policy will provide complete authentic undergraduate applications to its actual application readers. Each application will be provided to multiple readers, but with certain alterations: some will be provided with SAT scores above those actually received by the applicant, others with SAT scores lower than those actually received, and some will be provided applications with no SAT score at all, as if the applicant had chosen against submitting a score. Application readers will assign standard application scores to each application and record admission recommendations.
Intervention Start Date
2022-08-29
Intervention End Date
2022-09-30

Primary Outcomes

Primary Outcomes (end points)
The primary outcomes that we will measure for each application are:
1. An indicator for the applicant's being recommended to committee (a proxy for recommending admission).
2. An indicator for whether the applicant was recommended to committee or waitlist (as opposed to rejection).
3. The separate academic and non-academic reader scores assigned by the application readers.
4. An indicator for whether the applicant received a reader score (or each type separately) of four or greater.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
See the pre-analysis plan for an a more-detailed summary of additional outcomes.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
See the attached Experimental Protocol for our experimental design.
Experimental Design Details
Not available
Randomization Method
Randomization will be conducted in reseachers' office by a computer using Stata.
Randomization Unit
Randomization will be conducted first at the geography level (choosing ten 'super-geos' by which the university's admissions readers evaluate applications), and then at the individual level with geography.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
20 application readers will evaluate applications.
Sample size: planned number of observations
About 500 applications will be evaluated.
Sample size (or number of clusters) by treatment arms
Each application will be evaluated four times under the test-optional regime, in four treatment arms: large increase from actual standardized test score, small change in score, large decrease in score, and omission of the test score. Half of the selected applications will be evaluated four additional times under the test-mandatory regime, replacing the fourth arm with a chosen test score (see the experimental protocol).
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials

Documents

Document Name
Experimental Protocol
Document Type
other
Document Description
File
Experimental Protocol

MD5: 0a3e7dfeb661d6ae79801f8475a870d4

SHA1: cb4dcbdea031f94836fe4c29fde40a02386a646c

Uploaded At: August 24, 2022

IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information