Information Aversion in Admission Decisions

Last registered on December 21, 2023

View Trial History

Pre-Trial

Trial Information

General Information

Title

Information Aversion in Admission Decisions

RCT ID

AEARCTR-0012709

Initial registration date

December 16, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

December 21, 2023, 7:53 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country

United States of America

Region

Primary Investigator

Name

Yucheng Liang

Affiliation

Carnegie Mellon University

Contact Primary Investigator

Other Primary Investigator(s)

PI Name

Wenzhuo Xu

PI Affiliation

Carnegie Mellon University

Additional Trial Information

Status

In development

Start date

2023-12-18

End date

2024-03-31

Keywords

Behavior

Additional Keywords

JEL code(s)

Secondary IDs

Prior work

This trial is based on or builds upon one or more prior RCTs.

Abstract

This study investigates the demand and use of test scores in admission decisions.

External Link(s)

Registration Citation

Citation

Liang, Yucheng and Wenzhuo Xu. 2023. "Information Aversion in Admission Decisions." AEA RCT Registry. December 21. https://doi.org/10.1257/rct.12709-1.0

Sponsors & Partners

Experimental Details

Interventions

Intervention(s)

Intervention Start Date

2023-12-18

Intervention End Date

2024-03-31

Primary Outcomes

Primary Outcomes (end points)

Demand for test scores, admission decisions with and without test scores

Primary Outcomes (explanation)

Demand for test scores: participants choose whether they see a test score before making admission decisions
Admission decisions: participants admit 3 students from each group of 8. Some decisions are made with a test score, some without

Secondary Outcomes

Secondary Outcomes (end points)

Free-form reasoning for demand for test scores, advice to other participants about demand for test scores, decision time

Secondary Outcomes (explanation)

Free-form reasoning for demand for test scores: participants write about the reasons for their demand for test scores
Advice to other participants about demand for test scores: participants advise other participants about demand for test scores. They can write additional text to support their advice.
Decision time: decision time for the demand for test scores and admission decisions

Experimental Design

In the experiments, participants admit three students from each group of eight. Before that, they decide whether to see the students' test scores.

Experimental Design Details

Main Treatment:

Introduction and Understanding Check: Participants are welcomed and informed about the importance of their responses. They are required to pass understanding checks throughout the survey to proceed.

Context and Task Description: Participants learn they will be evaluating groups of students (some real, some fictional) who have completed a basic data analysis course. Their task is to select students for an advanced course.

Report Card Information: Each student's report card includes family income level and Test 1 score, with possible inclusion of Test 2 score. Family income is categorized into higher and lower income based on an income threshold. Test details, including scoring and preparation differences between income groups, are provided.

Test Prep Scenarios and Demand for Test 2 Scores: No student received test prep for Test 1. Participants are presented with various scenarios regarding test preparation for Test 2, which include no prep, test-specific prep for higher-income students, test-specific prep for all students, and general prep for higher-income students. They must decide whether to include Test 2 scores in their decision-making for some of the scenarios.

Admission Decisions: One test prep scenario is realized. Participants make admission decisions for several groups of students based on the information provided in the report cards. These decisions are influenced by the participant's earlier choice about including Test 2 scores in that test prep scenario.

Second elicitation of demand for Test 2 Scores: Participants are given a chance to confirm or revise their demand for Test 2 Scores, which may affect their final admission decision.

Advice for Future Participants: Participants are asked to provide advice for future participants based on the test prep situation they encountered and provide reasons for it.

Opinions on Education Policies: The survey concludes with questions about the participant’s views on test-blind and test-optional admission policies in higher education.

Demographic Information and Feedback: Finally, participants are asked to provide demographic information.

Three additional treatments only elicit demand for Test 2 scores under the no prep and test-specific prep for higher-income students scenarios.

Identity Blind Treatment: This treatment differs from the Main Treatment in that students' family income levels are not observable by the participants.

Noisy Test 1 Treatment: This treatment differs from the Main Treatment in that participants are informed that some unknown students are graded more leniently than others for Test 1.

Performance Prediction Treatment: This treatment differs from the Main Treatment in that participants are asked to select the top three performers in the advanced data analysis course. They are rewarded for their prediction accuracy, but the students are not affected by their predictions in any way.

Randomization Method

Done by qualtric's randomizer

Randomization Unit

Individual

Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters

900 participants

Sample size: planned number of observations

900 participants

Sample size (or number of clusters) by treatment arms

Main Treatment: 600. Everyone answers the demand question under the no prep scenario. 200 participants answer the demand question for each of the other three test prep scenarios. 150 participants make admission decisions under each of the four test prep scenarios.
Identity Blind Treatment: 100
Noisy Test 1 Treatment: 100
Performance Prediction Treatment: 100

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

Supporting Documents and Materials

IRB