AI Recommendations for Centralized University Applications

2026-04-20

2026-05-01

The study will include the following outcome variables:
A. Recommendation Use and Exploration
• Recommendation Adoption (Binary): Indicates whether the student reports that they would list one or more of the programs recommended by the platform rather than retain only their initial program choices.
• Exploration Intensity (Numerical): Measures the extent of student interaction with recommendations, such as number of recommendations viewed and whether the student chooses to see 5 additional recommendations.
• Exploration Breadth (Numerical): Captures how broadly the student explored across different fields of study, institutions, or geographic locations.
• Recommendation Consideration (Binary / Numerical): For each recommended program, indicates whether the student clicks that they would consider listing it.
• Recommendation Rejection (Binary / Numerical): For each recommended program, indicates whether the student clicks that they would not consider listing it.
• Choice-Set Update (Binary): Indicates whether the student changes any of the main degree programs selected after interacting with the app.
• Top-Choice Update (Binary): Indicates whether the student’s stated top choice changes after interacting with the app.
• Preference-List Revision Intent (Numerical): Measures how likely the student reports they are to change some of the choices in their final application before submission.
• Additional-Recommendation Click-Through (Binary): Indicates whether the student elects to view the optional 5 additional recommendations.
B. Baseline Preferences and Stated Choices
• Initial Main Choice Set (Categorical): The set of up to five degree programs the student selects before interacting with the app.
• Initial Top Choice (Categorical): The program the student identifies as their current top choice before interacting with the app.
• Post-Interaction Main Choice Set (Categorical): The set of up to five degree programs the student selects after interacting with the app.
• Post-Interaction Top Choice (Categorical): The program the student identifies as their top choice after interacting with the app.

C. Knowledge and Awareness
• Program Knowledge (Numerical): Assesses the student’s self-reported knowledge of available college programs after using the platform.
• Awareness of Available Options (Numerical): Measures whether the student reports being aware of the different degree program options available to them.
• Perceived Information Gain (Numerical): Measures whether the student reports that the app provided information about some programs that they did not already know.
• Program Discovery (Numerical): Measures whether the student reports that the app introduced them to programs they had not previously considered.
D. Trust, Understanding, and Goal Alignment
• Trust in Intent (Numerical): Measures trust that the platform is acting in the student’s best interest.
• Trust in Competence (Numerical): Measures trust that the platform is capable of generating accurate and appropriate recommendations.
• Understanding of Rationale (Numerical): Measures the student’s self-assessed understanding of why the recommendation was generated.
• Compatibility of Goals (Numerical): Measures perceived alignment between the platform’s objectives and the student’s own goals when forming the recommendation.
E. Confidence, Satisfaction, and Motivation
• Application Readiness (Numerical): Measures whether the student feels they have enough information to complete or finalize their application list.
• Admission Confidence (Numerical): Measures how confident the student feels that they will be admitted to a program they would be happy to attend.
• Decision Uncertainty (Numerical): Measures the extent to which the student expects to revise their choices before submitting the final application.
• Satisfaction with Experience (Numerical): Measures overall satisfaction with the platform experience, including ease of use, perceived usefulness, and relevance of recommendations.
• Sense of Ambition (Numerical): Captures the extent to which the platform experience encourages the student to consider more demanding, competitive, or aspirational college programs.
F. Perceived Usefulness and Future Demand
• Future Use Interest (Binary): Indicates whether the student would like to have access to the app again before submitting their final applications.
• Open-Ended Evaluation (Text): Captures what the student found helpful or unhelpful about the app and what changes could make it more useful to future students.
• Application-Process Reflection (Text): Captures students’ open-ended reflections on how they are thinking about choosing which degree programs to list.
Some measures may be based on self-reported responses, while others may rely on anonymized interaction data generated during use of the platform.

Students will:
1. randomly split to control and treatment groups (in ratio 1:2),
2. access the platform,
3. enter or confirm basic profile information and preferences relevant to college applications,
4. receive a recommended college-application portfolio generated by the platform based on their profile, and
5. complete a brief questionnaire measuring perceptions of the recommendation and intended adoption.

To study how students weigh different recommendation trade-offs, the platform will use student responses to present recommendations that emphasize different criteria (or make these criteria more/less salient), such as location preferences, fit with personal goals, or admission probability.

Not available

We use a computer-based random number generator to randomly split to control and treatment groups (ratio 1:2).

Individual students is the unit of randomization.

No

N/A

We aim for 1100 students across 14-17 schools.

We aim for 1100 students across 14-17 schools, split into control and treatment groups in 1:2 ratio, so 367 students are expected to be in control and 733 in treatment if we have 1100 participating students in total.