Homophily in Advice Seeking

Last registered on December 10, 2021

Pre-Trial

Trial Information

General Information

Title
Homophily in Advice Seeking
RCT ID
AEARCTR-0008674
Initial registration date
December 07, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
December 10, 2021, 11:03 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
University of Chicago

Other Primary Investigator(s)

PI Affiliation
UCLA

Additional Trial Information

Status
On going
Start date
2021-11-12
End date
2022-02-12
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This paper investigates homophily in advice seeking. Using administrative data from an online college student mentoring platform, we document that female students are 28 percent more likely to reach out to female mentors relative to male students, conditional on various observable characteristics. This preference may be driven by differences in the advice given by female mentors, or it may result from female students using mentor gender as a proxy for other unobservable mentor characteristics. If this preference persists even when female mentors are scarce or have less desirable characteristics than their male counterparts, then it may result in less mentoring time or lower mentorship quality for female students. However, if there are large benefits from homophily in the quality of advice received, then female students may be better off making this tradeoff. Ultimately, it is theoretically ambiguous whether homophily hurts or improves the quality of mentorship received.

Even though female students prefer to interact with female mentors, the patterns from the online mentoring platform provide little evidence that female mentors offer higher quality mentorship. Female mentors are less likely than male mentors to respond to messages from both male and female students. Furthermore, relative to male mentors, female mentors’ responses to female students are shorter. These descriptive patterns call into question whether the benefits of homophily outweigh its costs.

To causally isolate homophily by gender in advice seeking, and shed light on its costs and benefits to students, we adapt a common methodology for estimating willingness-to-pay for product and job attributes. Specifically, we estimate college students’ preferences for mentor characteristics by implementing a hypothetical choice preference elicitation in a setting that incentivizes truthful responses. Students are shown pairs of hypothetical mentors’ profiles and asked to choose which mentor they prefer. We investigate whether female students' preference for female mentors is primarily driven by demand for advice that only female professionals can provide, or if it is because students use mentor gender as a proxy for other desirable characteristics, such as approachability. Specifically, in the preference elicitation, alongside the hypothetical mentor profiles, we provide information from a prior mentee (whose gender is randomized) on their experience with the mentor, including the extent to which the mentor was knowledgable about job opportunities, easy to talk to/friendly, and gave personalized advice.
External Link(s)

Registration Citation

Citation
Gallen, Yana and Melanie Wasserman. 2021. "Homophily in Advice Seeking." AEA RCT Registry. December 10. https://doi.org/10.1257/rct.8674-1.0
Experimental Details

Interventions

Intervention(s)
We estimate college students’ preferences for mentor characteristics by implementing a hypothetical choice preference elicitation in a setting that incentivizes truthful responses (namely, receiving personalized help with finding a mentor based on responses). Students are shown pairs of hypothetical mentors’ profiles and asked to choose which mentor they prefer. The characteristics visible to students are: mentor first name, mentor occupation, mentor year of college graduation, whether the mentor is a first generation college goer, mentor availability, and (when shown) the rating of the mentor by a previous mentee (indicated by first name) on 3 dimensions (knowledgable about job opportunities, easy to talk to/friendly, and gave personalized advice). The gender of mentors and mentees is indicated by their first name. We investigate whether female students' preference for female mentors is primarily driven by demand for advice that only female professionals can provide, or if it is because students use mentor gender as a proxy for other desirable characteristics, such as approachability. Specifically, in the preference elicitation, alongside the hypothetical mentor profiles, we provide information from a prior mentee (whose gender is randomized) on their experience with the mentor, including the extent to which the mentor was helpful with finding an internship/job, easy to talk to/friendly, and gave personalized advice. We will randomize 50% of student participants to see pairs of profiles with ratings (ratings treatment) and 50% to see pairs of profiles without ratings (no ratings treatment). All other profile characteristics, and the ratings themselves, are randomized in each profile comparison.
Intervention Start Date
2021-11-12
Intervention End Date
2022-02-12

Primary Outcomes

Primary Outcomes (end points)
We test whether female students have a preference for female mentors in the treatment with ratings and without ratings. For men, our previous work suggests that there is no same-gender preference in mentoring so we do not anticipate finding such a preference in either treatment in this experiment.
Primary Outcomes (explanation)
We test for whether the coefficient from a regression of whether a profile is chosen by female students on the gender difference in profiles (dMale) is significantly different from 0, clustering at the student level, and whether the coefficients on dMale are different from one another in the two treatments. We conduct the same analysis for male students but do not anticipate a difference between the treatments or a significant effect of mentor gender.

Secondary Outcomes

Secondary Outcomes (end points)
We are also interested in homophily by whether a student/mentor is a first generation college goer and how these preferences are affected by ratings. The tests will be the same as above for gender. In addition to estimating the preference for female mentors, we also estimate students' average willingness to pay for female mentors in terms of other mentor characteristics, in particular mentor availability and occupation match.
Secondary Outcomes (explanation)
Willingness to pay is the ratio of coefficients on dMale and dAvailability, dSame occupation, etc, where d* is the difference in the levels of those characteristics between the two profiles in a given profile comparison. This is estimated using a linear probability model, and using a logit model (for robustness).

Experimental Design

Experimental Design
The survey is conducted online through Qualtrics. Upon opening the survey link, students are randomized into the ratings or no ratings condition. The students are then shown 30 pairs of mentor profiles and asked to choose their preferred mentor within each pair. The characteristics of each mentor profile as well as the ratings and gender of the previous mentee are random and independently drawn in each profile. In order to incentivize truthful responses, we tell students at the beginning of the study that their responses will be used to provide them with personalized advice on how to find mentors. After completing the profile comparisons, students are asked background questions about their demographic characteristics, advice-seeking behavior, and stated preferences for mentor characteristics. Within two weeks of completing the survey, students relieve personalized advice and a five dollar deposit into their student account.
Experimental Design Details
Randomization Method
Randomization is done by a computer (javascript in qualtrics at the time the survey is opened)
Randomization Unit
The student is the level of randomization
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
500 female students
Sample size: planned number of observations
15000 student-mentor pairs
Sample size (or number of clusters) by treatment arms
250 students in no ratings treatment, 250 students in ratings treatment. All other profile characteristics randomly and independently assigned and expected to be balanced across treatments.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Based on a prior version of the survey without ratings, we expect a mean preference for female mentors among female students of -0.045. We hope to detect with 80% power a fall of this estimate to at least -0.005. Given standard deviation of .766, we need 7470 profile comparisons and so 249 female students per treatment (we aim for 250). **Initial estimates indicate .01 intercluster correlation and mean of .045
IRB

Institutional Review Boards (IRBs)

IRB Name
UCLA Office of the Human Research Protection Program
IRB Approval Date
2021-10-26
IRB Approval Number
IRB#19-001661-AM-00001

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials