Smiles in Profiles: Improving Fairness and Efficiency Using Estimates of User Preferences in Online Marketplaces

Last registered on November 15, 2024

View Trial History

Pre-Trial

Trial Information

General Information

Title

Smiles in Profiles: Improving Fairness and Efficiency Using Estimates of User Preferences in Online Marketplaces

RCT ID

AEARCTR-0014771

Initial registration date

November 12, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

November 15, 2024, 1:49 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country

United States of America

Region

Primary Investigator

Name

Emil Palikot

Affiliation

Stanford University

Contact Primary Investigator

Other Primary Investigator(s)

PI Name

Susan Athey

PI Affiliation

Stanford University

Contact Investigator

PI Name

Dean Karlan

PI Affiliation

Northwestern University

Contact Investigator

PI Name

Yuan Yuan

PI Affiliation

Carnegie Mellon University

Contact Investigator

Additional Trial Information

Status

In development

Start date

2021-11-12

End date

2025-01-01

Keywords

Behavior, Finance & Microfinance

Additional Keywords

JEL code(s)

D0, D40, J0, J02, O1

Secondary IDs

Prior work

This trial does not extend or rely on any prior RCTs.

Abstract

Online platforms face the dual challenge of promoting fairness (ensuring non-discriminatory access) and maximizing efficiency (optimizing revenue generation). In this study, we explore the impact of profile image characteristics on these goals within a micro-lending marketplace. Using computer vision algorithms and observational data, we previously found that borrowers' choices in profile creation can influence both fairness and efficiency. To further investigate these findings, we are conducting a web-based randomized survey experiment. This experiment introduces profile images generated with Generative Adversarial Networks (GANs), which vary systematically by specific visual features—such as age, hair, and the presence of glasses. In addition to testing the effect of these features on lender demand, the experiment includes incentives to encourage realistic engagement with lending decisions.

External Link(s)

Registration Citation

Citation

Athey, Susan et al. 2024. "Smiles in Profiles: Improving Fairness and Efficiency Using Estimates of User Preferences in Online Marketplaces." AEA RCT Registry. November 15. https://doi.org/10.1257/rct.14771-1.0

Sponsors & Partners

Experimental Details

Interventions

Intervention(s)

In the intervention, subjects are shown 8 pairs of borrower profiles. 7 of them are generated using a computer vision algorithm and 1 pair consists of actual images of borrower active on Kiva.org. Subjects are then asked to pick the preferred image in each pair. Images randomly vary by the age of the person shown in the image, and whether the person wears glasses or the style of their hair.

Intervention (Hidden)

Intervention Start Date

2024-11-14

Intervention End Date

2024-11-16

Primary Outcomes

Primary Outcomes (end points)

The primary outcome is whether the subject choose a profile or not.

Primary Outcomes (explanation)

We will use the data on subjects' choices to estimate the impact of age, glasses, and hair style in determining the choice.

Secondary Outcomes

Secondary Outcomes (end points)

Secondary Outcomes (explanation)

Experimental Design

Subjects recruited on Proflic.co receive one of fifteen predefined protocols. Protocols consists of 8 pairs of images in which age of the person in the image and whether the person wears glasses or not and the hair style, is randomly assigned. images in 7 pairs are generated using a computer vision algorithm. Subjects are asked to choose one of the profiles in each pair. Thus, a treatment is a pair of images a subject is choosing from. Additionally, we include three attention checks; first, we ask for the reason people participate in microfinance; second, for the explanation of the choice decision, and third, for the occupation of the person shown in the previous slide.

Experimental Design Details

Randomization Method

Randomization is made by computer.

Randomization Unit

The unit of the randomization is an experimental session. Experimental sessions are choice instances in which a subject chooses between two profiles.

Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters

In an experimental protocol there are 16 slots. Borrower profiles, both those generated using AI and those of actual Kiva borrowers, are randomly assigned to these 16 slots. We generate 15 such protocols. 400 subjects are randomly assigned to these 15 protocols. The unit of the randomization is thus an experimental session. Experimental sessions are choice instances in which a subject chooses between two profiles.

Sample size: planned number of observations

3200 choice instances

Sample size (or number of clusters) by treatment arms

3200 choice instances

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

Each experimental subject observes 16 profiles. Thus, 400 subjects generate 6400 observations. For each feature, the treated group is a profile for which the feature takes the value of 1 and the control group when the value is 0. The size of treatment and control groups is 3200, and the baseline probability of being selected 0.5. Using the two-sample proportion test, we find that the minimum detectable effect is a 2.47 percentage point difference between treatment and control in the probability of being selected. We have the same minimum detectable effect for all features of interest.

Supporting Documents and Materials

IRB