Experimental Design Details
Overview of the Experiment
This project will implement randomized shocks to the bodyweight shown on dating application profiles to observe whether participants evaluation of these profiles is influenced by excess bodyweight. To test this hypothesis, participants are shown eight dating profiles of men or eight dating profiles of women, depending on which group they select as their preferred option. While the names, biographies, and general visual appearance of these profiles is the same, there are three separate versions of each profile’s photo – one that is healthy weight, one that is overweight, and one that is obese. The dating profile photos will be obtained from stock photos or generated from AI software. The version that each participant sees is randomly assigned.
Participants will be asked to rate the profile biography, the profile photo, and the overall profile on a scale of 1-10, where participants can select integers and 10 is the maximum score. Healthy weight profile photos serve as the control group, and the treatment is whether the participant was presented with the same profile displaying an overweight or obese photo.
As part of the sub-experiment, some of the profile biographies include information revealing that the profile has a higher-earning job while others will reveal a lower-earning job. As such, there are six versions of each profile. Let HW, OW, and OB denote whether the photos are of the individual who is healthy weight, overweight, and obese, respectively. Likewise, let LE and HE denote whether the individual has a lower-earning job or a higher-earning job. The six profile versions are HW-LE, HW-HE, OW-LE, OW-HE, OB-LE, OB-HE. The comparisons are always between profiles reporting the exact same profile biography where only the profile picture varies.
Main Analysis
The project will assess whether individuals penalize dating profiles when profile photos display excess bodyweight. The project will separately examine how profile photo bodyweight influences how individuals assess the profile photos, the profile biography (i.e., a non-appearance measure), and the overall profile.
• H10: Individuals rate dating profiles similarly, regardless of whether the dating profile displays excess bodyweight.
• H11a-H1c: Individuals who are presented with a dating profile displaying excess bodyweight rate the appearance, the profile photo text, and the overall profile lower than those presented with a dating profile displaying healthy bodyweight.
The project will use a randomized controlled trial to test whether appearance and non-appearances-based assessments of dating profiles is responsive to changes in the dating profile appearance. The causal analysis can be specified as follows:
Yije = α + β∙HEAVIERj + θje + εije
(1)
where the dependent variable, Yij,is the score that participant i assigned to photo group j with earnings group e. The independent variable of interest, HEAVIERj, is an indicator taking one the value of one if the image is the overweight or obese version of the photo and is zero otherwise. In this equation, β is identified by the fact that some participants will view the exact same profile photo and text description, except half of them will view the heavier version of that photo. The prediction is that individuals will rate heavier profiles lower relatively to the healthy weight profiles (i.e., β < 0) for each of the three outcomes. The inclusion of the photo group fixed effect, θje, is the make it so that the coefficient of interest is identified from comparing differences is ratings based on the weight differences with the same profile biography text and job description.
As a refinement, the specification will be modified to test whether individuals more harshly penalize obesity compared to overweight by replacing the independent variable of interest, HEAVIERj, with two separate indicators denoting whether the participant was shown the overweight version of the photo or the obese version of the photo. The control group will remain the healthy weight version of the photo.
Yije = α + β1∙OVERWEIGHTj + β2∙OBESEj + θje + εije (2)
The null hypotheses are that individuals do not penalize overweight and obesity relative to healthy weight (i.e., βt = 0 for t = 1,2), while the alternative hypotheses is that they do penalize overweight and obesity (i.e., βt ≠ 0 for t = 1,2). We also hypothesize that that individuals will penalize obesity at least as much as overweight ((i.e., β2 ≥ β1).
To increase precision and account for incidental differences between the treatment and control groups, the specifications will be augmented with individual-level demographic variables:
• Age and age squared
• Gender (Men vs. Women)
• Sexual orientation (Heterosexual vs. Other)
• Race/Ethnicity (Non-Hispanic White, Non-Hispanic Black, Hispanic, Other)
• Indicator for racial concordance with the profile photo
• Education (No College Degree, College Degree, Graduate Degree)
• BMI status (Underweight, Healthy Weight, Overweight, Obese)
• Self-described body type (Underweight, Healthy Weight, Overweight, Obese)
Accounting for BMI status calculated from reported height and weight and self-described body type will allow for the determination of whether the participant holds an accurate, lenient, or harsh body image. Specifically, Underweight will be assigned a value of 1, Healthy Weight a value of 2, Overweight a value of 3, and Obese a value of 4 for both BMI status and Self-Described Body Type. After taking the difference of these values, participants who described themselves as heavier than their BMI status will be classified as having a harsh body image, those who described themselves as lighted will be classified as having a lenient body image, and those who correctly described their bodies will be classified as having an accurate body image.
The inclusion of these variables also allows for testing of whether certain groups respond differently to pictures displaying excess bodyweight via the following specification:
Yije = α + β1∙HEAVIERj + β2∙HEAVIERj × DEMOi+ X’i∙γ + θje + εije (3)
where DEMOi denotes the heterogeneity group of interest (e.g., Men vs. Women; College Educated vs. Non-College Educated; Overweight or Obese vs. Not Overweight or Obese; Self-Described Overweight or Obese vs. Not Self-Described Overweight or Obese; Lenient Body Image vs. Accurate Body Image vs. Harsh Body Image). For each of these subgroups, the null hypothesis is that the demographic groups similarly penalize excess bodyweight (i.e., β2 = 0). The alternative hypotheses are that the groups do not similarly penalize excess bodyweight (i.e., β2 ≠ 0). Equation (2) will be similarly augmented to test for heterogeneity across the listed dimensions.
Secondary Analysis
It is possible that the penalty from excess bodyweight may be mitigated by relative socioeconomic position. To this possibility, the following specification includes an interaction term that adjust for whether the participant viewed the profile that revealed a higher-earning career.
Yije = α + β1∙HEAVIERj + β2∙HEAVIERj × HIGH-EARNe + X’i∙γ + θje + εije (4)
where the coefficients of interest, β1 and β2, measure the penalty from excess bodyweight and whether this penalty is mitigated by having a higher-earning job. The null and alternative hypotheses for β1 are the same as previously. For the new term, the null hypothesis is that having a higher-earning job does not alter the penalty from excess bodyweight (β2 = 0), while the alternative is that having a higher-earning job does offset the penalty (β2 ≠ 0). Equation (4) will be similarly augmented to test for heterogeneity by degree of excess bodyweight (as in equation (2)) and demographic group (as in equation (3)).
Timeline and Sample
For each profile, there are three photos (healthy weight, overweight, obese) and two earnings categories (higher-earning job and lower-earning job) for a total of six profiles. There are eight images of men and eight images, though participants will only see eight profiles of men or eight profiles of women. Subject to budget availability, 400-800 participants will be recruited to evaluate the profiles of men, and 400-800 separate participants will be recruited to evaluate the profiles of women. This will result in between approximately 66 and 133 participants reviewing each bodyweight-earnings group version of the profiles. Assuming 600 participants for