Countering gender stereotypes on social media: Experimental evidence

A growing literature develops novel NLP techniques to identify gender stereotypes on social media, but the analysis of potential countermeasures has received little attention. To fill this gap, we use an online experiment (between-subject design) that examines the efficacy of AI-generated counter-stereotypes, i.e., statements that challenge gender stereotypes. Specifically, we study two promising strategies (identified through a prestudy): (i) "Broadening Universals" (= stating that the stereotypical trait is not unique to the target group), and (ii) "Counter Facts" (= providing facts that contradict the stereotype).

We randomize participants into four groups of equal size:
(i) Control Group: Receives ten brief conversations (=1 statement plus 1 response) in social media language style on gender neutral topics such as travel and nature.
(ii) Gender Stereotypes: Receives ten brief conversations, where 5 conversations are on gender neutral topics as above, and 5 conversations pick up a common gender stereotype plus a neutral response.
(iii) Broadening Universals: Receives ten conversations, where 5 conversations are on gender neutral topics as above, and 5 conversations pick up a common gender stereotype plus a counter-stereotype stating that the trait is not unique to the target group.
(iv) Counter Facts: Receives ten conversations, where 5 conversations are on gender neutral topics as above, and 5 conversations pick up a common gender stereotype plus a counter-stereotype providing facts that contradict the stereotype.

After reading through the brief conversations, we ask our participants to conduct an Implicit Association Test (IAT, Greenwald et al., 1998), the most widely implemented test to elicit implicit biases. We will then compare differences in implicit biases across treatment groups.