The Impact of Comments on Social Media Campaigns

Last registered on July 29, 2024
View Trial History

Pre-Trial

Trial Information

General Information

Title
The Impact of Comments on Social Media Campaigns
RCT ID
AEARCTR-0013812
Initial registration date
July 22, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
July 29, 2024, 4:35 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Name
Lena Song
Affiliation
NYU
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Dante Donati
PI Affiliation
Columbia University
Contact Investigator

Additional Trial Information

Status
In development
Start date
2024-07-01
End date
2026-07-01
Keywords
Behavior
Additional Keywords
JEL code(s)
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
We measure the causal impact of comment sections on social media posts promoting racial justice. Conducted in two phases, it involves an online field experiment with over millions of Facebook users. We first collect and analyze comments to understand how political ideology influences engagement with posts. We then evaluates how exposure to different comment narratives affects subsequent engagement (comments, views), intentions (clicks), and off-platform behaviors (donations, petitions).
External Link(s)

Registration Citation

Citation
Donati, Dante and Lena Song. 2024. "The Impact of Comments on Social Media Campaigns." AEA RCT Registry. July 29. https://doi.org/10.1257/rct.13812-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
Our main intervention is to manipulate the comment section on Facebook ads.
Intervention Start Date
2024-07-01
Intervention End Date
2026-07-01

Primary Outcomes

Primary Outcomes (end points)
engagement (comments, views), intentions (clicks), and off-platform behaviors (donations, petitions)
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We will use tools on Facebook to vary what users see in the comment section of our posts.
Experimental Design Details
Not available
Randomization Method
randomization done by a computer (by Meta AB testing technology)
Randomization Unit
at the individual level
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
200 zip codes sets X 4 conditions per zip code = 800 clusters
Sample size: planned number of observations
3 million facebook users
Sample size (or number of clusters) by treatment arms
750,000 control (no comments), 750,000 treatment 1 (all comments), 750,000 treatment 2 (progressive comments), 750,000 treatment 3 (conservative comments)
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
Columbia University
IRB Approval Date
2024-04-21
IRB Approval Number
IRB-AAAU8166
Analysis Plan