Political Communication on WhatsApp

Last registered on March 16, 2021

Pre-Trial

Trial Information

General Information

Title
Political Communication on WhatsApp
RCT ID
AEARCTR-0007371
Initial registration date
March 15, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
March 16, 2021, 6:37 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
Harvard University

Other Primary Investigator(s)

Additional Trial Information

Status
In development
Start date
2021-03-14
End date
2021-04-30
Secondary IDs
Abstract
As communication has increasingly moved online, so have political conversations. In India, where the expansion of mobile internet access has been accompanied by the rise of social media, the relationship between social media and electoral outcomes is of particular interest. Yet little is known about the causal impact of social media on voters. Using an online field experiment, I will evaluate the impact of political groups on WhatsApp in the context of statewide election in Tamil Nadu, India. These groups are organized by political parties. Group members receive curated posts directly from party members, but are also able to post their own content to the groups and interact with each other. I will offer subjects an invitation link to a political WhatsApp group for their voting constituency, randomizing which of the two major political parties the assigned group is affiliated with. Conditional on clicking the hyperlink, I will further randomize subjects into one of three conditions: (1) a real group organized by the assigned party; (2) a broadcast-only group that mirrors the real group but where only official party content is forwarded but group members cannot post themselves; (3) a control, where upon clicking the link participants are redirected to a final survey page. Approximately 3-4 weeks later, I will resurvey participants, measuring political attitudes and beliefs. Conditions (1) and (3) provide an estimate of the treatment effect of the groups, while condition (2) allows me to parse the impact of the social interaction between group members relative to the official party messaging.
External Link(s)

Registration Citation

Citation
Carney, Kevin. 2021. "Political Communication on WhatsApp." AEA RCT Registry. March 16. https://doi.org/10.1257/rct.7371
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2021-03-14
Intervention End Date
2021-04-30

Primary Outcomes

Primary Outcomes (end points)
Political persuasion — Did treated subjects’ political opinions change as a result of the groups
Second-order beliefs — Did group participation influence perceptions of others’ beliefs and attitudes among a general public?
Group participation — How did click rates, join rates, leave rates, and attention vary across the groups?
Primary Outcomes (explanation)
We will measure political persuasion using a series of Likert-scale questions about the favorability of politicians and parties, and about agreement or disagreement with a series of political statements. These questions will be used to form a political ideology index.

We will measure second-order beliefs by asking people to predict the AIADMK and DMK vote shares in their constituency in the election.

We will measure group participation by exporting WhatsApp group chat histories, which include information about entry, exit and posts. To measure attention, we will perform a series of audits, posting a neutral, non-partisan message in the group and using WhatsApp’s “seen by” feature to measure which participants have looked at the groups in the 24-hours after they were posted.

Secondary Outcomes

Secondary Outcomes (end points)
Political affect — Did subjects affect towards their ideological opponents change as a result of the groups?
Political involvement — Do subjects feel personally invested in the outcome of the election?
Knowledge — Did the groups make members more informed?
Voting behavior — Did people vote and if so, for which candidates?
Secondary Outcomes (explanation)
Political affect will be measured using common survey measures of affective polarization, using a Likert scale of agreement or disagreement with a series of statements. Political involvement will involve asking people how personally invested they feel about the outcome of the election and how strongly they personally identify with a particular party. Knowledge measures will ask people factual questions about political candidates in their constituency and parties’ official platforms. Finally, we will ask people to report whether they voted and if so, for what candidate. For a subsample of the population, we will attempt to verify this by asking people to provide a photo of their left index finger, which is marked with indelible ink if they vote.

Experimental Design

Experimental Design
Prior to Tamil Nadu’s 2021 Legislative Assembly election, will recruit subjects to participate in the study using Facebook ads. We will target the ads at adults living in the 24 constituencies where we have WhatsApp group links for both major political parties contesting the election. The ads will link to a baseline survey on Qualtrics, where participants will answer questions about themselves, their media use, and their political opinions and attitudes. At the end of the survey, we will provide participants with a link to either a AIADMK or a DMK (the two leading political parties in Tamil Nadu) WhatsApp group. The partisan affiliation of the group will be randomly assigned, and participants are fully informed about the randomization. We will track clicks on the randomly assigned links. Participants who click a link will be redirected to one of two randomly assigned types of groups, or a control group.

Some participants will be redirected to a group organized by the assigned political party (condition 1). These groups exist outside the context of the study, are organized at the constituency level, and allow any member to freely post text messages, images, videos, and audio recordings. They may also reply to previous messages posted to the group. Additionally, party members that administer the group feed the group with a steady stream of curated official party messaging. All group members can see all content that is posted to the group while they are a member. The group that a participant is invited to will correspond to the constituency where they report that they vote. If a group is not available for their constituency, the subject will be offered a link for a group that is organized at state level.

Other participants will be assigned to a broadcast-only group (condition 2). These groups are also organized at the constituency level, and each possible group corresponds directly to a group from condition 1. However, in these groups, group members are not themselves allowed to post. Instead, only the messages posted to the corresponding condition 1 group are automatically forwarded to the group. Participants are informed about the nature of these groups and that they only contain content coming directly from party administrators.

Finally, some participants who click the assigned link will be assigned to a control group (condition 3) and redirected to a final page of the survey where they are told that no link will be offered.

Approximately 3-4 weeks later, we will recontact participants to complete an endline survey, where we will measure changes in their political beliefs, attitudes, and knowledge. To understand mechanisms, we will also elicit their perceptions of the political beliefs and attitudes of the general population in their constituency, to understand whether they under- or over-account for the degree of selection in the messaging they were exposed to.
Experimental Design Details
Randomization Method
All randomization will occur at the individual level. First, we will randomize the partisan affiliation of the group invitations, using an equal possibility of being assigned to each of the two major parties in Tamil Nadu (the AIADMK and the DMK).

Because we do not expect full compliance with the group invitations, we will randomize participants to a full group, a broadcast group, or a control group conditional on them clicking on a link to join a group for the assigned party. The link will simply state the partisan affiliation assigned but not which of the two types of groups (or control) the person has been assigned to — this will only be revealed once the link is clicked. We will assign participants to a full group, broadcast group, or control at an equal 1:1:1 ratio.
Randomization Unit
Individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
N/A
Sample size: planned number of observations
We aim to enroll 1,500 people in the experimental portion of the study — 500 each in the full group, broadcast group, and control conditions. Because only people who click the link that appears will actually be randomized to a group, the total number of baseline participants will exceed 1,500 but is unknown, as it will depend on take-up rates. Additionally, because we will recruit using Facebook ads, which vary in price and success rates, the actual sample size may be slightly larger or smaller, subject to the budget and click rates. We will only invite experimental participants, who clicked a link (and are therefore the relevant “compliers” for this intervention), to the endline survey.
Sample size (or number of clusters) by treatment arms
1/2 Assigned AIADMK link
1/2 Assigned DMK link

Which will be cross randomized with
1/3 Full group
1/3 Broadcast-only group
1/3 Control
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Harvard University Committee on the Use of Human Subjects
IRB Approval Date
2021-03-09
IRB Approval Number
IRB20-1862

Post-Trial

Post Trial Information

Study Withdrawal

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials