Digital Literacy and the Spread of Misinformation in Pakistan

Last registered on January 22, 2021

Pre-Trial

Trial Information

General Information

Title
Digital Literacy and the Spread of Misinformation in Pakistan
RCT ID
AEARCTR-0004003
Initial registration date
May 10, 2019
Last updated
January 22, 2021, 6:13 AM EST

Locations

Region

Primary Investigator

Affiliation
Lahore University of Management Sciences

Other Primary Investigator(s)

PI Affiliation
Lahore University of Management Sciences

Additional Trial Information

Status
Completed
Start date
2019-05-02
End date
2019-12-01
Secondary IDs
Abstract
The increasing availability of low-cost mobile phones and mobile Internet access in emerging markets has led to widespread use of social media platforms, making them an important source of news and place for social and political activity. This trend has brought many new users online including those with limited exposure to technology. Concurrently, we have also observed an increasing trend in the spread of misinformation on such platforms. The spread of misinformation can create and exacerbate polarization, while also affecting the beliefs and actions of users without a prior ideological leaning. We carry out a household level survey which provides unique first-hand evidence on the trends in social media use, sharing of news, and spread of misinformation among low and middle-income users in Pakistan. Using a list of actual news stories circulated on social media we create a test to measure the extent to which users are likely to believe and share misinformation. We evaluate the effectiveness of two interventions for countering misinformation in a randomized control setting. Our first intervention educates users about common features of misinformation through a video, while our second intervention in addition to the video provides specific feedback to users about their own past behaviour in engaging with misinformation. We measure the effectiveness of our interventions on future behaviour of participants in believing and sharing misinformation.
External Link(s)

Registration Citation

Citation
Ali, Ayesha and Ihsan Ayyub Qazi. 2021. "Digital Literacy and the Spread of Misinformation in Pakistan." AEA RCT Registry. January 22. https://doi.org/10.1257/rct.4003-3.1
Former Citation
Ali, Ayesha, Ayesha Ali and Ihsan Ayyub Qazi. 2021. "Digital Literacy and the Spread of Misinformation in Pakistan." AEA RCT Registry. January 22. https://www.socialscienceregistry.org/trials/4003/history/84359
Experimental Details

Interventions

Intervention(s)
We evaluate the effectiveness of two interventions for countering misinformation in a randomized control setting.
Intervention Start Date
2019-05-02
Intervention End Date
2019-05-27

Primary Outcomes

Primary Outcomes (end points)
The primary outcomes of interest for this project is the ability to correctly identify a news item as false or true at endline after receiving the intervention (relative to the control group).
Primary Outcomes (explanation)
The primary outcome takes the value 1 if the user correctly identifies fake and placebo news items as false, and true news as items as true, and 0 otherwise.

Secondary Outcomes

Secondary Outcomes (end points)
Additionally, we will also examine the effect of the intervention on how respondents engage with the news items. Our intervention is designed to educate users about common features of fake news and therefore, we can study whether users in the treatment group are more careful in checking extreme emotions, verifying news, and sharing it on social media when exposed to fake news.
Secondary Outcomes (explanation)
Extreme emotion: Constructed as a likert scale.
Verifying News: A binary. 1 if they want to check the news online 0 if they do not want to check it.
Sharing: A binary, 1 if they do not want to share a fake news item, 0 if they want to share a fake news item.

Experimental Design

Experimental Design
The study conducts an individual level baseline study, administers the intervention alongside the baseline survey and then conducts an endline survey three days after the first visit was conducted.
Experimental Design Details
Randomization Method
Households are divided in grids, which are from high population density constituencies. These grids are then randomly selected on a computer to be part of the study.
Randomization Unit
Randomization unit is a grid, a grid is a 100 x 100 meters of an area in Lahore. A grid usually has five households.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
150 grids of 100 x 100 meters
Sample size: planned number of observations
750
Sample size (or number of clusters) by treatment arms
50 grids with 5 users and total of 250 users each in control, treatment 1, and treatment 2
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
We carried out power calculations for our primary outcome, using the mean and standard deviation of the probability of correctly identifying a news as true or false, observed in the pilot data. Our calculations show that with a cluster randomized design, and user level outcomes measured at baseline and endline, assuming a significance level of 5\%, 80\% power, cluster size of 5, and intra-class correlation of 0.05, and 50 grids per arm, it will be possible to detect a standardized effect of 0.30. This amounts to a 23\% increase in the mean of the primary outcome from 0.63 to 0.77 (standard deviation 0.48). We expect to gain statistical power as we will measure and control for user's baseline score in correctly identifying news as well as user's baseline characteristics.
Supporting Documents and Materials

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
Institutional Review Board: Lahore University of Management Sciences
IRB Approval Date
2019-03-25
IRB Approval Number
LUMS-IRB/03252019
Analysis Plan

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
May 23, 2019, 12:00 +00:00
Data Collection Complete
No
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
No
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials