Belief formation, signal quality and information sources: Experimental evidence on air quality from Pakistan

Last registered on June 06, 2023

Pre-Trial

Trial Information

General Information

Title
Belief formation, signal quality and information sources: Experimental evidence on air quality from Pakistan
RCT ID
AEARCTR-0011489
Initial registration date
May 28, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
June 06, 2023, 3:29 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
UC Davis

Other Primary Investigator(s)

PI Affiliation
Colby College
PI Affiliation
Lahore University of Management Sciences
PI Affiliation
University of California, Davis
PI Affiliation
Williams College

Additional Trial Information

Status
On going
Start date
2023-01-12
End date
2023-07-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
An emerging body of empirical work emphasizes the demand for effective mitigation measures against severe seasonal ambient air pollution in developing cities. Yet, governments in developing countries often struggle to provide consistent and reliable air quality information. In such an environment, private alternatives to government services may improve citizens' access to air quality information. However, the efficacy of private alternatives may depend upon citizens' preferences for and beliefs about the accuracy of the information sources. We study how the source of environmental information affects citizens’ beliefs about the level of air quality and their protection measures against polluted air. In this field experiment, we vary the salience of information sources---the government vs. a private citizens group---of air pollution forecasts we provide in Lahore, Pakistan. We will measure the changes in people’s beliefs about air pollution levels and preferences for government services on air quality information through incentive-compatible elicitation games.
External Link(s)

Registration Citation

Citation
Gibson, Matthew et al. 2023. "Belief formation, signal quality and information sources: Experimental evidence on air quality from Pakistan." AEA RCT Registry. June 06. https://doi.org/10.1257/rct.11489-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
In our intervention, we send air quality forecasts via SMS to a sample of Lahore residents in a working class neighborhood. We developed an ensemble forecast model of day-ahead air pollution using data inputs from multiple sources, including government and private monitors. We then experimentally vary the salience of information sources when we give forecasts to the participants, holding constant the actual forecast. In one arm, respondents are told that the forecasts are constructed using data from Punjab Environmental Protection Department (EPD), a government agency responsible for reporting on air quality. In the other arm, respondents are told that the forecasts are constructed using data from a citizens’ group called Pakistan Air Quality Initiative (PAQI). We identify treatment effects on measures of their beliefs over air quality and preferences via incentive-compatible elicitation.
Intervention Start Date
2023-02-16
Intervention End Date
2023-06-30

Primary Outcomes

Primary Outcomes (end points)
1. Demand for air quality information as the willingness-to-pay (WTP) for SMS-based air quality forecasts
2. Beliefs about air quality levels as the absolute error of incentivized t + 1 forecast of PM2.5 concentration
3. Perceived accuracy of air-quality information source as the absolute error of incentivized guess of the SMS’s forecast
4. Preference for information source as the share of donations to government vs. citizens’ group
Primary Outcomes (explanation)
1. Demand for air quality information as the willingness-to-pay (WTP) for SMS-based air quality forecasts
• The outcome is defined as the amount respondents’ are willing to pay in PKR. We elicit respondents’ willingness to pay for the SMS forecast using the Becker- DeGroot-Marshak (BDM) method (Becker et al. 1964). In the endline survey, we ask for the respondents’ willingness to pay for the SMS-based air quality forecast messages. They have been receiving these messages for the past three months and we ask for their willingness to pay for an additional two months. In the prompt, we make the experimentally assigned source salient by reminding them that the forecast is built using data from the said source. The bid’s ceiling is set at PKR 400.
2. Beliefs about air quality levels as the absolute error of incentivized t + 1 forecast of PM2.5 concentration
• The outcome is defined as the absolute difference between the actual PM2.5 concentration and the respondent’s forecast, divided by the actual PM2.5 concentration. In both baseline and endline surveys, we ask respondents to make an incentivized guess of the air pollution level on day t + 1. In the baseline survey, we show respondents a table containing the average, minimum, and maximum of the average daily PM2.5 concentration over the last calendar week. We then ask them to forecast tomorrow’s average PM2.5 concentration. Respondents receive PKR 250 if their guess falls within 5% of the actual levels, PKR 150 if within 10%, and PKR 50 if within 20%. In the endline, we first elicit the forecast without the table containing the information from the previous calendar week. We then allow the respondents to revise their forecast after showing them the table.
3. Perceived accuracy of air-quality information source as the absolute error of incentivized guess of the SMS’s forecast
• The outcome is defined as the absolute difference between the respondent’s guess of the PM2.5 forecast generated by our model and their own forecast for t + 1. In the endline survey, we not only ask respondents to forecast the actual PM2.5 concentration for tomorrow, but also the value of our SMS forecast. The guess is financially incentivized, as in the guess for the actual PM2.5 concentration for tomorrow.
4. Preference for information source as the share of donations to government vs. citizens’ group
• The outcome is defined as the share of PKR 100 donated to a government agency for an environmental cause, as opposed to the citizen’s group. We offer an opportunity to donate PKR 100 between two sources for environment protection purposes: a government institution and PAQI.

Secondary Outcomes

Secondary Outcomes (end points)
All secondary outcomes are listed in the pre-analysis plan, which will be made publicly available after the conclusion of our trial.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We study how citizens update their beliefs about air quality and modify their behaviors, and how that depends on the information source. We also study how information quality affects citizens’ beliefs about the reliability of, and preference for, government vs. private sources. We will run a field experiment in which we provide a day-ahead pollution fore- cast via SMS to lower-middle income citizens in Lahore. We construct ensemble forecast models taking inputs from both government and private sources. We experimentally vary to which source the information is attributed (Government or Private). We collect incentivized forecast measures of air pollution before and after the intervention to track the evolution of citizens’ beliefs about air quality. We conduct lab-in-the-field games to identify citizens’ trust and preferences for the two sources, as well as their willingness to pay for the forecast service, which we provide for free during the experimental phase. We also measure impact on avoidance behavior from poor air quality by collecting time-use data and reported uses of high-filtration masks. Further detail will be made available in the pre-analysis plan after the conclusion of our trial.
Experimental Design Details
Randomization Method
We conducted the blocking and randomization on R using blockTools, a package that al- lows us to block observations on a large number of covariates and can accommodate continuous variables without discretizing them. We use the optimal-greedy algorithm and generate blocks using the Minimum Volume Elipsoid (MVE) estimator. We are primarily concerned about balance on outcome variables at baseline, as well as the “take-up” in terms of exposure and comprehension of our SMS forecast messages.
Randomization Unit
individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
1010
Sample size: planned number of observations
1010
Sample size (or number of clusters) by treatment arms
505 for government (EPD) arm, 505 for private (PAQI) arm
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
We estimate the minimum detectable effect sizes on the five hypotheses that we pre-specify in the pre-analysis plan, using new data from the baseline survey when available. The outcomes, hypotheses, sample means, and standard deviations are: 1. Willingness-to-pay (WTP) for SMS-based air quality forecasts is greater than 0 regardless of the source to which the information is attributed: 89.6 (45.2) 2. Willingness-to-pay (WTP) for SMS-based air quality forecasts is differentially affected by treatment: 89.6 (45.2) 3. Absolute error of incentivized t + 1 forecast of PM2.5 concentration, divided by the truth, is differentially affected by treatment: 0.72 (0.42) 4. Perceived accuracy of air-quality information source as the absolute error of incentivized guess of the SMS’s forecast is differentially affected by treatment: N/A 5. the amount out of PKR 100 donated to a government agency for an environmental cause, as opposed to the citizen’s group, is differentially affected by treatment: 50.1 (15.0) For hypotheses 1. and 2., we use the sample statistics from Ahmad et al. (2022) as we do not collect these outcomes in the baseline of this study. We do not have relevant statistics available from either the baseline or from Ahmad et al. (2022) for hypothesis 3., but we expect the outcome variable for it to have a similar distribution to the one for hypothesis 3.. We find that we are able to detect impacts of 0.43 standard deviations, which equals to PKR 19.4 in the willingness to pay (for hypothesis 2.), 0.18 for hypothesis 3., and 6.4 for hypothesis 5.. For the test of means for hypothesis 1., we find that we are powered to detect that willingness to pay is greater than PKR 3.6. Although the minimum detectable impact is fairly large relative to the standard deviation in this second scenario, the treatment effect sizes are relatively small in the outcomes’ units. Furthermore, there are several reasons why our assumptions may not hold or statistical precision could be improved. First, we hope to reduce standard errors by including controls selected via a double-post-selection method using LASSO. Assuming a 30-percent reduction in standard errors, the minimum detectable effects would be 0.30 standard deviations. Second, the willingness-to-pay statistic from Ahmad et al. (2022) may be outdated after two years of high inflation.
IRB

Institutional Review Boards (IRBs)

IRB Name
University of California, Davis
IRB Approval Date
2019-03-04
IRB Approval Number
IRB-1336133-1
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials