A Study on Graphical Representation

Last registered on July 15, 2019
View Trial History

Pre-Trial

Trial Information

General Information

Title
A Study on Graphical Representation
RCT ID
AEARCTR-0004331
Initial registration date
July 12, 2019

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
July 15, 2019, 9:16 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
United States of America
Region

Primary Investigator

Name
Zhuan Pei
Affiliation
Cornell University
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Jordan Matsudaira
PI Affiliation
Columbia University Teachers College
Contact Investigator
PI Name
Yi Shen
PI Affiliation
University of Waterloo
Contact Investigator
PI Name
Carl Lieberman
PI Affiliation
Princeton University
Contact Investigator
PI Name
Christina Korting
PI Affiliation
Cornell University
Contact Investigator

Additional Trial Information

Status
On going
Start date
2018-11-14
End date
2019-12-31
Keywords
Other
Additional Keywords
data visualization, regression discontinuity design, regression kink design, nonparametric econometrics
JEL code(s)
C1
Secondary IDs
Abstract
Regression discontinuity design (RDD) has gained traction across social science fields as a leading quasi-experimental strategy to assess policy impacts. Graphical representation is to a large extent responsible for this growing popularity, and it is now an integral part of any well-executed RDD study. While there is statistically-based guidance on RDD graphical representation, the success of a graphical technique should ultimately lie in the correct perception of policy impacts by the reader of the graph. To date, no research has been conducted to study the role of different RDD data visualization techniques on reader perception. This project fills this gap: it aims to 1) analyze how data visualization techniques affect perception by varying exposure to an array of graphical parameters in a randomized experiment, and 2) use findings from the experiment to establish a set of graphical guidelines for practitioners. Visual representation in the related regression kink design (RKD) will be similarly studied. The findings of this project will contribute to a broader literature on data visualization and serve to create a framework for
evaluating graphical practices in the social sciences.
External Link(s)

Registration Citation

Citation
Korting, Christina et al. 2019. "A Study on Graphical Representation." AEA RCT Registry. July 15. https://doi.org/10.1257/rct.4331-1.0
Former Citation
Korting, Christina et al. 2019. "A Study on Graphical Representation." AEA RCT Registry. July 15. https://www.socialscienceregistry.org/trials/4331/history/50003
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
Our treatments correspond to graphical representation choices.
Intervention Start Date
2018-11-14
Intervention End Date
2019-12-31

Primary Outcomes

Primary Outcomes (end points)
Binary variable indicating whether a graph was correctly classified by the participant.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Binary variable indicating the participant's stated level of confidence (from bonus choice).
Time spent per question.
Stated magnitude of discontinuity (for expert sample only).
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Subjects are randomized into different graphing treatments, shown a series of regression discontinuity or regression kink graphs, and are asked to classify whether a discontinuity or kink exists.
Experimental Design Details
Sequence of experiments (only for nonexperts):
Subject pool: Cornell University Johnson Business Simulation Lab subject pool
Part I: video tutorial followed by examples with written explanation
Part II: presentation of regression discontinuity/kink graphs; subject asked to classify discontinuity/kink in each graph and select bonus choice
Part III: exit survey and collection of information on demographics and statistical background

Expert sample:
Subject pool: Seminar participants at Cornell and Irvine econometrics seminar; NBER and IZA affiliates
Part I: presentation of regression discontinuity/kink graphs; subject asked to classify discontinuity/kink in each graph and estimate the size of discontinuity (if not presented the kink graph)
Part II: Elicit preferred graphing method and ask subject to predict the best/worst performing method within the nonexpert sample
Part III: exit survey and collection of information on demographics and research background

Eye tracking study:
Subject pool: Cornell University Johnson Business Simulation Lab subject pool and select Cornell faculty members
Part I: video tutorial (to nonexperts only)
Part II: presentation of regression discontinuity graphs; subject's eye movement is tracked and is asked to classify discontinuity in each graph
Part III: Think-aloud protocol where we review eye movements with a subset of participants
Randomization Method
Between subject randomization by a computer for regression discontinuity study
Mixed-design (within and between subject randomization) by a computer for regression kink study
Randomization Unit
Individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
we do not cluster sample
Sample size: planned number of observations
Nonexpert sample: 5 rounds of experiments; overall 1,632 unique surveys initiated (a subset of the subjects participated in more than one round of the experiment) Expert sample: Will send study link via email to up to 2,500 individuals. Eye tracking study: up to 50 subjects
Sample size (or number of clusters) by treatment arms
Nonexpert sample:
88 individuals per treatment arm for regression discontinuity studies
72 individuals per treatment arm for regression kink studies

Expert sample:
Not subject to fixed sample size: depending on the response rate
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
See attached OSF pre-registration plan, which we filed before each round of study
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
Princeton University Research Integrity & Assurance
IRB Approval Date
2018-09-17
IRB Approval Number
10948
IRB Name
University of Waterloo Office or Research Ethics
IRB Approval Date
2018-08-07
IRB Approval Number
23298
IRB Name
Columbia University Teachers College
IRB Approval Date
2018-11-12
IRB Approval Number
18-477
IRB Name
Cornell University IRB
IRB Approval Date
2018-07-31
IRB Approval Number
1807008155
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials