x

NEW UPDATE: Completed trials may now upload and register supplementary documents (e.g. null results reports, populated pre-analysis plans, or post-trial results reports) in the Post Trial section under Reports, Papers, & Other Materials.
A Study on Graphical Representation
Last registered on July 15, 2019

Pre-Trial

Trial Information
General Information
Title
A Study on Graphical Representation
RCT ID
AEARCTR-0004331
Initial registration date
July 12, 2019
Last updated
July 15, 2019 9:16 AM EDT
Location(s)
Region
Primary Investigator
Affiliation
Cornell University
Other Primary Investigator(s)
PI Affiliation
Columbia University Teachers College
PI Affiliation
University of Waterloo
PI Affiliation
Princeton University
PI Affiliation
Cornell University
Additional Trial Information
Status
On going
Start date
2018-11-14
End date
2019-12-31
Secondary IDs
Abstract
Regression discontinuity design (RDD) has gained traction across social science fields as a leading quasi-experimental strategy to assess policy impacts. Graphical representation is to a large extent responsible for this growing popularity, and it is now an integral part of any well-executed RDD study. While there is statistically-based guidance on RDD graphical representation, the success of a graphical technique should ultimately lie in the correct perception of policy impacts by the reader of the graph. To date, no research has been conducted to study the role of different RDD data visualization techniques on reader perception. This project fills this gap: it aims to 1) analyze how data visualization techniques affect perception by varying exposure to an array of graphical parameters in a randomized experiment, and 2) use findings from the experiment to establish a set of graphical guidelines for practitioners. Visual representation in the related regression kink design (RKD) will be similarly studied. The findings of this project will contribute to a broader literature on data visualization and serve to create a framework for
evaluating graphical practices in the social sciences.
External Link(s)
Registration Citation
Citation
Korting, Christina et al. 2019. "A Study on Graphical Representation." AEA RCT Registry. July 15. https://doi.org/10.1257/rct.4331-1.0.
Former Citation
Korting, Christina et al. 2019. "A Study on Graphical Representation." AEA RCT Registry. July 15. http://www.socialscienceregistry.org/trials/4331/history/50003.
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details
Interventions
Intervention(s)
Our treatments correspond to graphical representation choices.
Intervention Start Date
2018-11-14
Intervention End Date
2019-12-31
Primary Outcomes
Primary Outcomes (end points)
Binary variable indicating whether a graph was correctly classified by the participant.
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Binary variable indicating the participant's stated level of confidence (from bonus choice).
Time spent per question.
Stated magnitude of discontinuity (for expert sample only).
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
Subjects are randomized into different graphing treatments, shown a series of regression discontinuity or regression kink graphs, and are asked to classify whether a discontinuity or kink exists.
Experimental Design Details
Sequence of experiments (only for nonexperts): Subject pool: Cornell University Johnson Business Simulation Lab subject pool Part I: video tutorial followed by examples with written explanation Part II: presentation of regression discontinuity/kink graphs; subject asked to classify discontinuity/kink in each graph and select bonus choice Part III: exit survey and collection of information on demographics and statistical background Expert sample: Subject pool: Seminar participants at Cornell and Irvine econometrics seminar; NBER and IZA affiliates Part I: presentation of regression discontinuity/kink graphs; subject asked to classify discontinuity/kink in each graph and estimate the size of discontinuity (if not presented the kink graph) Part II: Elicit preferred graphing method and ask subject to predict the best/worst performing method within the nonexpert sample Part III: exit survey and collection of information on demographics and research background Eye tracking study: Subject pool: Cornell University Johnson Business Simulation Lab subject pool and select Cornell faculty members Part I: video tutorial (to nonexperts only) Part II: presentation of regression discontinuity graphs; subject's eye movement is tracked and is asked to classify discontinuity in each graph Part III: Think-aloud protocol where we review eye movements with a subset of participants
Randomization Method
Between subject randomization by a computer for regression discontinuity study
Mixed-design (within and between subject randomization) by a computer for regression kink study
Randomization Unit
Individual
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
we do not cluster sample
Sample size: planned number of observations
Nonexpert sample: 5 rounds of experiments; overall 1,632 unique surveys initiated (a subset of the subjects participated in more than one round of the experiment) Expert sample: Will send study link via email to up to 2,500 individuals. Eye tracking study: up to 50 subjects
Sample size (or number of clusters) by treatment arms
Nonexpert sample:
88 individuals per treatment arm for regression discontinuity studies
72 individuals per treatment arm for regression kink studies

Expert sample:
Not subject to fixed sample size: depending on the response rate
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
See attached OSF pre-registration plan, which we filed before each round of study
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
Princeton University Research Integrity & Assurance
IRB Approval Date
2018-09-17
IRB Approval Number
10948
IRB Name
University of Waterloo Office or Research Ethics
IRB Approval Date
2018-08-07
IRB Approval Number
23298
IRB Name
Columbia University Teachers College
IRB Approval Date
2018-11-12
IRB Approval Number
18-477
IRB Name
Cornell University IRB
IRB Approval Date
2018-07-31
IRB Approval Number
1807008155
Analysis Plan

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
No
Is data collection complete?
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports, Papers & Other Materials
Relevant Paper(s)
REPORTS & OTHER MATERIALS