Cultural institutions: The Effect of Physical and Digital Layouts on Visitor Experience

Last registered on May 26, 2022

Pre-Trial

Trial Information

General Information

Title
Cultural institutions: The Effect of Physical and Digital Layouts on Visitor Experience
RCT ID
AEARCTR-0009433
Initial registration date
May 25, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
May 26, 2022, 11:46 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
IESE Business School

Other Primary Investigator(s)

PI Affiliation
IESE Business School
PI Affiliation
London Business School

Additional Trial Information

Status
In development
Start date
2022-05-23
End date
2022-06-12
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
A fundamental operational decision faced by museums and other major cultural institutions is the design of a display arrangement for their collection of items (artworks, objects, media contents), which in turn affects the visitors' interactions with the offered experiences. In this project, we collaborate with the Van Gogh Museum in Amsterdam to carry out experiments that help us causally identify potential links between layout-level decisions made by the experience providers and the visitor engagement, satisfaction levels, and learnings. The multimedia guides provided by the museum help us record visitors' movements and activities across time and space through granular event logs. Hence, the visitors navigate through both the physical layout, i.e. the arrangement of exhibits on display, and the digital layout, i.e. the "tour" of artworks recommended by the multimedia guide. In this study, we intend to analyse the interplay between these two types of content provision on the way visitors interact with the exhibits. Accordingly, across the testing period, we have two interventions, driven by our theoretical conjectures, that alter the offerings on the multimedia guides. The corresponding effect on the visitor engagement is measured, on a first level, through a Difference-in-Differences methodology; on a second level, we use a path-based Multinomial Logit model developed in previous research (paper in writing with same authors using retrospective data), to study the differences in the sequential choices made by the visitors across the treatment and control groups.
External Link(s)

Registration Citation

Citation
Aouad, Ali, Abhishek Deshmane and Victor Martínez de Albéniz. 2022. "Cultural institutions: The Effect of Physical and Digital Layouts on Visitor Experience." AEA RCT Registry. May 26. https://doi.org/10.1257/rct.9433-1.0
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2022-05-25
Intervention End Date
2022-06-12

Primary Outcomes

Primary Outcomes (end points)
We intend to track three different types of outcome variables. They are:

1. Visitor movement and clickstream activity: We track the path taken by the visitors across the museum during their visit using granular MMT logs. Based on this, we are able to build the following metrics:
a) Number of stops viewed/audio clips listened to (referred to as a "hit" hereinafter),
b) Number of hits per artwork,
c) Proportion of the audio clips listened to/abandonment,
c) Total visit time,
d) Time per artwork, and
e) Number of artworks skipped.

Note that from these individual outcomes, we can build our aggregate primary outcome measuring the hit rate of a particular artwork on a day, which will be the focus of our initial DiD analysis.

2. Visitor satisfaction: For this, we set up an online survey that contains questions related to visit satisfaction. As a part of this project, we instated a system in the museum to nudge visitors to go online and fill out these surveys after finishing their visit. Through this survey, we are able to track:
a) Visitor overall satisfaction,
b) Quality of experience with respect to the multimedia guide, and
c) Willingness of the visitor to recommend the museum to their friends/colleagues/family.

3. Visitor learning: Finally, pedagogical goals are central to the operations of the museum. Our online survey also allows us to administer a quiz which the visitors can fill out towards the end of the visit. Four multiple choice questions regarding the artworks on display are asked to the visitors with the grades and correct answers made available after the conclusion of the survey and quiz.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The Van Gogh Museum provides multimedia guides (MMTs) to its visitors, a standard offering across many museums in the world. The objective of MMTs is to enable visitors to engage with the artworks on display in an informed and interactive way. As a result of this, the visitors not only navigate the physical space in the museum, but also they also browse through the digital layout of the multimedia guide. The MMT provides the recommendation of a 'Highlights tour' that covers the masterpieces of the museum, as well as a longer, more exploratory 'Leisure tour', which includes a number of additional artworks on display. In this study, we intend to understand the effect of such services on visitor engagement (main outcome), measured by the consumption of MMT contents. The first part of the study examines the role of MMTs in promoting and signalling the importance of the artworks on display by including them in the Highlights tour. To this end, we will modify the composition of the Highlights tour by adding/removing comparable artworks and by tracking the effects of this intervention on visitor engagement. In the second part of the study, we focus on the interplay between the physical and digital spaces. Our intervention symmetrically alter the MMT offering in a high density room (i.e. a room where a large proportion of the artworks on display is featured in the Highlights tour) and a low density room. We track the differential effect of these alterations on visitor engagement.
Experimental Design Details
Randomization Method
Due to the practical operational constraints faced by the museum and the presence of interference between simultaneous visitors, it is not possible to randomize the visitors' assignment to the different arms (i.e., compositions of Highlights tour) using a standard A/B test. Hence, we adopt a switchback design over three weeks, comprising three different treatment exposures, as explained above.

Each day is assigned to a condition: control (baseline MMT composition), treatment 1 (switch of A and B), and treatment 2 (switch of C and D) according to a pre-determined schedule which remains unknown to the incoming visitors. Additionally, we construct a control group for our DiD analysis using the sample of visitors that exclusively utilise the Leisure tour, thereby remaining unaffected by our interventions on the Highlights tour.

Our switchback schedule is designed to exactly balance the day-of-week seasonality effects across the three arms, while approximately minimizing the number of days separating the control buckets from each of the two intervention buckets under the operational constraints faced by the museum. Note that since nearly all visitors make one-time visits throughout our study, there is no risk of temporal interference.
Randomization Unit
The outcomes are measured at the visitor session-level. Our analysis will be conducted with time-based clustered standard errors (two hours buckets and daily).
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
There is no random assignment of the treatment conditions. Instead, we have constructed a switchback schedule over 21 days.
Sample size: planned number of observations
While the planned number of observations is contingent on the number of visitors that show up in the museum, based on the number of visitors recorded within a time period of two weeks of the study, we expect the following numbers: Visitor movement (audio guide-based): 1200 per day * 21 days = 25200 Visitor satisfaction (survey-based): 50 per day * 21 days = 1050
Sample size (or number of clusters) by treatment arms
If C denotes the days with control condition, T and T' are the days with the two treatment conditions, we expect sample size Num(.) to be:
Num(C) = Num(T) = Num(T') = 8400
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
IESE Institutional Review Board for Research in Social Sciences and Humanities- IESE IRB
IRB Approval Date
2022-03-31
IRB Approval Number
IESE.2022.09
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials