Advice Giving Experiment

Last registered on August 10, 2026

View Trial History

Pre-Trial

Trial Information

General Information

Title

Advice Giving Experiment

RCT ID

AEARCTR-0018830

Initial registration date

August 09, 2026

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

August 10, 2026, 5:06 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Name

Ashley Perry

Affiliation

New York University Abu Dhabi

Contact Primary Investigator

Other Primary Investigator(s)

PI Name

Ben Weidmann

PI Affiliation

University College London

Contact Investigator

PI Name

David Deming

PI Affiliation

Harvard

Contact Investigator

Additional Trial Information

Status

In development

Start date

2026-08-12

End date

2028-08-14

Keywords

Behavior, Firms & Productivity, Lab

Additional Keywords

Advice, Managment, Qualitative Data

JEL code(s)

D83, D23, D91

Secondary IDs

Prior work

This trial does not extend or rely on any prior RCTs.

Abstract

Advice as verbal communication, plays an important role in the functioning of organizations but is hard to assess and measure precisely. In this study we plan to measure advice-giving ability of individuals in a novel abstract setting by randomly assigning advice from a singlw advisor to many advisees, resulting in a quantitative measure of advice-giving skill. Then for the same advisors we will measure their advice-giving in a more naturalistic setting, resulting in a qualitative measure of advice-giving skill. We will explore if the quantitative measure and the qualitative measure correlate strongly, despite different surface features, a positive finding would suggest that we are measuring a
real, transferable individual difference. Moreover, we will explore if advising skill is distinct from other individual skills and characteristics, such as IQ and interpersonal or social skills.

External Link(s)

Registration Citation

Citation

Deming, David, Ashley Perry and Ben Weidmann. 2026. "Advice Giving Experiment." AEA RCT Registry. August 10. https://doi.org/10.1257/rct.18830-1.0

Sponsors & Partners

Experimental Details

Interventions

Intervention(s)

We randomly assign voice recordings.

Intervention Start Date

2026-08-12

Intervention End Date

2026-08-30

Primary Outcomes

Primary Outcomes (end points)

The primary outcomes of interest are:

I) Quantitative measure of advice-giving skill. How well an individual's advice improves performance on a novel 'sequence task'. Specifically, can some advisors reliably improve advisee performance conditional on task specific-skill. Our proposed analysis and framework is described in Section 4 of the pre-analysis plan. The pre-analysis plan also includes a detailed description of the sequence task (See Section 3.2.1)

II) Qualitative measure of advice-giving skill. The subjective quality of an individuals advice with respect to a real-work workplace dilemma.

Primary Outcomes (explanation)

The quantitative advice measure is determined using the sequence task. There is a detailed description of the task and how it is measured in pre-analysis plan. The general idea is that an individual advisor i, their advice-giving skill is determined by the the average performance of the randomly selected group of advisees their advice was shared with conditional on their sequence task performance prior to hearing the advice.

The Qualitative advice measure is determined using an LLM to evaluate the quality of the advice, calibrated by humans. The individual advisor engages in a real-time conversation with an AI chatbot. This transcript is then scored by an LLM. The pre-analysis plan has a detailed description of the AI chatbot interaction.

Secondary Outcomes

Secondary Outcomes (end points)

Secondary Outcomes (explanation)

Experimental Design

Experimental Design Details

Not available

Randomization Method

The randomization will be done by computer. We will randomly allocated advice of participants, known as advisors, to another set of participants, known as advisees. We will use block randomization to ensure advice allocation is balanced based on the advisee gender and sequence task performance prior to hearing the advice. For the sequence task performance if an advisee scores less than 11 they are classified as low and high otherwise. Details of the randomization are found in the pre-analysis plan (See Section 3.1.4).

Randomization Unit

We randomize the allocation of an audio recording of advice to participants known as advisees.

Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters

157 individuals in the role of advisors and 64 individuals in the role of advisees per advisor, giving a total of 10,048 advisees.

Sample size: planned number of observations

157 advisors and 64 advisees per advisor, giving a total of 10,048 advisees.

Sample size (or number of clusters) by treatment arms

157 individuals in the role of advisors and 64 individuals in the role of advisees per advisor, giving a total of 10,048 advisees.

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

Given the novelty of our design – coupled with the fact that we are exploring new measurement instruments – our power calculations are extremely uncertain. Weidmann et al 2026). Both Weidmann and Deming (2021) and Weidmann et al (2026) use repeated randomization and a similar estimation strategy. However, as we are using a different, and newly devised task, we are unable to use their data to perform precise power calculations.

Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

IRB