How does the prospect of success or failure nudge students to higher achievement? A large-scale randomized vignette experiment

Last registered on January 21, 2022

Pre-Trial

General Information

Title

RCT ID

AEARCTR-0008673

Initial registration date

December 07, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

December 10, 2021, 10:58 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated

January 21, 2022, 11:16 AM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Country

Hungary

Region

Primary Investigator

Name

Tamas Keller

Affiliation

Research Center for Educational and Network Studies, Hungarian Academy of Sciences, Centre for Social Sciences; TÁRKI Social Research Institute, Budapest

Contact Primary Investigator

Other Primary Investigator(s)

Additional Trial Information

Status

In development

Start date

2021-12-08

End date

2022-08-31

Keywords

Education

Additional Keywords

JEL code(s)

Secondary IDs

Prior work

This trial does not extend or rely on any prior RCTs.

Abstract

How to frame nudging to students to motivate them to high achievement? To answer this question, we plan a large-scale randomized vignette experiment among university students that test the impact of differently framed nudging messages on students’ motivation. All students at the University of Szeged receive a random vignette via SMS out of eight possible vignettes. Vignettes describe students the hypothetical exam with systematically different wording along three dimensions (framing, regulatory focus, and exam difficulty). We test how the different descriptions of the vignettes influence students’ anticipated effort (our outcome) measured by students’ motivation in preparing for the hypothetical exam.

External Link(s)

Registration Citation

Citation

Keller, Tamas. 2022. "How does the prospect of success or failure nudge students to higher achievement? A large-scale randomized vignette experiment." AEA RCT Registry. January 21. https://doi.org/10.1257/rct.8673-1.1

Sponsors & Partners

Interventions

Intervention(s)

Students will read a short description of a (hypothetical) exam. This description is referred to as the vignette. Students receive a random vignette out of eight possible vignettes (vignette universe). The matching of students with the specific vignette occurs based on a random number.

Intervention Start Date

2021-12-08

Intervention End Date

2022-02-12

Primary Outcomes

Primary Outcomes (end points)

Our outcome variable is students’ effort intended to invest preparing for the exam. The variable will be measured on a scale ranging from 0 to 10 by deploying the question described above.

Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)

Secondary Outcomes (explanation)

Experimental Design

The vignettes in the vignette universe differ in three dimensions (framing, regulatory focus, and exam difficulty). There are two options in every dimension, which equals the eight possible vignettes that students can theoretically receive (2×2×2 = 8). In short, the two specific options in each dimension are the varying part of the vignettes that make them unique

Experimental Design Details

See the detailed registration plan

Randomization Method

The matching of students with the specific vignette occurs based on a random number.

Randomization Unit

Students

Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters

This is not a cluster-randomized design

Sample size: planned number of observations

2000

Sample size (or number of clusters) by treatment arms

Students receive the eight vignettes with equal likelihood, thus ca. 250 students will answer each vignette

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

We calculate the power using the program: Optimal Design (Spybrook et al. 2011). The calculated minimum detectable Cohen’s d effect size for the p-value of 0.0167 (two-sided t-test) and sample size of 2,000 is 0.14. Thus, our design has 80% power to detect substantially small effects

Supporting Documents and Materials

IRB