SMS Nudges for Better Academic Performance: a Non-Placebo Randomized Controlled Trial

Last registered on October 26, 2021

Pre-Trial

Trial Information

General Information

Title
SMS Nudges for Better Academic Performance: a Non-Placebo Randomized Controlled Trial
RCT ID
AEARCTR-0008399
Initial registration date
October 25, 2021
Last updated
October 26, 2021, 2:56 PM EDT

Locations

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Primary Investigator

Affiliation
University of Hohenheim

Other Primary Investigator(s)

PI Affiliation
University of Hohenheim
PI Affiliation
University of Hohenheim

Additional Trial Information

Status
In development
Start date
2021-11-03
End date
2022-03-31
Secondary IDs
Prior work
This trial is based on or builds upon one or more prior RCTs.
Abstract
The shift to online teaching during the COVID-19 pandemic has challenged Universities to maintain high students' motivation and performance. This study will investigate the effectiveness of text messages (SMS) as a nudging tool to increase learning engagement and improve academic performance. Specifically, we will test if SMS can improve academic performance (as measured by final exam grades) regardless of the information they convey. Our hypothesis is that receiving SMS motivates the students to study regularly throughout the semester.
We will run a non-placebo randomized controlled trial (RCT) at the University of Hohenheim (Germany) within a large postgraduate course in "Multivariate Data Analysis" (MVDA; winter term 2021/2022). The students (who will provide their consent to take part in this study) will be randomly assigned to treatment and control groups. The former will receive a total of 34 short SMS throughout the semester, whereas the latter group will have access to the same information via the ILIAS online learning platform. By providing the control group the information on a platform (and not via text messages) allows us to directly measure the motivational effect of such messages (as oppose to content). Before the start of the experiment, the students will participate in a survey that collects basic socio-demographic variables. At the end of the experiment, but before the first exam period, we will ask the students to complete an end-line survey about their course satisfaction and study experience during the semester.
External Link(s)

Registration Citation

Citation
Brandt, Alicia, Hamid Oskorouchi and Alfonso Sousa-Poza. 2021. "SMS Nudges for Better Academic Performance: a Non-Placebo Randomized Controlled Trial." AEA RCT Registry. October 26. https://doi.org/10.1257/rct.8399-1.0
Experimental Details

Interventions

Intervention(s)
The treatment group will receive a set of SMS at random days (times) of the week (day) (Monday-Friday, 8AM-7PM).
The SMS will comprise reminders or will provide information related to the topics thought in the online class.
The control group will not receive any SMS, but has access to the messages in a dedicated folder of the ILIAS online platform.
Intervention Start Date
2021-11-10
Intervention End Date
2022-02-02

Primary Outcomes

Primary Outcomes (end points)
-Final exam grades (only first exam period)
-Online tests (during the semeter)
Primary Outcomes (explanation)
-Online tests (during the semeter):
- Pass/Failed (50% correct answers)
- Number of attempts (proxy for engagment during the semester)

Secondary Outcomes

Secondary Outcomes (end points)
Course satisfaction (placebo)
Use of the online forum (transmission channel)
Regular studies (transmission channel)
Coping with online teaching (transmission channel)
Correct response for specific exam questions on which topic SMS were sent (placebo)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
A non-blinded non-placebo RCT with two equally-sized arms.
Treatment group: receives reminder and lecture-related SMS (no new information) at random days and times (7AM-7PM, Mon-Fri).
Control group: no SMS, but content available on the ILIAS online teaching platform.

-Balance of baseline characteristics after randomization.
-Estimation of the intention-to-treat effect (ITT) of SMS nudges on primary and secondary outcomes via regression analysis with and without control variables (baseline characteristics)

Notes: SMS design and allocation are blind to the lecturers teaching and preparing the final exam and the online tests.
Experimental Design Details
Not available
Randomization Method
Randomization done in office by a computer.
The sample will be equally divided into treatment and control group.
Randomization Unit
Individuals (students)
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
1 University postgraduate course
Sample size: planned number of observations
200-250 individuals
Sample size (or number of clusters) by treatment arms
100-125 individuals
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Final exam grade (0-100), Alpha=5%, Beta=80%, N=210, SD=14.6 (computed using 2020-2021 data): Minimum Detectable Effect Size: 3.5 points. Online tests (0-10), Alpha=5%, Beta=80%, N=1300, SD=1.17 (computed using 2020-2021 data): Minimum Detectable Effect Size: 0.18 points.
IRB

Institutional Review Boards (IRBs)

IRB Name
University of Hohenheim
IRB Approval Date
2020-05-11
IRB Approval Number
N/A