One Email to Students: Can a Light-Touch Intervention Make a Difference?

Last registered on December 02, 2020
View Trial History

Pre-Trial

Trial Information

General Information

Title
One Email to Students: Can a Light-Touch Intervention Make a Difference?
RCT ID
AEARCTR-0006396
Initial registration date
September 15, 2020

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
September 17, 2020, 8:08 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
December 02, 2020, 11:57 AM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Country
United States of America
Region

Primary Investigator

Name
Darren Page
Affiliation
University of Iowa
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Travis Williams
PI Affiliation
University of Iowa
Contact Investigator

Additional Trial Information

Status
In development
Start date
2020-09-16
End date
2021-05-15
Keywords
Behavior, Education
Additional Keywords
JEL code(s)
I23, D91
Secondary IDs
Abstract
This study measures the effects of different forms of instructor communication in an online/hybrid class setting at a large, public university. Study participants randomly receive different forms of communications from their instructors. We hypothesize that various forms of communication in this setting has the potential to affect course grades, the frequency of help-seeking, and overall perceptions of instructor quality.
External Link(s)

Registration Citation

Citation
Page, Darren and Travis Williams. 2020. "One Email to Students: Can a Light-Touch Intervention Make a Difference?." AEA RCT Registry. December 02. https://doi.org/10.1257/rct.6396-2.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
Students receive various forms of instructor communication.
Intervention Start Date
2020-09-16
Intervention End Date
2021-02-26

Primary Outcomes

Primary Outcomes (end points)
Course performance, perception of instructors, and frequency of help-seeking.
Primary Outcomes (explanation)
Course performance is measured by the following set of variables: final course grade, grade on subsequent exam, indicator for course passing. Perception of instructor quality and TA quality are indices of quality developed by a set of 5 questions related to perceptions of instructors. Frequency of help-seeking measures number of times students correspond with their instructor and TA over email and sought help via office hours.

Secondary Outcomes

Secondary Outcomes (end points)
Perceptions of instructor and TA across different categories, grades on subsets of course material, knowledge of the professor and TA's office hours dates, interest in further courses of the same discipline, time spent each week on course materials, perceptions of course difficulty.
Secondary Outcomes (explanation)
Categories of perceptions include "the professor (TA) cares whether or not I learn the material", "my professor (TA) is available to talk to me and answer my questions", "my professor (TA) is interested in teaching the course", "my professor understands the material and is able to communicate it clearly". Subsets of course material include attendance measures, homework, exams, online interaces.

Experimental Design

Experimental Design
Students receive different forms of communication from course instructors.
Experimental Design Details
Experimental sample includes students who score below 70% on the first course exam. This sample is randomized to receive one of two treatments (an email from the TA or an email from the instructor) or control (no email of this type). Randomization is stratified within discussion sections to have a 1:1:1 ratio into the three statuses.
Randomization Method
Randomization is conducted with a random number generator in Stata.
Randomization Unit
Randomization occurs at the student level.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
109 discussion sections.
Sample size: planned number of observations
2,000 students in data collection phase and 600 students in the experimental phase. Actual numbers depend on fractions of students who consent to participate and fraction scoring sufficiently low on the first exam.
Sample size (or number of clusters) by treatment arms
200 in each treatment arm and 200 in control for a total of 600.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
With standard errors clustered at discussion section level, we can detect an effect of 2.8 percentage points for overall course score 80 percent of the time. This assumes a standard deviation of 9 in overall course scores. We can detect an effect in office hour visits of 0.53, with standard deviation of 1.7. We can detect an effect of 4.2 percentage points on the second exam, with standard deviation 14.5.
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
University of Iowa Institutional Review Board
IRB Approval Date
2020-09-08
IRB Approval Number
202004489
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials