An Information Intervention Targeting Effort in a College Course

Last registered on October 04, 2023
View Trial History

Pre-Trial

Trial Information

General Information

Title
An Information Intervention Targeting Effort in a College Course
RCT ID
AEARCTR-0012201
Initial registration date
September 28, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
October 04, 2023, 4:42 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Name
Mariana Carrera
Affiliation
Montana State University
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Andrew Hill
PI Affiliation
Montana State University
Contact Investigator

Additional Trial Information

Status
On going
Start date
2023-08-16
End date
2024-09-30
Keywords
Education
Additional Keywords
JEL code(s)
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
We conduct an information intervention via tailored messages sent to students of a large introductory course at a public university. Messages reference the student's observed assignment completion, thus far in the course, and provide statistics on the typical passing rates of students on that trajectory in the past. Outcome variables studied will be the number of subsequent assignments successfully completed on time and the subjective expectations, halfway through the semester, of own future assignment completion in the second half of the course.
External Link(s)

Registration Citation

Citation
Carrera, Mariana and Andrew Hill. 2023. "An Information Intervention Targeting Effort in a College Course." AEA RCT Registry. October 04. https://doi.org/10.1257/rct.12201-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
Students are randomly assigned to treatment or control. A tailored information message is sent via email to students in the treatment group. The message differs with whether the student has completed 0, 1, 2, or 3 of the first 3 assignments given in the course. Information includes the student's number of assignments completed and the historical passing rate of students with the same level of assignment completion over the first three assignments. The message also states how the passing rates of students in the past varied with their level of assignment completion in the subsequent six assignments. The control group instead receives a generic message about the importance of course assignments.
Intervention Start Date
2023-09-25
Intervention End Date
2023-12-06

Primary Outcomes

Primary Outcomes (end points)
1. Number of Problems & Applications assignments completed successfully (>70%).
2. Subjective guess of the number of Problems & Applications assignments they will successfully complete before the spring deadline.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Grade in the course
Secondary Outcomes (explanation)
In the subgroup who completed fewer than 3 of the first 3 Problems & Applications assignments successfully and on-time, we will study the binary outcome of passing the course as well as the average course grade.
In the subgroup who completed all 3 of the first 3 P&A assignments successfully and on-time, we will consider average course grade as a secondary outcome.

Experimental Design

Experimental Design
Students are categorized for tailored information messages based on their initial completion of the first 3 Problems & Applications (P&A) assignments in the course. There are different messages tailored for students who completed 0, 1, 2, or 3 of these first 3 assignments.

In each of these categories, half of the students are randomized to treatment and half to control. All students receive an email via the learning management system, but only the students in the treatment group receive tailored information based on their past performance and highlighting the success rates of past students with different levels of assignment completion.
Experimental Design Details
Not available
Randomization Method
Randomization done with software (StataSE 15) and stratified by: #P&A completed successfully of first 3, participation in first-day survey, gender (if participated in first-day survey), and enrollment in a discussion section.
Randomization Unit
Individual.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
582 students
Sample size: planned number of observations
582 students
Sample size (or number of clusters) by treatment arms
288 Control, 294 Treatment
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
We have 80% power to conduct a one-sided test at the 5% significance level, with minimum detectable effect size of a 0.35 increase in the number of the last 6 P&A assignments successfully completed. Our estimate of the mean of this outcome variable is 4.08 (SD = 1.72) so this would be a 8.5% increase.
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
Montana State University IRB
IRB Approval Date
2023-01-30
IRB Approval Number
2022-613-AH092022-EXEMPT
Analysis Plan