How do pre-set deadlines and reminders affect graduation rates and performance in vocational colleges?

Last registered on February 12, 2021
View Trial History

Pre-Trial

Trial Information

General Information

Title
How do pre-set deadlines and reminders affect graduation rates and performance in vocational colleges?
RCT ID
AEARCTR-0007146
Initial registration date
February 12, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
February 12, 2021, 11:05 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country
Israel
Region

Primary Investigator

Name
Miri Stryjan
Affiliation
Aalto University
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Naomi Gershoni
PI Affiliation
Ben-Gurion University of the Negev
Contact Investigator

Additional Trial Information

Status
On going
Start date
2018-04-01
End date
2021-12-31
Keywords
Behavior, Education, Labor
Additional Keywords
Vocational education, Technical college, Deadlines
JEL code(s)
Secondary IDs
Abstract
We evaluate an intervention taking place within vocational colleges in Israel, which establishes a framework with monitored deadlines and reminders for delivery of the proposal and submission of the final project required to graduate with a diploma. The goal is to study whether this way of adding structure and reducing uncertainty about the submission date can improve student performance, with particular focus on the completion of the requirements for a diploma. It will combine a randomized field experiment with collection of administrative and survey data on students in the vocational colleges. The research falls under the general headings of labor economics and economics of education, and also uses insights from behavioral economics.

The fundamental reason for low educational outcomes among students who are already enrolled in educational programs is the exertion of too little effort (see e.g. Oreopoulus et al., 2019; Clark et al., 2020). Several different yet interrelated reasons were offered for this shortcoming. First, low effort may be a results of flawed time perception leading to an underestimation of the time required to complete a task (Kahneman and Tversky, 1979). Second, procrastination which is a type of self-control problem may lead students to end-up exerting less effort than initially intended, even if their time estimations were accurate. Procrastination is often the product of present bias, which is the human tendency to focus on the here-and-now and value immediate rewards over long-term benefits (O'Donoghue and Rabin, 1999). Bisin and Hyndman (2020) document that present bias is prevalent among college students. Third, time management was pointed out as challenging for students, which again relates to problems with the perception of time (Francis‐Smythe and Robertson,1999). Previous studies show that setting clear, well-organized deadlines can increase timely submission of projects (Ariely and Wertenbroch, 2002). When looking at long-term goals, it was shown that setting such goals and then listing the intermediate steps required to achieve them can improve academic performance (Morisano et al., 2010). Inspired by these findings, in this experiment we implemented deadlines for the “final project” proposal and submission, accompanied by text message reminders.

Theoretically, diplomas and certificates are expected to decrease labor market frictions and alleviate adverse selection problems. Empirically, however, the causal effect of diplomas on labor market outcomes is difficult to identify, due to endogeneity problems (Gunderson and Oreopoulus, 2020). Provided that our intervention affects graduation rates and timing, a secondary aim of this project is to address the bigger question of the labor market returns to vocational education. Here, we plan to combine our data on students with administrative labor market data and utilize an Instrumental Variable approach to causally estimate the returns to vocational education.
External Link(s)

Registration Citation

Citation
Gershoni, Naomi and Miri Stryjan. 2021. "How do pre-set deadlines and reminders affect graduation rates and performance in vocational colleges? ." AEA RCT Registry. February 12. https://doi.org/10.1257/rct.7146-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
Context:
In Israel, high school graduates can earn a practical engineering diploma, which is a professional degree awarded by technical colleges. These studies are a combination of practical vocational training and broader theoretical studies. The Israeli National Institute for Technical Training (NITT), which operates within the Ministry of Labor, Social Affairs and Social Services, sets the criteria and regulates these studies, covering over 70 colleges and more than 30 fields of study. To graduate with a diploma, students are required to complete around 2,000 academic hours; to undertake standardized national certification exams; and to submit a final project which demonstrates the acquired knowledge and requires the application of practical skills. As in academic studies in Israel, the government regulates tuition and subsidizes the cost of training.
Israeli technical colleges, like many post-secondary educational settings, struggle with low graduation rates (or high drop-out rates). In 2016, it was estimated that only 49% of students enrolled in Israel’s technical colleges completed their course of study and only 63% of those who completed all courses were eventually eligible to receive a practical engineer’s diploma. In comparison, the drop-out rate among academic college students in Israel is only 8,7% (Feniger et al., 2016).
The timely submission and defense of the final project was pointed out as the main reason for failing to graduate (Ministry of Economy and Industry, 2017) and is therefore targeted in our intervention, which is focused on increasing the percentage of students who submitted and successfully defended their final project, thereby increasing the percentage of students who graduate with diplomas.

Intervention:
We partner with the NITT (National Institute for Technical Training) and use a randomized control methodology with randomization within matched pairs of college-by-major units, and ask how pre-announced deadlines and reminders affect completion and graduation rates, graduation timing, and for what type of students the intervention is more effective. Study participants included 4,900 students from 89 departments (major by college) in 33 technical colleges in Israel. Departments were randomly assigned to either the control group (43 departments or the experimental (treatment) group (46 departments). The experiment focused on four study majors: architecture, electrical engineering, civil engineering, and software programming.

Procedure and data collection:
The experimental intervention consists in recommending a deadline for making a project proposal and setting a specific deadline for the final submission (well in advance of the date) combined with sending text message (SMS) to students in treated departments, including motivational messages and reminders about upcoming deadlines. Control group departments use “business as usual” where no pre-defined deadline is set for proposal or project submission, but instead, defenses are scheduled on a rolling bases when at least 10 students are ready to submit and defend their project.
A short follow up survey will be administrated to students via SMS and email. The survey includes questions on the students' educational and labor market history, their attitudes and motivation for studying, and their labor market expectations. Students can choose to take the survey in either Hebrew or Arabic.
Intervention Start Date
2019-01-01
Intervention End Date
2020-02-01

Primary Outcomes

Primary Outcomes (end points)
Likelihood to complete final project, likelihood to graduate with a diploma, graduation timing.
Primary Outcomes (explanation)
Primary outcome data will be based on administrative data from NITT.

Secondary Outcomes

Secondary Outcomes (end points)
final project grade, exam grades, labor market outcomes.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Starting in April 2018, the research team planned the intervention together with NITT. In January 2019, the deadline setup was announced and explained to the head of each department in the treatment group. The recommended deadline for proposal submission was April 2019. This deadline was communicated to department heads (who were asked to instruct the students accordingly), and backed-up by personal SMS reminders to the treatment group students. In August 2019 the treated units (departments) were assigned specific defense dates. SMS nudges and reminders were regularly sent to students in the treated units between April 2019 and the pre-scheduled defense date.

Treatment was assigned using a pair-wise randomization procedure (Bruhn and McKenzie, 2009) in which departments from the same field of study were matched based on previous rates of graduation with diploma, the ethnic/religious target population of the college and the number of students.
Experimental Design Details
Randomization Method
The randomization was conducted by the researchers on a computer.
Randomization Unit
The treatment unit is the major-by-college combination, referred to here as “department”.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
89 major-by-college units, also referred to as departments
Sample size: planned number of observations
4,900 students from 89 clusters (major*college units) in 33 technical colleges.
Sample size (or number of clusters) by treatment arms
43 departments; n = 2,215 students. Treatment group: 46 departments; n = 2,685 students.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
Human Subjects Research Committee of Ben-Gurion University
IRB Approval Date
2020-06-09
IRB Approval Number
1889-1
IRB Name
Human Subjects Research Committee of Ben-Gurion University
IRB Approval Date
2020-04-23
IRB Approval Number
1844-1
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials