Experimental Design Details
In this research study, we test whether committing to a regular study time encourages enhances student performance, and whether various enforcement mechanisms can further strengthen this effect. We test whether self-efficacy messaging can boost marginalized students’ self-expectations of performance and in turn their eventual performance. Finally, we implement a tutoring program to test whether supplementing online instruction with personalized, virtual tutoring results in cost-effective learning gains.
The first of the interventions is designed to answer the question of whether committing to a regular and structured study time will encourage students to stick to that committed time, and whether this consistency in turn translates into higher eventual performance. To that end, we will provide a randomly chosen subset of students with the option to commit to a regular study time (RST). We will ask those students that opt in to record the time or times that they plan to dedicate to the course each week. However, it is not immediately obvious whether asking students to commit to a regular study time will result in them doing so in practice. For this reason, we also plan to test the impact of various enforcement mechanisms (EM). One enforcement mechanism will be a message provided to a random subset of students that the course staff can monitor usage by looking at timestamps. The second enforcement mechanism will be email reminders sent either one third of the way through the course, two thirds of the way through the course, or at both times. These reminders will encourage students to stick to their committed study time and provide an indication of how closely they have been adhering to that time. Again, the option of receiving these reminders will be randomly assigned (students will have to opt in). We plan to compare ultimate performance in the course between the control and treatment groups.
The second intervention is designed to test whether providing self-efficacy messages can improve self-expectation of performance and eventual performance, particularly among marginalized populations such as non-native English speakers and female students. To that end, we will include in the entrance survey self-efficacy messages that provide factual information on who did well in spring 2013. Self-efficacy categories include gender and primary language spoken at home. Exposure to these messages will be randomly assigned; some students will receive no message as a control. The first stage will be captured in a subsequent question in the entrance survey that measures students’ self-expectation of performance. If this first stage is strong, then we can measure the impact of receiving a self-efficacy message on eventual performance.
The final intervention is designed to test whether students would make use of personalized, virtual tutoring provided on top of the course content, and in turn whether having access to personalized, virtual tutoring has an impact on eventual performance. All students will be offered the opportunity to enter a lottery for tutoring. Of those that sign up, 500 will be randomly selected to receive tutoring with a group of 20 other students. Tutoring services will consist of weekly online group review sessions, availability for individual questions over email (on assignments or on lectures) on a weekly basis, and a final exam group review session. The tutor will in effect play the role that teaching assistants play in residential education. We plan to monitor the level of engagement between tutors and tutees and to examine the effect of having access to a tutor on eventual performance.