NEW UPDATE: Completed trials may now upload and register supplementary documents (e.g. null results reports, populated pre-analysis plans, or post-trial results reports) in the Post Trial section under Reports, Papers, & Other Materials.
My Professor Cares: Experimental Evidence on the Role of Faculty Engagement
Initial registration date
May 20, 2020
May 21, 2020 1:33 PM EDT
University of California, Davis
Other Primary Investigator(s)
University of California, Davis
Additional Trial Information
Despite a growing body of literature that instructors “matter” in higher education, there is virtually no evidence about how their actions influence student outcomes. In this paper we provide experimental evidence on the impact of specific faculty behaviors aimed at increasing student success. We test the effect of professor feedback on student success in higher education classrooms though a "light-touch" randomized intervention. We present results from a small pilot in an introductory-level microeconomics course at a comprehensive research university, and the scale-up conducted in over 30 classrooms and nearly 4,000 students at a large broad-access university. The intervention consisted of several strategically-timed E-mails to students from the professor indicating keys to success in the class, the professor’s knowledge of the students’ current standing in the course, and a reminder of when the professor is available. Results from the pilot show that students in the treatment group scored higher on exams, homework assignments, and final course grade, compared to students in the control group. Results from the larger experiment at a broad access institution are more mixed—we find significant positive effects on student perceptions of the professor and course for all students. However, we only find positive achievement effects for our target population, first year students from underrepresented minority groups. Finally, we replicated the pilot to test the robustness of these results and again find positive effects on student achievement at the large comprehensive university. We conclude that in certain settings and with some students, targeted feedback from professors can lead to meaningful gains in achievement.
Carrell, Scott and MIchal Kurlaender. 2020. "My Professor Cares: Experimental Evidence on the Role of Faculty Engagement." AEA RCT Registry. May 21.
We test the effect of increased and individualized professor feedback on student success. The “light-touch” intervention consisted of strategically-timed personalized e-mails to students from the professor indicating the professor’s knowledge of the students’ current standing in the course, keys to success in the class, and a reminder of when the professor is available.
Intervention Start Date
Intervention End Date
Primary Outcomes (end points)
Course grade, percentage of points earned in the course, and course passing.
Primary Outcomes (explanation)
Secondary Outcomes (end points)
To assess plausible mechanisms of treatment effects, we collected data on student perceptions of the professor and course. These included whether the professor was approachable, available, and cared as well as how well the student felt supported and informed.
Secondary Outcomes (explanation)
The study was implemented in two separate waves during the spring of 2016 and fall of 2017 at a large representative, broad-access four-year institution. We randomly chose 30 large undergraduate courses (serving over 120 students) in each respective term and identified the instructor of record. Collaborating with the Campus Center for College and Career Readiness we recruited these professors by sending personalized letters signed by both the Provost and Dean of Undergraduate Studies. In total, 34 faculty members across 18 different course subjects participated in the study, with nearly 4,000 total students in the treatment and control groups.
In the spring of 2016 the treatment group comprised of a randomly selected one-half of the students in 14 large undergraduate classes. To assess whether spillovers may be biasing our estimates, in the Fall of 2017, we drew our treatment group in two different ways. In eight large classes (>120 students) we randomly selected one-third of students into treatment (in contrast to the half randomized in the first wave). Additionally, we randomly selected the entire class to receive treatment in the ten cases where the professor taught two sections of the identical course. Second, rather than providing two targeted emails as was done in the pilot (at a 10-week quarter system course), we chose to have three targeted emails in the scale-up (at 16-week semester system courses). During spring of 2016, the first email entailed an initial “welcome to my class” message containing strategies to succeed in the course. The second and third emails were targeted performance feedbacks at the midway point in the course and just before the final exam. In the fall of 2017, similar to the pilot, we asked professors to give students in the treatment targeted feedback based on the first “meaningful” assignment as well as midway through the course and just before the final exam.
Experimental Design Details
For the large classes, students were randomly selected within the class using a random number generator in MS Excel. For the ten cases where the professor taught two sections of the identical course, we randomly selected one class using a computer generated "coin flip".
Randomization was done within classrooms for the large classes and across classrooms for the ten cases where the professor taught two sections of the identical course.
Was the treatment clustered?
Sample size: planned number of clusters
Sample size: planned number of observations
Sample size (or number of clusters) by treatment arms
2,121 students in control, 1,809 treatment, 43 classrooms, 27 courses subjects, and 34 different instructors.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
We conducted several power analyses prior to implementation. For example, if the baseline course passing rate is 80%, at the 5% level of statistical significance, we would be able to detect treatment effects of 0.078 percentage points or greater. For course performance, we will be able to detect even small treatment effects. For example, at 14 percentage point standard deviation in course grades, we will be able to detect effect sizes as small as 3.195 percentage points.
INSTITUTIONAL REVIEW BOARDS (IRBs)
University of California, Davis
IRB Approval Date
IRB Approval Number
Post Trial Information
Is the intervention completed?
Is data collection complete?