Enhancing the Efficacy of Teacher Incentives through Loss Aversion: A Field Experiment

Researchers worked with the Chicago Heights school district and the teachers' union during the 2010-11 school year. The Chicago Heights district has nine K-8 schools with a total of approximately 3,200 students.

150 teachers joined the program and each was randomly assigned to one of five groups: 1) individual gain, 2) team gain 3) individual loss 4) team loss and 5) control. The program aimed to link financial incentives to student achievement, hoping to increase teacher effort in the classroom.

2010-10-01

2011-06-30

Math, reading and science scores from the ThinkLink Predictive Assessment (a low stakes standardized assessment) and the high-stakes Illinois State Achievement Test (ISAT) (K through 2nd grade students take the Iowa Test of Basic Skills (ITBS)); teacher survey results (i.e. hours grading, hours tutoring outside of class, money spent on class materials, etc)

(all compared across grade level subgroups, demographic subgroups)

Researchers worked with the Chicago Heights school district and the teachers' union during the 2010-11 school year. 150 teachers joined the program and each was randomly assigned to one of five groups: 1) individual gain, where teachers received a bonus at the end of the year on a "pay for percentile" formula based on students' test score improvement, 2) team gain, where teachers received a bonus based on improvement in performance versus improvement of students taught by a "teammate" teacher, 3) individual loss, where teachers received $4,000 at the beginning of the school year and would have to return the money if student performance was less than $4,000, 4) team loss, where teachers received a bonus at the start of the year with the actual bonus based on improvement relative to the teammate teacher's students, 5) control, where teachers did not receive a bonus.

The researchers find that students whose teachers were in the "loss" treatments improved math test scores by between 0.201 and 0.338 standard deviations, relative to the control group. However, there was no significant difference in math scores between the "gain" treatments and the control group. Additionally, they reject the null hypothesis that the "gain" and "loss" treatments have the same treatment effects. There was no statistical difference between the "individual" and "team" groups between the "gain" or "loss" models.

Overall, teachers were motivated by loss aversion although the influence of attrition, cash-in-advance and potential cheating could pose a threat to this interpretation.

Before any randomization occurs, researchers paired all teachers in each school with their closest match by grade, subject(s), and students taught. Different rules were followed for teachers teaching one homeroom all day and teachers teaching multiple homerooms in one days. For the former, teachers are randomly assigned to one of the four treatments, or the control group, subject to the restriction that teachers in the “team treatments” must be in the same treatment group as his/her teammate. For teachers teaching multiple homerooms in a day, students are grouped by class and subject and assigned to one of the 5 groups.

From there, the randomization procedure uses a computer algorithm to randomize homerooms. To improve balance among the control group and the four treatment arms, over a pure random draw, the researchers re-randomized teachers after the initial draw. First, they calculated a balance statistic for the initial assignments, defined as the sum of the inverse p-values from tests of balance across all five groups. The algorithm then searches for teachers to “swap” until it finds a switch that does not violate any of the rules outlined above. If switching these teachers’ treatments would improve the balance statistic, the switch is made; otherwise it is ignored. The algorithm continues until it has tested forty potential swaps.

Homerooms. For teachers teaching the same homeroom all day, teachers are randomly assigned to one of the four treatment groups or the control subject to their "teacher teammate" being in the same treatment or control group. For teachers teaching multiple homerooms, the students are grouped by class and subject and assigned to one of the 5 groups, again subject to their "teacher teammate" being in the same treatment or control group if they are in the "team gain" or "team loss" groups.

Pairs of teachers
(Teachers are randomly assigned to one of the four treatment groups or the control group)

Yes

124-141 clusters (homerooms) [tables 3-4 on pages 27-8]
141 clusters (homerooms) [table 5B on page 30]

3,200 students in school district and 150 teachers participated (page 7)

38 homerooms (656 observations) control group, 48 homerooms (981 observations) gain group, 59 homerooms (982 observations) loss group [table 2 on page 26]

Domestic attempts to use financial incentives for teachers to increase student achievement have been ineffective. In this paper, we demonstrate that exploiting the power of loss aversion--teachers are paid in advance and asked to give back the money if their students do not improve sufficiently--increases math test scores between 0.201 (0.076) and 0.398 (0.129) standard deviations. This is equivalent to increasing teacher quality by more than one standard deviation. A second treatment arm, identical to the loss aversion treatment but implemented in the standard fashion, yields smaller and statistically insignificant results. This suggests it is loss aversion, rather than other features of the design or population sampled, that leads to the stark differences between our findings and past research.

Fryer, R. G., Levitt, S. D., List, J., & Sadoff, S. (2012 July). "Enhancing the efficacy of teacher incentives through loss aversion: A field experiment." Working Paper.

https://www.povertyactionlab.org/sites/default/files/publications/926%20Teacher%20Incentives%20through%20Loss%20Aversion%20NBER%20Jul2012.pdf