x

Please fill out this short user survey of only 3 questions in order to help us improve the site. We appreciate your feedback!
Enhancing the Efficacy of Teacher Incentives through Loss Aversion: A Field Experiment
Last registered on February 05, 2017

Pre-Trial

Trial Information
General Information
Title
Enhancing the Efficacy of Teacher Incentives through Loss Aversion: A Field Experiment
RCT ID
AEARCTR-0001971
Initial registration date
February 05, 2017
Last updated
February 05, 2017 8:56 PM EST
Location(s)
Primary Investigator
Affiliation
Harvard University
Other Primary Investigator(s)
PI Affiliation
University of California San Diego, Rady School of Management
PI Affiliation
University of Chicago, Department of Economics
PI Affiliation
University of Chicago, Department of Economics
Additional Trial Information
Status
Completed
Start date
2007-09-01
End date
2012-07-31
Secondary IDs
Abstract
Policymakers agree that strong teachers are necessary to a high-quality education. However, it can be difficult to identify the most successful teachers, particularly in developing countries where absenteeism is pervasive. To better motivate teachers, researchers have studied teacher incentive programs, the link between teacher pay and student achievement. However, there is little evidence that suggests that these standard incentive programs enhance teacher performance. Researchers analyzed whether framing bonus payments in terms of losses (utilizing the notion of loss aversion) rather than gains would cause teachers to respond differently. The main result is that teachers with the loss aversion model had students with higher math scores (between 0.201 and 0.338 standard deviations) than those in the standard incentive program.
External Link(s)
Registration Citation
Citation
Fryer, Roland et al. 2017. "Enhancing the Efficacy of Teacher Incentives through Loss Aversion: A Field Experiment." AEA RCT Registry. February 05. https://doi.org/10.1257/rct.1971-1.0.
Former Citation
Fryer, Roland et al. 2017. "Enhancing the Efficacy of Teacher Incentives through Loss Aversion: A Field Experiment." AEA RCT Registry. February 05. http://www.socialscienceregistry.org/trials/1971/history/13762.
Experimental Details
Interventions
Intervention(s)
Researchers worked with the Chicago Heights school district and the teachers' union during the 2010-11 school year. The Chicago Heights district has nine K-8 schools with a total of approximately 3,200 students.

150 teachers joined the program and each was randomly assigned to one of five groups: 1) individual gain, 2) team gain 3) individual loss 4) team loss and 5) control. The program aimed to link financial incentives to student achievement, hoping to increase teacher effort in the classroom.
Intervention Start Date
2010-10-01
Intervention End Date
2011-06-30
Primary Outcomes
Primary Outcomes (end points)
Math, reading and science scores from the ThinkLink Predictive Assessment (a low stakes standardized assessment) and the high-stakes Illinois State Achievement Test (ISAT) (K through 2nd grade students take the Iowa Test of Basic Skills (ITBS)); teacher survey results (i.e. hours grading, hours tutoring outside of class, money spent on class materials, etc)

(all compared across grade level subgroups, demographic subgroups)
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
Researchers worked with the Chicago Heights school district and the teachers' union during the 2010-11 school year. 150 teachers joined the program and each was randomly assigned to one of five groups: 1) individual gain, where teachers received a bonus at the end of the year on a "pay for percentile" formula based on students' test score improvement, 2) team gain, where teachers received a bonus based on improvement in performance versus improvement of students taught by a "teammate" teacher, 3) individual loss, where teachers received $4,000 at the beginning of the school year and would have to return the money if student performance was less than $4,000, 4) team loss, where teachers received a bonus at the start of the year with the actual bonus based on improvement relative to the teammate teacher's students, 5) control, where teachers did not receive a bonus.

The researchers find that students whose teachers were in the "loss" treatments improved math test scores by between 0.201 and 0.338 standard deviations, relative to the control group. However, there was no significant difference in math scores between the "gain" treatments and the control group. Additionally, they reject the null hypothesis that the "gain" and "loss" treatments have the same treatment effects. There was no statistical difference between the "individual" and "team" groups between the "gain" or "loss" models.

Overall, teachers were motivated by loss aversion although the influence of attrition, cash-in-advance and potential cheating could pose a threat to this interpretation.
Experimental Design Details
Randomization Method
Before any randomization occurs, researchers paired all teachers in each school with their closest match by grade, subject(s), and students taught. Different rules were followed for teachers teaching one homeroom all day and teachers teaching multiple homerooms in one days. For the former, teachers are randomly assigned to one of the four treatments, or the control group, subject to the restriction that teachers in the “team treatments” must be in the same treatment group as his/her teammate. For teachers teaching multiple homerooms in a day, students are grouped by class and subject and assigned to one of the 5 groups.

From there, the randomization procedure uses a computer algorithm to randomize homerooms. To improve balance among the control group and the four treatment arms, over a pure random draw, the researchers re-randomized teachers after the initial draw. First, they calculated a balance statistic for the initial assignments, defined as the sum of the inverse p-values from tests of balance across all five groups. The algorithm then searches for teachers to “swap” until it finds a switch that does not violate any of the rules outlined above. If switching these teachers’ treatments would improve the balance statistic, the switch is made; otherwise it is ignored. The algorithm continues until it has tested forty potential swaps.
Randomization Unit
Homerooms. For teachers teaching the same homeroom all day, teachers are randomly assigned to one of the four treatment groups or the control subject to their "teacher teammate" being in the same treatment or control group. For teachers teaching multiple homerooms, the students are grouped by class and subject and assigned to one of the 5 groups, again subject to their "teacher teammate" being in the same treatment or control group if they are in the "team gain" or "team loss" groups.

Pairs of teachers
(Teachers are randomly assigned to one of the four treatment groups or the control group)
Was the treatment clustered?
Yes
Experiment Characteristics
Sample size: planned number of clusters
124-141 clusters (homerooms) [tables 3-4 on pages 27-8]
141 clusters (homerooms) [table 5B on page 30]
Sample size: planned number of observations
3,200 students in school district and 150 teachers participated (page 7)
Sample size (or number of clusters) by treatment arms
38 homerooms (656 observations) control group, 48 homerooms (981 observations) gain group, 59 homerooms (982 observations) loss group [table 2 on page 26]
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
Study has received IRB approval. Details not available.
IRB Approval Date
Details not available
IRB Approval Number
Details not available
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
Yes
Intervention Completion Date
June 30, 2011, 12:00 AM +00:00
Is data collection complete?
Yes
Data Collection Completion Date
June 30, 2011, 12:00 AM +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
124-141 clusters (homerooms) [tables 3-4 on pages 27-8]
141 clusters (homerooms) [table 5B on page 30]
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
3,200 students in school district and 150 teachers participated (page 7)
Final Sample Size (or Number of Clusters) by Treatment Arms
38 homerooms (656 observations) control group, 48 homerooms (981 observations) gain group, 59 homerooms (982 observations) loss group [table 2 on page 26]
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports, Papers & Other Materials
Relevant Paper(s)
Abstract
Domestic attempts to use financial incentives for teachers to increase student achievement have been ineffective. In this paper, we demonstrate that exploiting the power of loss aversion--teachers are paid in advance and asked to give back the money if their students do not improve sufficiently--increases math test scores between 0.201 (0.076) and 0.398 (0.129) standard deviations. This is equivalent to increasing teacher quality by more than one standard deviation. A second treatment arm, identical to the loss aversion treatment but implemented in the standard fashion, yields smaller and statistically insignificant results. This suggests it is loss aversion, rather than other features of the design or population sampled, that leads to the stark differences between our findings and past research.
Citation
Fryer, R. G., Levitt, S. D., List, J., & Sadoff, S. (2012 July). "Enhancing the efficacy of teacher incentives through loss aversion: A field experiment." Working Paper.
REPORTS & OTHER MATERIALS