Enhancing the Efficacy of Teacher Incentives through Loss Aversion: A Field Experiment

Last registered on February 05, 2017

Pre-Trial

Trial Information

General Information

Title
Enhancing the Efficacy of Teacher Incentives through Loss Aversion: A Field Experiment
RCT ID
AEARCTR-0001971
Initial registration date
February 05, 2017

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
February 05, 2017, 8:56 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Primary Investigator

Affiliation
Harvard University

Other Primary Investigator(s)

PI Affiliation
University of California San Diego, Rady School of Management
PI Affiliation
University of Chicago, Department of Economics
PI Affiliation
University of Chicago, Department of Economics

Additional Trial Information

Status
Completed
Start date
2007-09-01
End date
2012-07-31
Secondary IDs
Abstract
Policymakers agree that strong teachers are necessary to a high-quality education. However, it can be difficult to identify the most successful teachers, particularly in developing countries where absenteeism is pervasive. To better motivate teachers, researchers have studied teacher incentive programs, the link between teacher pay and student achievement. However, there is little evidence that suggests that these standard incentive programs enhance teacher performance. Researchers analyzed whether framing bonus payments in terms of losses (utilizing the notion of loss aversion) rather than gains would cause teachers to respond differently. The main result is that teachers with the loss aversion model had students with higher math scores (between 0.201 and 0.338 standard deviations) than those in the standard incentive program.
External Link(s)

Registration Citation

Citation
Fryer, Roland et al. 2017. "Enhancing the Efficacy of Teacher Incentives through Loss Aversion: A Field Experiment." AEA RCT Registry. February 05. https://doi.org/10.1257/rct.1971-1.0
Former Citation
Fryer, Roland et al. 2017. "Enhancing the Efficacy of Teacher Incentives through Loss Aversion: A Field Experiment." AEA RCT Registry. February 05. https://www.socialscienceregistry.org/trials/1971/history/13762
Experimental Details

Interventions

Intervention(s)
Researchers worked with the Chicago Heights school district and the teachers' union during the 2010-11 school year. The Chicago Heights district has nine K-8 schools with a total of approximately 3,200 students.

150 teachers joined the program and each was randomly assigned to one of five groups: 1) individual gain, 2) team gain 3) individual loss 4) team loss and 5) control. The program aimed to link financial incentives to student achievement, hoping to increase teacher effort in the classroom.
Intervention Start Date
2010-10-01
Intervention End Date
2011-06-30

Primary Outcomes

Primary Outcomes (end points)
Math, reading and science scores from the ThinkLink Predictive Assessment (a low stakes standardized assessment) and the high-stakes Illinois State Achievement Test (ISAT) (K through 2nd grade students take the Iowa Test of Basic Skills (ITBS)); teacher survey results (i.e. hours grading, hours tutoring outside of class, money spent on class materials, etc)

(all compared across grade level subgroups, demographic subgroups)
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Researchers worked with the Chicago Heights school district and the teachers' union during the 2010-11 school year. 150 teachers joined the program and each was randomly assigned to one of five groups: 1) individual gain, where teachers received a bonus at the end of the year on a "pay for percentile" formula based on students' test score improvement, 2) team gain, where teachers received a bonus based on improvement in performance versus improvement of students taught by a "teammate" teacher, 3) individual loss, where teachers received $4,000 at the beginning of the school year and would have to return the money if student performance was less than $4,000, 4) team loss, where teachers received a bonus at the start of the year with the actual bonus based on improvement relative to the teammate teacher's students, 5) control, where teachers did not receive a bonus.

The researchers find that students whose teachers were in the "loss" treatments improved math test scores by between 0.201 and 0.338 standard deviations, relative to the control group. However, there was no significant difference in math scores between the "gain" treatments and the control group. Additionally, they reject the null hypothesis that the "gain" and "loss" treatments have the same treatment effects. There was no statistical difference between the "individual" and "team" groups between the "gain" or "loss" models.

Overall, teachers were motivated by loss aversion although the influence of attrition, cash-in-advance and potential cheating could pose a threat to this interpretation.
Experimental Design Details
Randomization Method
Before any randomization occurs, researchers paired all teachers in each school with their closest match by grade, subject(s), and students taught. Different rules were followed for teachers teaching one homeroom all day and teachers teaching multiple homerooms in one days. For the former, teachers are randomly assigned to one of the four treatments, or the control group, subject to the restriction that teachers in the “team treatments” must be in the same treatment group as his/her teammate. For teachers teaching multiple homerooms in a day, students are grouped by class and subject and assigned to one of the 5 groups.

From there, the randomization procedure uses a computer algorithm to randomize homerooms. To improve balance among the control group and the four treatment arms, over a pure random draw, the researchers re-randomized teachers after the initial draw. First, they calculated a balance statistic for the initial assignments, defined as the sum of the inverse p-values from tests of balance across all five groups. The algorithm then searches for teachers to “swap” until it finds a switch that does not violate any of the rules outlined above. If switching these teachers’ treatments would improve the balance statistic, the switch is made; otherwise it is ignored. The algorithm continues until it has tested forty potential swaps.
Randomization Unit
Homerooms. For teachers teaching the same homeroom all day, teachers are randomly assigned to one of the four treatment groups or the control subject to their "teacher teammate" being in the same treatment or control group. For teachers teaching multiple homerooms, the students are grouped by class and subject and assigned to one of the 5 groups, again subject to their "teacher teammate" being in the same treatment or control group if they are in the "team gain" or "team loss" groups.

Pairs of teachers
(Teachers are randomly assigned to one of the four treatment groups or the control group)
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
124-141 clusters (homerooms) [tables 3-4 on pages 27-8]
141 clusters (homerooms) [table 5B on page 30]
Sample size: planned number of observations
3,200 students in school district and 150 teachers participated (page 7)
Sample size (or number of clusters) by treatment arms
38 homerooms (656 observations) control group, 48 homerooms (981 observations) gain group, 59 homerooms (982 observations) loss group [table 2 on page 26]
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Study has received IRB approval. Details not available.
IRB Approval Date
Details not available
IRB Approval Number
Details not available

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
June 30, 2011, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
June 30, 2011, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
124-141 clusters (homerooms) [tables 3-4 on pages 27-8]
141 clusters (homerooms) [table 5B on page 30]
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
3,200 students in school district and 150 teachers participated (page 7)
Final Sample Size (or Number of Clusters) by Treatment Arms
38 homerooms (656 observations) control group, 48 homerooms (981 observations) gain group, 59 homerooms (982 observations) loss group [table 2 on page 26]
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Abstract
Domestic attempts to use financial incentives for teachers to increase student achievement have been ineffective. In this paper, we demonstrate that exploiting the power of loss aversion--teachers are paid in advance and asked to give back the money if their students do not improve sufficiently--increases math test scores between 0.201 (0.076) and 0.398 (0.129) standard deviations. This is equivalent to increasing teacher quality by more than one standard deviation. A second treatment arm, identical to the loss aversion treatment but implemented in the standard fashion, yields smaller and statistically insignificant results. This suggests it is loss aversion, rather than other features of the design or population sampled, that leads to the stark differences between our findings and past research.
Citation
Fryer, R. G., Levitt, S. D., List, J., & Sadoff, S. (2012 July). "Enhancing the efficacy of teacher incentives through loss aversion: A field experiment." Working Paper.

Reports & Other Materials