x

Please fill out this short user survey of only 3 questions in order to help us improve the site. We appreciate your feedback!
Back to History

Fields Changed

Trial
Field Before After
Status In Development Completed
Abstract I design a field experiment where for a sample of roughly 850 teachers employed in 39 schools, I study how repeated public recognition impacts their short and long run performance. In each school I define the top performing employees based on teacher value added to the student grades, since the beginning of the first semester. After 4 months of recording performance, in a randomly selected half of the schools, the top performing employees are publicly praised in an on-line message posted on the school messaging board. The message is observed by staff, students and parents. Following the first round of intervention, I continue collecting data on changes in teacher performance throughout the remaining academic year and repeat the intervention at regular intervals. After the end of the academic year, I collect data on final grades and attendance for all students and on national evaluation test scores for final year students. Some of the underlying mechanisms that generate the teacher behavior in the field are tested. This study is a significant improvement on the existing body of evidence, by being the first field experiment to jointly (i) document the long-run effects of public recognition and (ii) study the effects of repeated recognition in the field. In a large field experiment I study how repeated public praise impacts the short and long run performance of 900 teachers in 39 schools. For both recipients and non-recipients of recognition, I analyze the effect of repeated public praise on performance as measured by student grades, attendance, and performance on anonymously graded high-stake exams. In a random half of the schools, the best teachers (according to teacher value added) are praised in an online message posted on the school messaging board. When teachers are praised (not praised) in the first round, their students perform significantly better (worse) in subsequent months. Using the fact that praise is repeatedly given, I test different mechanisms that could explain teacher behavior. Results are in line with teachers having status concerns, and learning about their relative performance through praise. Recognition has large and persistent effects, but repeatedly praising teachers does not induce further performance changes.
Last Published January 18, 2018 05:14 PM July 31, 2018 06:18 AM
Primary Outcomes (Explanation) Teacher value added is given by the gap between the first grade and a weighted average of all the subsequent grades of a student, where the final grade is given a weight of 50% and for all other intermediate grades, the remaining weight is equally distributed. The first grade is considered to be the baseline performance of a student, in the beginning of an academic year. Although teachers can have the same classes for a number of years in a row, the material each year is significantly different and thus reflects a new baseline start for each student. The final grade before the Christmas break is given a higher weight for two reasons: (i) it is the least noisy measurement of teacher value added since it allows for the longest period of time to pass since the baseline performance grade is recorded and (ii) final tests before the Christmas break tend to be more general and include a larger share of the material studied up to that point, avoiding situations where students can perform poorly on one particular chapter, despite having a higher performance overall. For each teacher, the value added will be a weighted average of all the individual student level added values since the beginning of the academic year. Those teachers who are in the top 25% highest TVA within their subject will qualify for recognition and will be publicly praised. Teacher value added is given by the gap between the first grade and a weighted average of all the subsequent grades of a student, where the final grade is given a weight of 50% and for all other intermediate grades, the remaining weight is equally distributed. The first grade is considered to be the baseline performance of a student, in the beginning of an academic year. For each teacher, the value added will be a weighted average of all the individual student level added values since the beginning of the academic year. Those teachers who are in the top 25% highest TVA within their subject will qualify for recognition and will be publicly praised. More details about how teacher value added is calculated throughout the experiment can be find in the attached "Experiment Design and Analysis Plan".
Experimental Design (Public) SETTING The experiment focuses on teachers in 39 schools coming from 15 different regions of Romania. The teachers used in this experiment are those that teach the 9 academically relevant subjects for which students have to undertake exams: mathematics, native language, English language, physics, biology, chemistry, computer science, history and geography. All the schools in this experiment have voluntarily purchased the rights to use an on-line education platform which allows teachers to interact more efficiently with students and parents. While the schools using this environment might not be fully representative of the average school (they have better performing students on average), there will be no selection into treatment since the schools are not aware of being part of a field experiment. The usage of the platform comes at a small monthly cost which is covered by the parents of students. The main features on the platform are the ability to post grades and attendance on-line, and communicate in real-time through messages sent between all parties involved. Access to this data allows me to monitor the performance of all students and teachers in the school for an entire academic year. In particular, I can observe the (i) grades and (ii) attendance of all the students in the school, which will be used as the main performance measures for teachers. Schools will be randomly assigned to either treatment and control, and they will be stratified on (1) the baseline performance of the students at the school level, (2) the average teacher value added at the school level and (3) the size of the school. The baseline performance of the school is given by an average of the first grade that all the students in the school receive in the beginning of the year. The size of the school is given by the number of teachers employed at the unit. Schools which are assigned to the treatment group will receive "public recognition". More precisely, a sub-sample of the "best performing teachers" will be publicly praised through a message posted on-line through the education platform. The first round of recognition will be unannounced, and it will come as a surprise. The first round of intervention will announce that public praise will be given in the future, throughout the rest of the year, to ensure that all teachers have the same expectations. Subsequent rounds of recognition will be given at regular time-intervals until the end of the academic year. QUALIFYING FOR RECOGNITION Deciding which teachers are top-performers can be tricky since schools differ in quality and there is no official standard requirement from teachers. Since the recognition is publicly given, it is important that it is perceived as deserved by all observers. In other words, one wants to avoid situations where too few teachers qualify for recognition, or too many. Top performers will be determined on the basis of student grade improvement, referred henceforth as Teacher Value Added (TVA). Namely, for each teacher in the school the change in student performance will be recorded over a period of 3 and a half months. By the end of December, most students will have at least two grades for the 9 subjects of interest. Those teachers who are in the top 25% highest TVA within their subject will qualify for recognition and will be publicly praised. In this experiment, praise will be publicly given to teachers, such that everyone in the school, including students and their parents, will be able to observe the top performers. Due to this design feature, randomization will be done at the school level. As such, in the sample of 39 schools, a random half will be assigned to treatment and a random half will be assigned to control. RANDOMIZATION Since schools differs somewhat in the sample, stratification on some characteristics is important. The three main strata to be considered are (1) student quality, (2) teacher quality and (3) school size. Firstly, because there is a positive selection of the best students into higher quality schools, the initial grade students receive in the beginning of the year is a good proxi for this selection effect. To account for differences in the underlying quality of the students at each school, the randomization will be stratified by the baseline performance of the school. Baseline performance is an average of all the initial grades of the students across the 9 academic subjects of interest. Secondly, since schools can hire teachers with different quality levels on average, the distribution of teacher value added across schools is another important factor. The average teacher value added at the school level will be the second characteristic on which the sample will be stratified. Furthermore, I will also stratify the sample on the size of the school. Post randomization, I will check whether the treatment is balanced across regions, degree of urbanization and on whether the school is public or private. INTERVENTION In the schools which qualify for treatment, all teachers, students and parents will be able to see on the school's electronic messaging board a message praising the top performers in the school. For the exact phrasing on the message see the attached document titled "Experiment design and analysis plan" SETTING The experiment focuses on teachers in 39 schools coming from 15 different regions of Romania. The teachers used in this experiment are those that teach the 9 academically relevant subjects for which students have to undertake exams: mathematics, native language, English language, physics, biology, chemistry, computer science, history and geography. All the schools in this experiment have voluntarily purchased the rights to use an on-line education platform which allows teachers to interact more efficiently with students and parents. While the schools using this environment might not be fully representative of the average school (they have better performing students on average), there will be no selection into treatment since the schools are not aware of being part of a field experiment. The usage of the platform comes at a small monthly cost which is covered by the parents of students. The main features on the platform are the ability to post grades and attendance on-line, and communicate in real-time through messages sent between all parties involved. Access to this data allows me to monitor the performance of all students and teachers in the school for an entire academic year. In particular, I can observe the (i) grades and (ii) attendance of all the students in the school, which will be used as the main performance measures for teachers. Schools will be randomly assigned to either treatment and control, and they will be stratified on (1) the baseline performance of the students at the school level, (2) the average teacher value added at the school level and (3) the size of the school. The baseline performance of the school is given by an average of the first grade that all the students in the school receive in the beginning of the year. The size of the school is given by the number of teachers employed at the unit. Schools which are assigned to the treatment group will receive "public recognition". More precisely, a sub-sample of the "best performing teachers" will be publicly praised through a message posted on-line through the education platform. The first round of recognition will be unannounced, and it will come as a surprise. The first round of intervention will announce that public praise will be given in the future, throughout the rest of the year, to ensure that all teachers have the same expectations. Subsequent rounds of recognition will be given at regular time-intervals until the end of the academic year. QUALIFYING FOR RECOGNITION Deciding which teachers are top-performers can be tricky since schools differ in quality and there is no official standard requirement from teachers. Since the recognition is publicly given, it is important that it is perceived as deserved by all observers. Top performers will be determined on the basis of student grade improvement, referred henceforth as Teacher Value Added (TVA). Those teachers who are in the top 25% highest TVA within their subject will qualify for recognition and will be publicly praised. In this experiment, praise will be publicly given to teachers, such that everyone in the school, including students and their parents, will be able to observe the top performers. Due to this design feature, randomization will be done at the school level. As such, in the sample of 39 schools, a random half will be assigned to treatment and a random half will be assigned to control. RANDOMIZATION Since schools differs somewhat in the sample, stratification on some characteristics is important. The three main strata to be considered are (1) student quality, (2) teacher quality and (3) school size. Firstly, because there is a positive selection of the best students into higher quality schools, the initial grade students receive in the beginning of the year is a good proxy for this selection effect. To account for differences in the underlying quality of the students at each school, the randomization will be stratified by the baseline performance of the school. Baseline performance is an average of all the initial grades of the students across the 9 academic subjects of interest. Secondly, since schools can hire teachers with different quality levels on average, the distribution of (baseline) teacher value added across schools is another important factor. The average teacher value added at the school level will be the second characteristic on which the sample will be stratified. Furthermore, I will also stratify the sample on the size of the school. INTERVENTION In the schools which qualify for treatment, all teachers, students and parents will be able to see on the school's electronic messaging board a message praising the top performers in the school. For the exact phrasing on the message see the attached document titled "Experiment design and analysis plan"
Planned Number of Clusters 39 schools and/or 850 teachers. The treatment will be clustered at the school level. There are 39 schools in the sample.
Sample size (or number of clusters) by treatment arms There will be 21 schools in the treatment group and 18 schools in the control group. There are 21 schools in the treatment group and 18 schools in the control group.
Back to top

Fields Removed