Public Recognition and Long Run Employee Performance

Last registered on July 31, 2018

Pre-Trial

Trial Information

General Information

Title
Public Recognition and Long Run Employee Performance
RCT ID
AEARCTR-0002604
Initial registration date
January 17, 2018

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
January 18, 2018, 5:14 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
July 31, 2018, 6:18 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
Erasmus University

Other Primary Investigator(s)

Additional Trial Information

Status
Completed
Start date
2017-12-22
End date
2018-07-31
Secondary IDs
Abstract
In a large field experiment I study how repeated public praise impacts the short and long run performance of 900 teachers in 39 schools. For both recipients and non-recipients of recognition, I analyze the effect of repeated public praise on performance as measured by student grades, attendance, and performance on anonymously graded high-stake exams. In a random half of the schools, the best teachers (according to teacher value added) are praised in an online message posted on the school messaging board. When teachers are praised (not praised) in the first round, their students perform significantly better (worse) in subsequent months. Using the fact that praise is repeatedly given, I test different mechanisms that could explain teacher behavior. Results are in line with teachers having status concerns, and learning about their relative performance through praise. Recognition has large and persistent effects, but repeatedly praising teachers does not induce further performance changes.
External Link(s)

Registration Citation

Citation
Cotofan, Maria. 2018. "Public Recognition and Long Run Employee Performance." AEA RCT Registry. July 31. https://doi.org/10.1257/rct.2604-2.0
Former Citation
Cotofan, Maria. 2018. "Public Recognition and Long Run Employee Performance." AEA RCT Registry. July 31. https://www.socialscienceregistry.org/trials/2604/history/32360
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
The intervention is the provision of public praise to the best performing teachers in a random sample of schools. In particular, teachers will qualify for recognition if they are among the top 25% performers in terms of teacher value-added, since the beginning of the school year. Recognition is given publicly through the means of an on-line message which is viewed by all school staff, students and parents.
The first round of public recognition is unannounced and comes as a surprise. Subsequent rounds of recognition are announced and the intervention is repeated two more times until the end of the school year.
Intervention Start Date
2018-01-22
Intervention End Date
2018-04-30

Primary Outcomes

Primary Outcomes (end points)
There are three outcomes:
1. The change in teacher value added for all teachers before and after each of the 3 rounds of recognition.
2. The change in student attendance for all teachers before and after each of the 3 rounds of recognition.
3. Finals student grades on a high-stake exam in the end of the school year.
Primary Outcomes (explanation)
Teacher value added is given by the gap between the first grade and a weighted average of all the subsequent grades of a student, where the final grade is given a weight of 50% and for all other intermediate grades, the remaining weight is equally distributed. The first grade is considered to be the baseline performance of a student, in the beginning of an academic year.

For each teacher, the value added will be a weighted average of all the individual student level added values since the beginning of the academic year. Those teachers who are in the top 25% highest TVA within their subject will qualify for recognition and will be publicly praised. More details about how teacher value added is calculated throughout the experiment can be find in the attached "Experiment Design and Analysis Plan".

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design

SETTING

The experiment focuses on teachers in 39 schools coming from 15 different regions of Romania. The teachers used in this experiment are those that teach the 9 academically relevant subjects for which students have to undertake exams: mathematics, native language, English language, physics, biology, chemistry, computer science, history and geography. All the schools in this experiment have voluntarily purchased the rights to use an on-line education platform which allows teachers to interact more efficiently with students and parents. While the schools using this environment might not be fully representative of the average school (they have better performing students on average), there will be no selection into treatment since the schools are not aware of being part of a field experiment.

The usage of the platform comes at a small monthly cost which is covered by the parents of students. The main features on the platform are the ability to post grades and attendance on-line, and communicate in real-time through messages sent between all parties involved.

Access to this data allows me to monitor the performance of all students and teachers in the school for an entire academic year. In particular, I can observe the (i) grades and (ii) attendance of all the students in the school, which will be used as the main performance measures for teachers.

Schools will be randomly assigned to either treatment and control, and they will be stratified on (1) the baseline performance of the students at the school level, (2) the average teacher value added at the school level and (3) the size of the school. The baseline performance of the school is given by an average of the first grade that all the students in the school receive in the beginning of the year. The size of the school is given by the number of teachers employed at the unit.

Schools which are assigned to the treatment group will receive "public recognition". More precisely, a sub-sample of the "best performing teachers" will be publicly praised through a message posted on-line through the education platform. The first round of recognition will be unannounced, and it will come as a surprise. The first round of intervention will announce that public praise will be given in the future, throughout the rest of the year, to ensure that all teachers have the same expectations. Subsequent rounds of recognition will be given at regular time-intervals until the end of the academic year.

QUALIFYING FOR RECOGNITION

Deciding which teachers are top-performers can be tricky since schools differ in quality and there is no official standard requirement from teachers. Since the recognition is publicly given, it is important that it is perceived as deserved by all observers.

Top performers will be determined on the basis of student grade improvement, referred henceforth as Teacher Value Added (TVA).
Those teachers who are in the top 25% highest TVA within their subject will qualify for recognition and will be publicly praised. In this experiment, praise will be publicly given to teachers, such that everyone in the school, including students and their parents, will be able to observe the top performers. Due to this design feature, randomization will be done at the school level. As such, in the sample of 39 schools, a random half will be assigned to treatment and a random half will be assigned to control.

RANDOMIZATION

Since schools differs somewhat in the sample, stratification on some characteristics is important. The three main strata to be considered are (1) student quality, (2) teacher quality and (3) school size.

Firstly, because there is a positive selection of the best students into higher quality schools, the initial grade students receive in the beginning of the year is a good proxy for this selection effect. To account for differences in the underlying quality of the students at each school, the randomization will be stratified by the baseline performance of the school. Baseline performance is an average of all the initial grades of the students across the 9 academic subjects of interest.

Secondly, since schools can hire teachers with different quality levels on average, the distribution of (baseline) teacher value added across schools is another important factor. The average teacher value added at the school level will be the second characteristic on which the sample will be stratified.

Furthermore, I will also stratify the sample on the size of the school.

INTERVENTION

In the schools which qualify for treatment, all teachers, students and parents will be able to see on the school's electronic messaging board a message praising the top performers in the school. For the exact phrasing on the message see the attached document titled "Experiment design and analysis plan"
Experimental Design Details
Randomization Method
Randomization done using STATA
Randomization Unit
Randomization is done at the school levels, stratified according to:
-baseline performance of the students
-baseline performance of teachers
-school size
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
The treatment will be clustered at the school level. There are 39 schools in the sample.
Sample size: planned number of observations
850 teachers and 20.000 students.
Sample size (or number of clusters) by treatment arms
There are 21 schools in the treatment group and 18 schools in the control group.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
The minimum detectable effect size for our main outcomes are in the attached document "Experiment design and analysis plan".
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials