x

Please fill out this short user survey of only 3 questions in order to help us improve the site. We appreciate your feedback!
Private Recognition and Teacher Performance
Last registered on April 25, 2017

Pre-Trial

Trial Information
General Information
Title
Private Recognition and Teacher Performance
RCT ID
AEARCTR-0002030
Initial registration date
April 25, 2017
Last updated
April 25, 2017 2:02 PM EDT
Location(s)
Region
Primary Investigator
Affiliation
Erasmus University
Other Primary Investigator(s)
PI Affiliation
Erasmus University Rotterdam
PI Affiliation
Erasmus University Rotterdam
Additional Trial Information
Status
In development
Start date
2017-03-03
End date
2018-08-25
Secondary IDs
Abstract
Does providing recognition to well-performing teachers make them better teachers? We study this issue using a randomized controlled trial. Following midterm assessment of teachers by students, we give private recognition to a random subset of teachers who score well (above an ex-ante decided threshold). Recognition is given during a personal phone call between the teacher and a representative of the employer. In addition to the average treatment effect, we also examine some of the mechanisms that can drive the treatment effect. For that purpose, we use questionnaires and incentivized games to collect data on teachers' reciprocity (several types), conditional altruism towards the employer, and reputation concerns.
External Link(s)
Registration Citation
Citation
Cotofan, Maria, Robert Dur and Max van Lent. 2017. "Private Recognition and Teacher Performance." AEA RCT Registry. April 25. https://doi.org/10.1257/rct.2030-1.0.
Former Citation
Cotofan, Maria et al. 2017. "Private Recognition and Teacher Performance." AEA RCT Registry. April 25. http://www.socialscienceregistry.org/trials/2030/history/16882.
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details
Interventions
Intervention(s)
Our treatment is the provision of private recognition to a random sample of teachers that qualify for it. Teachers qualify for recognition if they scored above a certain threshold on intermediate student evaluations. All teachers are informed about their scores on the intermediate evaluations before the treatment is administered.
Intervention Start Date
2017-03-13
Intervention End Date
2018-08-25
Primary Outcomes
Primary Outcomes (end points)
1. The change in the student evaluation score for each individual teacher that qualifies for private recognition (before and after the treatment is administered)

2. The change in the number of absences of students recorded by each teacher qualifying for recognition, before and after the treatment is administered.

3. The course grades of students. For this outcome variable we also control for past performance of the students as a proxy for their initial ability.
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
This experiment will have three main stages which we describe below.

1. The questionnaire stage
A survey is conducted among all teachers consisting of questions whose purpose is to record levels of direct and indirect reciprocity, conditional altruism and self-worth/reputation concerns.

2. The field experiment stage
In the end of the second week of the course, the students will be asked to evaluate and rate all the teaching assistants based on their teaching skills.

In the third week of the course, private recognition will be provided to those teachers that score sufficiently high on the intermediate evaluations.

After the final exam, students will be asked again to evaluate their teachers. The grades on all the final exams for the course will be recorded as well for all courses involved. Differences between (i) intermediate and final evaluation forms, (ii) changes in the attendance and (iii) the final grades of students will be used as outcome measures in evaluating the average treatment effect of the intervention. Heterogeneous treatment effects will be explored based on the teachers' preferences we elicited using both surveys and incentivized games.

3. The lab experiment stage

After the treatment is administered for the teachers of interest, a subset of the sample will be invited to also play a number of incentivised games with the purpose of eliciting the same preferences that were self-reported in the survey. Also for ease of future replications, the games will be based on standard measures in the literature designed to elicit the preferences of interest: direct reciprocity, indirect reciprocity, conditional altruism, reputation concerns.

Experimental Design Details
This experiment will have three main stages which we describe below. 1. The questionnaire stage One week before the start of the course, a survey is conducted among all teaching assistants. The survey consists of questions whose purpose is to record the preferences of the subjects. Specifically, self-reported levels of direct and indirect reciprocity, conditional altruism and reputation concerns/self-worth are recorded. The teaching assistants will be told that the survey is part of a research project on-going at the university, with the purpose of studying how people make decisions. 2. The field experiment stage In the end of the second week of tutorials the students will be asked to evaluate and rate all the teaching assistants based on their teaching skills. In the third week of the course, a representative of the employer will make a phone call to both treated and control. During the phone call she will ask the respondents about their experience with some of the services offered by the university to its teaching assistants. At the end of the phone call, those in the treatment group will receive an unexpected message of recognition for their performance. After the final exam, students will be asked again to fill in the evaluation forms for the teaching assistants that were in charge of their tutorials. The grades on the exams for the course will be recorded as well for all other courses the students attend that block. Differences between intermediate and final evaluation forms and changes in the presence in the tutorial will be used as outcome measures in evaluating the average treatment effect of the intervention. Heterogeneous treatment effects will be explored based on the different preferences stated in the surveys by teaching assistants. 3. The lab experiment stage After the treatment is administered for the teachers of interest, during a festive gathering organized by the university for all the teaching assistants, all those who taught a course in the past academic year will be invited to also play a number of incentivised games with the purpose of eliciting the same preferences that were self-reported in the survey. We expect that only a sub-sample of those invited will actually show up. For ease of future replications, the games will be based on standard measures in the literature designed to elicit the preferences of interest: direct reciprocity, indirect reciprocity, conditional altruism, reputation concerns. The experiment will take roughly 30 minutes and the participants will undertake it using their mobile phones, following a link that will be sent to their e-mail addresses by the experimenters. The design of the experiment is closely related to the methodology implemented by Stanca (2009) in his paper "Measuring indirect reciprocity: Whose back do we scratch". The paper defines 3 types of reciprocity: direct (DR), indirect generalized reciprocity (IGR) and indirect social reciprocity (IRS). IGR is is defined as adopting a helpful action towards someone else, at one's own material cost, because some other person's intentional behavior was perceived to be helpful to oneself (if A helps B, then B helps C) which corresponds to our definition of indirect reciprocity (henceforth IR). IRS is defined as adopting a helpful action towards someone else, at one's own material cost, because that person's intentional behavior was perceived to be helpful to some other person (if A helps B, then C helps A) which corresponds to our definition of conditional altruism (henceforth CA). In the beginning of the experiment, besides a show-up fee the participants will be given an endowment of n tokens. The experiment will take place in three rounds, each design to test one of the mechanisms of interest. In each round participants will be given either the part "A" or the part "B", each with a probability of a half. Participants will be instructed that in the beginning of each round they will randomly be matched in pairs (A,B). The matching will only last for one round (a one-shot interaction) and their identity will remain anonymous in order to prevent strategic choices. All subjects A_i will be told that they can choose an amount a_i, an integer between 0 and n, that they can send to the player B_i with whom they are matched, in the beginning of each round. The amount will be subtracted from A_i's payoff, multiplied by 3 and added to the payoff of B_i. Round 1 (DR): Before being informed about the amount a_i, player B_i will be asked to provide the amount b_i they want to send back to player A_i for every possible decision in the decision set of A_i, namely alpha=(0, 1, 2,.....n). In the end, B_i is informed about the amount a_i and the corresponding amount b_i is multiplied by 3 and sent back to player A_i. Round 2 (IR): Before being informed about the amount a_i sent by player A_i, player B_i will be asked to provide the amount b_i they want to send to another player A_j for every possible decision in the decision set of A_i. B_i is informed about the amount a_i and the corresponding amount b_i is multiplied by 3 and added to the payoff of player A_j. Round 3 (CA): Player B_i will be asked to provide the amount b_i they want send to another player A_j for every possible decision in the decision set of A_j regarding the number of tokens they chose to send to another player B_j. B_i is informed about the amounts a_i and a_j and the corresponding amount b_i is multiplied by 3 and added to the payoff of player A_j. To isolate the "reputation concerns" mechanism using monetary incentives we decided to offer all respondents the opportunity of spending some of their show-up fee in order to buy a certificate stating that they were teaching assistants during the academic year. The respondents will have three options: (i) not spending anything, (ii) spending a very small amount of money to receive a print-out of the certificate or (iii) spending some extra money in order to receive a framed copy of the certificate. We believe that for labor market purposes, neither (ii) nor (iii) are really needed since the teachers can just add this information to their CVs without paying for it. As such, choosing (ii) or (iii) as an option captures some of the self-worth and/or reputation concerns that respondents might have (since the -framed- certificate also offers the possibility of displaying one's achievement), with (iii) signaling a higher reputation concern than (ii). At the end of the academic year, respondents will be asked to answer a couple of questions designed to understand the relationship between teaching assistants and the frequency with which they communicate and share information with each other. Furthermore, an open question will be asked to see if teaching assistants were "unhappy" about any other issues regarding the course. The results to this question will be used as a robustness check to ensure that spillovers between treatment and control do not occur, such that individuals in the control group found out that they would qualify for recognition, but they nonetheless did not receive it. Also, two questions regarding how they perceive the importance of their job will be asked. This can help us study yet an additional theoretical channel: if some teaching assistants are not aware about the fact that their work is very important to students, they might put little effort in. However, since this is revealed through the treatment, effort can simply increase through an update in the beliefs about the importance of their teaching to students. As such, teaching assistants will be asked "how important they think their job is", and whether "the Tutor Academy has contributed during the course to the perceived importance of their job".
Randomization Method
Randomization done using STATA.
Randomization Unit
Randomization is going to happen at the teacher level based on the following characteristics
-The course each teacher teaches
-The language in which the teacher teaches (English or Dutch)
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
Approximately 200 teachers
Sample size: planned number of observations
Approximately 200 teachers
Sample size (or number of clusters) by treatment arms
There will be approximately 100 teachers in the treatment and 100 teachers in the control group.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
The minimum detectable effect (MDE) size for our main outcomes are: See the attached document "Detailed experimental design and analysis plan".
Supporting Documents and Materials

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
Education Management Office, Erasmus School of Economics
IRB Approval Date
2017-01-12
IRB Approval Number
/
Analysis Plan

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
No
Is data collection complete?
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports, Papers & Other Materials
Relevant Paper(s)
REPORTS & OTHER MATERIALS