Incentives: Evidence from Uganda

I implement two types of social comparative feedback regimes, within and across-class group comparisons, and two types of incentive regimes, financial and reputation rewards. I also allow for their interaction. In total, eight different treatment groups are formed. More than 5000 primary and secondary school students in Uganda, who were repeatedly tested and interviewed over one academic year. In total, five testing rounds were administered between January and December 2012.

2012-01-01

2012-12-31

Performance in Mathematics and English, students' expectations about their performance in Mathematics and English collected before and after every testing (which happened 5 times in total), students' subjective levels of effort exerted before and after every testing, students' subjective happiness before and after every testing, and students overall happiness and stressed measured by standard psychological questionnaires. Finally I have data about friends of each student within his/her class before and after the experiment.

Students were randomly assigned to two feedback treatment groups and a control group. Students in the within-class feedback group (within) were randomly divided into units of three to four classmates and evaluated as units (using the unit averages) within their respective classes. Students in the within group received feedback about their own performance, the performance of their unit members, and the relative position of their unit within their class. Moreover, they received detailed information about whether they and the position of their unit have improved or worsened from one testing round to the next.
Students in the across-class feedback group (across) were evaluated as a class (using the class average) and were compared to other classes of the same grade in the district. Students in the across group received feedback about their own performance, the average performance of their class, the relative position of their class with respect to other classes, and whether they and/or their class have improved or worsened between two subsequent testing rounds
Students in the control group received no information but only took exams. In order to motivate all students to participate in the experiment, we began our visits reminding students that the exams serve as an additional practice for national leaving examinations (which are compulsory for seventh graders in primary schools and fourth and sixth graders in secondary schools).
In order to study the effects of monetary and non-monetary rewards, students were orthogonally re-randomized at the school level into a tournament for financial or reputational rewards. Re-randomization took place after round 4 and students had no information about tournaments beforehand. The qualification criteria differed based on initial randomization into treatments but the general rule was to reward the 15% best performing students/units/classes, and the 15% most improved students/units/classes. In order to avoid confusion, students were given exact information regarding the number of winning students/units/classes. Therefore, all students, despite the class size or treatment allocation, had the same probability of winning. Students in the monetary reward treatment groups could win 2,000 UGX (about .80 USD). Students in the reputational reward group could receive a certificate and their names were announced in the most popular local newspaper in the region, Bukedde.
Overall, the orthogonal randomization divided the sample into 9 groups – one control group, four sole treatment groups (i.e., one type of treatment only) and four combined treatment groups (two types of feedback interacted with two types of rewards). Such cross-cutting design allows me to compare the impact of feedback and reward incentives on students’ performance expectations and their well-being, and to study the complementarities of feedback and rewards.

In order to increase the balance between control and treatment groups, the sample was stratified along three dimensions: school area (the sample divided into four areas differing by level of remoteness), average school performance in national testing (above average or below average), and student level (grade 6 and 7 of primary education, and grades 1 to 4 of secondary education). The randomization was done in two stages. First, after the stratification of the sample by school performance and area, I randomized the whole sample of 53 schools into treatment and control groups in a 2:1 ratio (which resulted into 36 (17) schools being selected into treatment (control) group). In the second stage, I divided classes of the treatment schools randomly into within-class feedback and across-class feedback groups in a 1:1 ratio (class-level randomization). In this scenario, no student in a control-group school received any treatments, and students in the treatment group schools received either within- or across-class feedback depending on the type of the intervention their class was randomized into. Exposure to the treatment is the only difference in the outcomes between the control and treatment groups.

I did two-stage randomization, first at the school level (to randomize the schools into feedback-treatment and control groups) and then at the class level I randomized classes into within group versus across group feedback treatment groups.

No

In total I had 53 schools in my sample - 31 primary and 22 secondary schools. In primary schools I included students in the 6th and 7th grades, while in the secondary schools I included students in the "O-level", which means from grade 1 to grade 4. In total, 150 classes participated.

I had 7,209 students who participated in the first testing round. The number changes in each round due to new admissions and high absence and drop-out rates.

There are 17 schools in the control group, 36 in the treatment group (combining within-class and across-class feedback treatment groups). In terms of classes, there are 48 classes in the control group, 50 in the within-class feedback group and 52 in the across-class feedback group. In terms of students, based on the number of students present in the baseline testing, I had 2,343 students in the control group, 2,395 among the within-class feedback, and 2,412 among the across-class feedback group.

The minimum detectable effect size equals 0.15.

In this paper, I study persistency of overconfidence. I bring evidence from a large-scale field experiment (N=5,102) on whether providing detailed repeated feedback to students evaluated in groups in different types of tournaments helps them predict their performance more accurately. Students in my experiment are overly overconfident and group feedback only helps them offset their inflated beliefs created by task repetition. Feedback exhibits diminishing returns to information. Improving self-assessment is associated with higher performance but lower happiness. Finally, inability to accurately assess own performance may serve as an additional channel to explain gender differences in Math performance.

Katreniak Celik, Dagmara. 2019. “Persistent overconfidence: evidence from a field experiment in Uganda.” Working paper.

https://docs.google.com/viewer?a=v&pid=sites&srcid=ZGVmYXVsdGRvbWFpbnxkYXNhY2VsaWt8Z3g6MzI3MmUxMDhlMGEzZWM2Yg

Throughout our lives, we are routinely offered different incentives as a way to motivate us, such as absolute and relative performance feedback, and symbolic, reputation or financial rewards. Many researchers have studied the effects of one or more of these incentives on how people change their performance. However, there can also be important psychological outcomes in terms of stress and happiness. The current paper contributes to the literature by explicitly accounting for this performance-versus-well-being trade-off introduced by incentives. I implement two types of social comparative feedback regimes, within and across-class group comparisons, and two types of incentive regimes, financial and reputation rewards. The results show that rewards can lead to an increase in students’ performance up to 0.28 standard deviations (depending on whether students received feedback and what type), but at a cost of higher stress and lower happiness, whereas comparative feedback alone (without rewards) increases performance only mildly, by 0.09 to 0.13 standard deviations, but without hurting students’ well-being. More stressed students exert less effort, perform worse and attrite by 29 percent more compared to those who are stressed minimally. Furthermore, the results help to identify gender-specific responses to different incentive schemes. Boys strongly react to rewards with or without feedback. In contrast, girls react to rewards only if they are also provided with feedback. Finally, the paper contributes to the expanding literature on incentivized interventions in developing countries by using a rich dataset of more than 5000 primary and secondary school students in Uganda, who were repeatedly tested and interviewed over a full academic year.

Katreniak Celik, Dagmara. 2019. “Dark side of incentives: evidence from a field experiment in Uganda.” CERGE-EI

https://sites.google.com/site/dasacelik/DarkSideOfIncentives_Celik_JMP.pdf?attredirects=0