The effects of non-financial incentives on the retention and performance of volunteer tutors

BRAC introduced the Chhatrabandhu (CB) Program in 2007 with the primary objective of improving the educational outcomes of disadvantaged secondary school students in rural Bangladesh. The CB program is a nationwide program that involves more than 40,000 volunteer tutors and 1,400 schools in 55 districts across Bangladesh. In 2015, we embed a large-scale randomized controlled experiment in the flagship CB program to examine the effects of offering non-financial incentives on volunteers’ dropout rates and performance. Our experiment involves 4,162 volunteer tutors who provide free after-school tutoring services in mathematics and English to more than 5,000 underprivileged secondary students in 495 schools. The volunteer tutors in our experiment are assigned randomly into one of three treatment groups several months after they have signed up to the CB program. In the first group, tutors are offered a certificate of excellence awarded in a public ceremony conditioning on their students having achievement gains. If their students do not experience achievement gains, they are awarded a certificate of appreciation in private. In the second group, tutors are also offered a certificate of excellence awarded conditioning on their students having achievement gain, but the certificate is awarded privately. If their students do not experience achievement gains, they are awarded a certificate of appreciation in private. In the third group, tutors are awarded a certificate of appreciation (regular certificate) in private regardless of their students’ performance. This regular certificate is BRAC’s status-quo policy for all volunteer tutors who complete the CB program, including those who work with schools that are not part of our field experiment.

2015-04-20

2016-08-31

We examine the effects of non-financial incentives on volunteers’ dropout decision and their students’ performance. We focus on three primary outcomes: (1) whether a volunteer has dropped out of the program before the final survey (right after students take the national public examination, in-school examination, and standardized tests); (2) students’ mathematics and English performance in the grade eight national public examination or grade seven in-school examination; and (3) students’ mathematics and English performance in standardized tests designed and administered by BRAC's Research and Evaluation Division.

Dropout is an indicator variable that takes the value of one if a volunteer has dropped out of the tutor program before students take the national public examination or in-school examination.

Students’ performance in the grade eight national public examination or grade seven in-school examination is measured using the grade point in the relevant subject (mathematics or English).

Students’ performance in the standardized test is measured using the raw score in the relevant subject (mathematics or English).

In April 2014, we selected 495 non-government secondary schools from a list of schools in which BRAC was introducing the CB program for the first time to participate in our experiment. The selected schools are located in 49 districts (78 sub-districts) in which BRAC has the local resources to implement the experiment. From these 495 schools, our experiment includes 4,162 volunteer tutors (CBs). Between June and August of 2014, these volunteer tutors are matched with 5,730 grade 6 or 7 students from disadvantaged families.

Between January and February of 2015, we conduct a baseline survey on the CBs in our experiment. The survey collects information about their demographic, social, and economic characteristics, past academic achievement, as well as motivations/reasons for joining the program. The baseline survey also collects information about CBs’ students, such as the students’ performance in the grade 5 national public examination and the students’ socio-economic status.

We then assign each of the 495 schools randomly to one of three treatment groups. The three treatment groups are: (T1) performance-contingent public-recognition certificate group; (T2) performance-contingent private-recognition certificate group; and (T3) participation-based certificate group. In T1, tutors are offered a certificate of excellence awarded in a public ceremony if any of their students experiences gains in average grade points between grade-8 (or grade-7) and grade-5 examinations in mathematics and English. If their students do not experience achievement gains, they are awarded a certificate of appreciation in private. In T2, tutors are also offered a certificate of excellence awarded conditioning on their students having achievement gains, but the certificate is awarded privately. Similarly, if their students do not experience achievement gains, they are awarded a certificate of appreciation in private. In T3, tutors are awarded a certificate of appreciation in private regardless of their students’ performance.

We announce the treatments to the CBs around April of 2015 by mail and phone. CBs are not informed about the other treatments they are not assigned to. Because randomization is at the school level and secondary schools in rural Bangladesh are generally located far from each other, the possibility of information spillover is low.

In late 2015, we collect information about the dropout status of volunteer tutors from the local BRAC offices. In early 2016, we conduct a final survey on 3,012 CBs, while BRAC RED team administers mathematics and English standardized tests on 3,024 randomly selected students of CBs. Between March and May 2016, we collect information about students’ performance in the in-school and national public examinations.

Randomization is performed in office with the help of a computer and STATA.

Randomization was done at the school level. We ensure that average characteristics of volunteers and students are balanced across treatments.

Yes

495 schools (165 schools per treatment category).

4,162 volunteer tutors and 5,730 students.

165 schools in T1
165 schools in T2
165 schools in T3.
Approximately 1,400 CBs and 1,900 students per treatment arm.

Analysis Plan