Back to History

Fields Changed

Registration

Field Before After
Abstract This study consists of a randomized evaluation of several programs to encourage academic achievement among upper-primary school students (grades 4-8) in rural Malawi. In each of the two years of the study different programs are being evaluated. In Year 1, two scholarship programs are compared with a control group. The first is a “relative” merit-based scholarship program which provides rewards to students based on performance relative to a comparison group with similar baseline test scores. The second is a standard merit-based scholarship program in which students with the top overall test scores receive a scholarship. In addition to the scholarship programs, information on a student's rank is randomly provided to a randomly selected half of the students to examine the impact of providing this additional information.. In the second year, the relative merit-based scholarship program is being evaluated alongside a program that provides after-school tutoring services to randonly-selected students. The researchers will estimate the impacts of and complementarities between the two programs, as well as the effects of pre-existing social networks effectiveness of the tutoring services. This study consists of a randomized evaluation of several programs to encourage academic achievement among upper-primary school students (grades 4-8) in rural Malawi. In each of the two years of the study different programs are being evaluated. In Year 1, two scholarship programs are compared with a control group. The first is a “relative” merit-based scholarship program which provides rewards to students based on performance relative to a comparison group with similar baseline test scores. The second is a "standard" merit-based scholarship program in which students with the top overall test scores receive a scholarship. In addition to the scholarship programs, information on a student's rank is randomly provided to a randomly selected half of the students to examine the impact of providing this additional information. The researchers will estimate the impacts of each type of scholarship program on student test scores as well as impacts of and interactions with the information treatment. In the second year, the relative merit-based scholarship program is being evaluated alongside a program that provides after-school tutoring services to randomly-selected students. The researchers will estimate the impacts of each program on student test scores and complementarities between the two programs. The researchers will also examine how pre-existing social networks mediate the effectiveness of the interventions.
Last Published May 03, 2016 10:04 AM September 20, 2016 10:44 AM
Intervention (Public) There is a different mix of interventions in each year. In 2014-2015, the main interventions consisted of two merit-based scholarship programs and feedback on student rank.. The first type of scholarship was a "relative merit-based" scholarship in which students were awarded a scholarship in the June 2015 standardized tests in six subjects, based on their performance relative to a reference group. The reference group consisted of bins of 100 students categorized by score on the December 2014 standardized tests. The top 15 students in the June 2015 final exam in each bin received a scholarship consisting of a cash award of 4500 Malawian Kwacha (approximately $6.60). Instead of the cash award, students could also choose from a set of school supplies whose value were similar to the cash award. The second type of scholarship was a standard merit-based scholarship program in which the top 15 percent of students on the final exam were given the award. In addition, a randomly selected half of the students were provided feedback in the form of their test scores on the March 2015 mid-term exam. In 2016, the main interventions consist of the relative merit-based scholarship program, as described above, and an after-school mathematics tutoring program. The relative merit-based scholarship program uses the March 2016 exam scores to form bins of similar students. The final exam is conducted in July 2016. During the second year, only the mathematics portion of the test is considered for the incentive. The mathematics tutoring program consists of after-school tutoring for a subgroup of students between May and June 2016. Tutors are recruited, trained, and supervised by Africa Future Foundation. Half of students in selected grades will be eligible for tutoring, with the remaining students eligible for tutoring in subsequent semesters. Students eligible for tutoring will by randomly assigned to tutoring groups of sizes of 4, 6, 8, or 10. There is a different mix of interventions in each of the two years of the study. In 2014-2015, the main interventions consist of two merit-based scholarship programs and feedback on student rank. The first type of scholarship is a "relative" merit-based scholarship in which students are awarded a scholarship in the June 2015 standardized tests in six subjects, based on their performance relative to a reference group. The reference group consists of bins of 100 students categorized by score on the December 2014 standardized tests. The top 15 students in the June 2015 final exam in each bin receive a scholarship consisting of a cash award of 4500 Malawian Kwacha (approximately $6.60). Instead of the cash award, students can also choose from a set of school supplies whose value are similar to the cash award. The second type of scholarship is a "standard" merit-based scholarship program in which the top 15 percent of students on the final exam across schools are given the award. In addition, a randomly selected half of the students are provided feedback in the form of their test scores on the March 2015 mid-term exam. In 2016, the main interventions consist of the relative merit-based scholarship program, as described above, and an after-school mathematics tutoring program. The relative merit-based scholarship program uses the March 2016 exam scores to form bins of 100 similar students each. The final exam is conducted in July 2016. During the second year, only the mathematics portion of the test is considered for the incentive. The mathematics tutoring program consists of after-school tutoring for a subgroup of students between May and June 2016. Tutors are recruited, trained, and supervised by Africa Future Foundation. Half of students in selected grades are eligible for tutoring, with the remaining students eligible for tutoring in subsequent semesters. Students eligible for tutoring will by randomly assigned to tutoring groups of sizes of 4, 6, 8, or 10.
Primary Outcomes (End Points) In the first year, the main outcomes are test scores and rank on the June 2015 final exams. In the second year, the main outcomes are mathematics test scores and rank on the July 2016 final exams. Test scores and rank on other tests are also outcomes of interest in the second year. Additional outcomes in both years consist of student attendance and survey-based outcomes on study effort, motivation, and additional cognitive testing. In the first year, the main outcomes are test scores and rank on the June 2015 final exams. In the second year, the main outcomes are mathematics test scores and rank on the July 2016 final exams. Test scores and rank on other tests are also outcomes of interest in the second year. Additional outcomes in both years consist of student attendance and survey-based outcomes on study effort, motivation, and additional cognitive testing, as well as measures of teacher effort. Finally, we will examine peer effects of the interventions through self-reported peer networks (taken at baseline) as well as within classrooms. Initially, we plan to examine heterogeneity by baseline exam score, teacher quality, and size of tutoring classes.
Experimental Design (Public) In 2014-2015, the relative merit-based and merit-based scholarships were randomized across 118 grades (5-8) within 31 schools. The feedback was randomized individually to half of the students within these schools and grades. The grades were assigned as follows: Group 1 (46 grades): Merit-based scholarship based on final exam rank, feedback of the midterm exam rank given to half of the students. Group 2 (42 grades): Relative merit-based scholarship based on final exam rank, feedback of the midterm exam rank given to half of the students. Group 3 (30 grades): No scholarship, feedback of the midterm exam rank given to half of the students. In 2016, 122 grades (4-7) in 31 schools are randomized as follows: Group 1 (21 grades): No scholarship, no tutoring Group 2 (40 grades): No scholarship, tutoring randomly assigned to half of students Group 3 (21 grades): Scholarship, no tutoring Group 4 (40 grades): Scholarship, tutoring randomly assigned to half of students Students eligible for tutoring will be randomly assigned to tutoring groups of sizes of 4, 6, 8, or 10. In 2014-2015, the relative merit-based and merit-based scholarships are randomized across 118 grades (5-8) within 31 schools. The feedback is randomized individually to half of the students within these schools and grades. The grades are assigned as follows: Group 1 (46 grades): Merit-based scholarship based on final exam rank, feedback of the midterm exam rank given to half of the students. Group 2 (42 grades): Relative merit-based scholarship based on final exam rank, feedback of the midterm exam rank given to half of the students. Group 3 (30 grades): No scholarship, feedback of the midterm exam rank given to half of the students. In 2016, 122 grades (4-7) in 31 schools are randomized 1) to either scholarship or no scholarship and 2) to tutoring or not tutoring. Within classrooms assigned to tutoring, half of the students are eligible for tutoring. Group 1 (21 grades): No scholarship, no tutoring Group 2 (40 grades): No scholarship, tutoring randomly assigned to half of students Group 3 (21 grades): Scholarship, no tutoring Group 4 (40 grades): Scholarship, tutoring randomly assigned to half of students Students eligible for tutoring will be randomly assigned to tutoring groups of sizes of 4, 6, 8, or 10.
Randomization Unit Year 1: Relative merit-based scholarships, merit-based scholarships, and the control group were assigned at the grade-school level. Whether the student received information on relative ranking in the class was assigned at the student level. Year 2: Relative merit-based scholarships are assigned at the grade-school level. Tutoring class is assigned to grades, then to randomly selected students within selected grades. Year 1: Relative merit-based scholarships, merit-based scholarships, and the control group are assigned at the grade-school level. Whether the student received information on relative ranking in the class is assigned at the student level. Year 2: Relative merit-based scholarships are assigned at the grade-school level. Tutoring class is assigned to grades, then to randomly selected students within selected grades.
Sample size (or number of clusters) by treatment arms Year 1: 46 grades merit scholarship, 42 grades relative merit scholarship, 30 grades control. 4000 students in feedback treatment, 4000 students not in feedback treatment. Year 2: Group 1 (21 grades): No scholarship, no tutoring Group 2 (40 grades): No scholarship, tutoring randomly assigned to 1/3 of students (approximately 1300 students with tutoring, 1300 without) Group 3 (21 grades): Scholarship, no tutoring Group 4 (40 grades): Scholarship, tutoring randomly assigned to 1/2 of students (approximately 1300 students with tutoring, 1300 without) Year 1: 46 grades merit scholarship, 42 grades relative merit scholarship, 30 grades control. 4000 students in feedback treatment, 4000 students not in feedback treatment. Year 2: Group 1 (21 grades): No scholarship, no tutoring (approximately 1300 students) Group 2 (40 grades): No scholarship, tutoring randomly assigned to 1/2 of students (approximately 1300 students with tutoring, 1300 without) Group 3 (21 grades): Scholarship, no tutoring (approximately 1300 students) Group 4 (40 grades): Scholarship, tutoring randomly assigned to 1/2 of students (approximately 1300 students with tutoring, 1300 without)
Back to top