Scaling Teacher Performance Pay Contracts in Rwanda

The study evaluates five designs for scaling teacher incentive contracts. These are based on a core design "Model A", with each of four variations changing one dimension of the design at a time.
The base design, Model A, incentivizes teachers based on "Five Ps": Presence, Preparation, Pedagogy, Participation in CAMIS, and Pupil Learning. Presence and preparation are measured by Sector Education Inspectors (SEIs) during unannounced visits, with the latter referring to the completion of lesson plans. Pedagogy is measured by SEIs during an announced visit, using the Government of Rwanda's structured classroom observation procedure. Participation in CAMIS refers to the use of the Comprehensive Assessment Management Information System to record end-of-term marks for students on the Comprehensive Assessment, and is observed using the CAMIS database itself. Finally, pupil learning is defined as learning outcomes relative to others of similar baseline achievement levels (Barlevy and Neal, 2012) using the end-of-year Comprehensive Assessment in each year as the basis for learning assessment. Audits of 5 percent of pupils' recorded exam scores, using paper records in schools, will be used to enforce fidelity of the exam system. Teachers are ranked within districts on each metric, and the teacher's evaluation score is a weighted average of these components, with measures of teacher inputs comprising 50 percent of the total weight, and the pupil learning metric comprising the remaining 50 percent. Teacher payoffs are a linear function of their percentile rank on this composite metric, ranging from zero to five percent of salary.
The four variations modify this design as follows: Variation B1 adds head teachers' subjective assessment of teachers' activities to the set of "input" measures. Variation B2 provides termly feedback on the performance score, using end-of-term Comprehensive Assessment data to produce projected learning growth measures. Variation B3 replaces the linear payoff function with a three-tiered, stepwise function that provides no bonus to those below the 25th percentile, a bonus of 3 percent of salary to those between the 25th and 75th percentile, and a bonus of 5 percent to those in the top quartile. Variation B4 augments the audit scheme by retesting 5 percent of pupils.

2023-01-19

2025-07-15

The study has five primary outcomes, corresponding to the incentivized dimensions of teacher performance:
- Pupil learning;
- Teacher presence at the school;
- Teacher pedagogy;
- Teacher preparation; and
- Teacher use of CAMIS for end-of-term and end-of year assessments.

Primary outcomes correspond to the incentivized measure of teacher performance.
In the cases of outcomes (1) - (4), these are based on low-stakes measures collected by Innovations for Poverty Action (and not the high-stakes versions of these used for teachers' imihigo and collected separately by government officials). As described below, high-stakes versions of these measures will be analyzed as secondary outcomes.

In addition to the primary outcomes above, we will estimate and test for impacts on the following secondary outcomes:
- High-stakes analogs of all (low-stakes) primary outcomes (with the exception of CAMIS data completion);
- Teacher retention;
- Student drop-out;
- Number of pupils unsupervised at time of surprise visit;
- Teacher satisfaction; and
- Head teacher activities

Schools are included in the study sample on the basis of having lower- and upper-primary comprehensive examination data submitted for the 2021-2022 school year. There are 481 study-eligible schools, comprising 120 sectors in the 10 study districts.
Assignment is conducted at the "sector" level; there are 62 treated sectors, containing 248 treated schools, and 39 control sectors, containing 95 treatment-eligible control schools.

Not available

Active-treatment sectors were selected by simple randomization, stratified within districts. Among active-treatment sectors, 20,000 potential assignments of treatment were generated, stratifying assignment by district. Potential assignments were deemed "feasible" if they met criteria for balance in the shares of schools treated across treatment arms (given clusters have heterogeneous sizes). The implemented treatment assignment was selected by simple random selection from among feasible assignments.

Assignment is conducted at the "sector" level, a sub-district geo-political unit.

Yes

There are 120 study-eligible clusters ("sectors"). Of these, 101 sectors (343 schools) are selected for low-stakes measurement, though administrative-data outcomes will be observed in all clusters.

Low-stakes measurement will take place in 343 schools. All lower-primary teachers in these schools will be observed for presence, preparation, and pedagogy. A sample of 30 pupils per grade will be assessed in low-stakes measurements of learning outcomes.

Low-stakes measurement schools: 39 sectors (95 schools)
Base design (version A): 14 sectors (54 schools)
Variation B1: 12 sectors (48 schools)
Variation B2: 13 sectors (48 schools)
Variation B3: 12 sectors (57 schools)
Variation B4: 11 sectors (41 schools)