Carrots or Sticks: The Impact of Incentives and Monitoring on the Performance of Public Extension Staff

This project proposes to support the work of the Government of Punjab, Pakistan, overall, and the Ministry of Agriculture in particular, in their effort to improve service delivery through improved systems of client outreach and higher quality service provision by public sector employees. For the Ministry of Agriculture, this entails improving the performance of all staff providing advisory services to farmers—in particular, field staff, who are on the front lines of this effort.

Over the last year, all field staff -- Agriculture Extension Officers (AOs), Agriculture Assistants (AIs), and Field Assistants (FAs) – have been provided a smartphone equipped with a monitoring system (Agri-Smart) to track their location and activities. The Ministry of Agriculture is now putting in place performance based pay incentives using the Agri-Smart system, to reward greater outreach efforts by field staff. The goal is to improve agricultural productivity which is seen as critical for both poverty reduction and growth.

Agricultural service delivery involves complex tasks where effort is multi-dimensional with some dimensions harder to monitor than others. This is a setting where the choice is unclear between the known advantages and disadvantages of objective versus subjective performance evaluation systems. Further, since bias is a common issue in all subjective evaluation systems, there is a potential benefit for increased monitoring of supervisors themselves.

We proposed to the Ministry of Agriculture to evaluate the impact of three alternative performance pay systems which vary in terms of objectivity and alignment of staff incentives with the Directorate.

We propose to implement and evaluate three different bonus schemes for FAs/AIs and AOs against a control group. For all groups T1-T3 plus controls, workers will receive their base salary as under the status quo. In the control group, workers will receive TA/DA payment as they always have: they will file claims for TA/DA without supporting documents or data, and they are not guaranteed payment of the claim. In the three treatment groups, T1-T3, workers will be eligible to receive one out of four possible amounts:
• Level 0: no bonus
• Level 1: base bonus
• Level 2: low bonus
• Level 3: high bonuses

T1: OUTREACH Incentives based on Outreach Measure
The data generated by the Agri Smart System provides valuable information about the activities of FAs and AOs, which can be used to construct transparent and objective measures of outreach on a monthly basis. The different levels of bonus are based on adequacy thresholds set separately for AOs and FAs/AIs to reflect their different roles.

The Agri-Smart system will automatically determine the level of bonus that any agricultural extension worker qualifies for in a given month. This information will appear on a payment request interface which will generate a request for payment to the system as well as an SMS to the staff member informing him/her of the incoming payment.

The advantage of this incentive structure is that performance is transparently measured. It provides a good measure of an important dimension of performance: outreach. However, this measure may be too narrow. In particular, it is unclear how well it can capture variation in the quality of services delivered.

T2: SUBJECTIVE Incentives based on Supervisor’s Assessment of Performance
Within each tehsil, the DDO is responsible for the performance of field staff. As supervisors of FAs and AOs, DDOs have access to relevant information about dimensions of performance that cannot be captured in Agri-Smart and are thus not included in T1. T2 is an incentive scheme based on an evaluation by the DDO using the outreach measures from T1 along with his own subjective assessment.
The DDO can access the Agri-Smart data through a simple dashboard which provides key outreach statistics for each extension worker. Using this information as well as his own assessment of the worker, he will selects a level of bonus for each worker and enter it in the payment interface accordingly. The payment interface will then send the request for payment to the system as well as an SMS to the worker to inform him/her of the incoming payment.

T2 takes advantage of the ability of the supervisor to monitor dimensions of performance that are not easily quantifiable and allows flexibility to account for external factors that affect performance. On the other hand, T2 suffers from the drawbacks of subjective evaluations: lack of transparency and evaluator bias (e.g. favoritism, centrality bias, and leniency bias, discussed in the previous section).

T3: SUBJECTIVE PLUS is T2 + Career Incentives for Supervisors
Relative to T2, T3 increases career incentives for the DDOs by introducing the transmission of information on outcomes to the DDO’s managers: the DG of Agriculture Extension and the Secretary of Agriculture. This monthly report will be based on information in the Agri-Smart database, as well as information drawn from farmer callbacks and crop/season specific data (such as seed quality used, timing of crop planting, input usage, water usage and finally yield) aggregated at the tehsil level.

The advantage of T3 over T2 is closer alignment of incentives among different levels of the agriculture extension system and minimization of bias in the subjective evaluation. This comes at the cost of additional time and financial resources spent on monitoring. For a T3-like scheme to be scaled up, its outperformance over T2 must be sufficient to justify the additional system costs of incentivizing the supervisors. On the other hand, monthly summary reports of tehsil performance may get routinized lowering the marginal cost of implementing T3.

Controls
In tehsils assigned to this group, the existing TA/DA allowances will continue to be disbursed according to the existing claim system.

2017-11-01

2019-01-31

1. Ag Extension worker level outcomes:
a. Outreach measures as measure by the Agri-Smart such as Compliance,
Distance traveled, Farmers reached, Total farmers advised through extension activities in a given month, Proportion of scheduled activities completed; proportion of time spent on extension activities in a given month, Number village visits
b. Job satisfaction, Perception of fairness, Collaboration with colleagues as measured in a survey

2. Farmer level data
a. Farmer feedback on extension services,
b. Farmer Technology adoption
c. Farmer knowledge
d. Agricultural Productivity

3. Tehsil level data
Agricultural productivity, Yield for all major and minor crops.

We proposed to the Ministry of Agriculture to evaluate the impact of three alternative performance pay systems which vary in terms of objectivity and alignment of staff incentives with the Directorate.

Our impact evaluation covers the entire population of field agricultural extension agents (FAs/AIs/AOs) in Punjab:
126 tehsils, with approximatively 20 workers per tehsil.

We randomized the incentive schemes at the tehsil level.

Randomization done in office by a computer.
We stratified by districts.
We make a 100 draws for the allocation and for each draw run the following tests: A. that treatments are balanced across some baseline variables and B. that for each sub-sample of our treatments, we run a joint test of significance for all the variable vars.
We choose the draw with the smallest joint F.

Tehsil level

Yes

126 tehsils

126 tehsils with about 2,524 workers (out of which 382 are AOs)

T1: 31 tehsils
T2: 31 tehsils
T3: 32 tehsils
Control: 32 tehsils