The Organization and Impact of Relational Sourcing on Smallholder Farmers in the Rwanda Coffee Sector

In partnership with a leading coffee exporter, we set up a pilot study in which we design, implement and evaluate the impact of a Farmer Development Program (FDP) in the Rwanda coffee chain.

First, we evaluate the impact of the FDP on the farmers eligible to join the program using a regression-discontinuity-design (RDD). We examine the spill-over of the FDP program on farmers initially not invited to join the FDP program randomizing the share of farmers eligible to join FDP across zones. Additionally, using experimental methods, we examine the differential impact of two relationship-building organizational interventions embedded into the FDP: i) a relationship manager and ii) a communication intervention.

In this pilot project, we roll out the FDP in four washing stations. We plan to roll out the program to additional coffee washing stations based on the learnings.

In this pilot project, we focus on the design and implementation of a Farmer Development Program (FDP) in Rwanda. The FDP program consists out of three in-person sessions, two agricultural sessions, and one relationship-building session. During the relationship-building session, farmers learn about the expectations and benefits associated with the FDP program. Furthermore, the farmers will have access to a remote call-center service that allowed farmers to ask questions and get support.

The design of the evaluation is as follows. First, we empirically examine whether participation in the FDP improves relational practices between farmers and coffee washing stations (CWS). In light of the skewness of farmers’ deliveries to the company and the difficulty of tracing farmers that might only deliver through intermediaries, the implementing partner was keen to invite to FDP farmers for which there was a reasonable chance of successfully building relationships. We therefore surveyed all farmers in the catchment areas of the four washing stations and matched the resulting information with administrative delivery data from the company. We scored farmers based on observed characteristics and randomized the percentages of invited farmers participating in the FDP across zones. The first group consists of 16 zones with 80% invited farmers (high-intensity zones), and the second group covers 16 zones with 40% invited farmers to the FDP (low-intensity zones). This research design permits evaluating i) the impact of being eligible to join the FDP on relational practices between farmers and CWSs through an RDD and ii) the potential spill-overs on the bottom 20% of farmers from the program.

A second objective is to obtain insights into how best organize relationships with farmers. In this pilot project, we roll out two relationship-building elements within the FDP program: i) relationship manager and ii) communication intervention.

** Relationship building intervention: Relationship manager **
In this relationship-building intervention, we evaluate the impact of having a designated relationship manager (RM) deliver the in-person farmer sessions associated with the FDP. We randomized farmers into three groups: i) Fixed RM, ii) Control RM, and iii) Placebo RM stratifying by zone and gender (64 strata)

Treatment 1: Fixed RM
Farmers in this condition have a single designated RM who will be present during all the agricultural sessions and deliver the relationship-building session. Farmers in this condition are invited to four sessions: one introduction session – during which the farmers meet their designated RM -, two agricultural sessions during which the RM was present, and one relationship-building session – during which the designated RM clarifies the FDP and the associated expectations and benefits to the farmers.
The fixed RMs are CWS manager and accountants from the four pilot washing stations. Hence, there might be some prior familiarity (elicited at baseline) between the invited farmers and the fixed RMs.

Control 1: Control RM
Farmers were invited to the two sessions on agricultural practices and invited to the relationship-building session. In this control condition, farmers do not have a single designated RM. Instead, farmers see a different RM during every session. In this condition, the farmers see a fixed relation manager (i.e., their local CWS manager or accountant) during one of the agricultural sessions.

Control 2: Placebo RM
Farmers were invited to the two agricultural sessions and the relationship-building session. To disentangle the impact of attending an additional session (i.e., the introduction session as provided to farmers in the fixed RM condition), farmers assigned to the Placebo RM are also invited to attend a placebo session with a RM. However, similar to the Control RM condition, farmers do not have a single designated RM and see a different RM every session.
The RMs in the control and placebo conditions are CWS managers and accountants in other non-pilot washing stations. While there is no prior-session familiarity between farmers and the RMs, we can ensure similarity in backgrounds to the RMs.

** Relationship building intervention: Communication**
This intervention is cross-randomized with the relationship manager intervention. We randomly assign the FDP farmers to either one of four communication methods used to invite farmers to sessions, and information during harvest season.

1. SMS only:
Farmers in this group serve as a control group and only receive a text message with inviting them to the next session.

2. SMS + Fixed RM Call
Additional to receiving an SMS message, the farmers receive call from the fixed RM to invite them to the next FDP session. As mentioned above, the fixed RM is the local CWS’ manager or accountant and most likely known by the farmer.

3. SMS + Placebo RM Call
Additional to receiving an SMS message, the farmers receive call from the placebo RM to invite them to the next FDP session. The placebo RM is a non-local CWS manager or accountant and is most likely not known by the farmer.

4. SMS + Call from Call Center:
Additional to receiving an SMS message, the farmers receive call from a member of the call center to invite them to the next FDP session. The farmer does not know the call center member.

The underlying objective of these interventions is to gain insight into how better organize the FDP in a cost-effectively manner. In particular, we are interested in understanding whether calls are effective in generating engagement relative to SMS and whether calls form a designated RM are particularly effective. Assignment to the above described relationship-building intervention is stratified within zones, gender and RM treatment arms. The assignment to the communication treatment was held constant across the three sessions of the FDP.

2021-12-19

2022-08-31

1. Farmer attendance at sessions and engagement with call center.
2. Adoption of agricultural practices
3. Coffee cherry delivery (KG)

1. Farmer attendance – the number of FDP sessions the farmer attended. These include two agricultural practices sessions and one session on relationship-building. Engagement with the call center – number of calls made to the call center. Both are recorded in administrative data.

2. Best agricultural practices – the extent to which a farmer implemented the agricultural practices covered during the FDP agricultural session. These practices include organic fertilizers, rejuvenating or replacing old trees, pruning, mulching, weeding, shading trees, and pest management. The source of this data is midline/endline surveys and audits.

3. Coffee cherry delivery (KG) - the volume of coffee delivered by a farmer to the partner’s CWS during the harvest season 2022. The source of this data is administrative. We do not expect to find a short-term impact of the FDP on the delivery volume of coffee in this pilot study. Coffee delivery volume is treated as a long-term outcome, given that implementation of agricultural techniques will only result in increased production volume over a substantial period of time. There might be a short-run impact if FDP reduces side-selling which is however reported to be low by farmers at baseline.

1. Clarity on FDP program expectations and benefits
2. Trust in CWS
3. Traceability of transactions
4. Quality of deliveries

1. Clarity on FDP - the extent to which the farmer is aware of the expectations [behavior expected from the farmer] and benefits [incentives are rewards available to farmers when complying with expectations] associated with the FDP.

2. Trust in CWS – the extent to which the farmer reports to trust the CWS. Trust indicators include trust in CWS, RM, CWS promises, CWS management, and CWS farmers.

3. Traceability of transactions – the share of the farmer’s coffee that is traceable as a result of the farmer using a farmer ID card during coffee cherry deliveries at the CWS.

4. Quality coffee cherries – the quality of the coffee cherries at the point of delivery at the CWS. Quality of the cherries is reflected by the quality indicated assigned and the price paid to the farmer.

The empirical strategy of this pilot project consists of a combination of quasi-experimental and experimental methods. We focus on the following three elements: i) the impact of the FDP, ii) the spillover effect of the FDP, and iii) the impact of relationship-building interventions.

First, we evaluate the impact of the FDP. All farmers in the catchment areas are scored at baseline based on deliveries and other observable characteristics. Our sample is split into two groups: 60% of the farmers above the corresponding threshold within their zones (FDP treatment), and the remaining 40% of farmers scoring below the threshold (FDP control). To overcome potential self-selection bias, we exploit the fact that all farmers were scored against pre-determined criteria and were not able to ex-ante manipulate scores to participate in the FDP. By focusing on those farmers just above and below the threshold, we can use a regression discontinuity design (RDD) framework to causally identify the FDP impacts.

Second, we randomized 32 zones in our sample frame into two groups. The first group includes 16 zones with 80% invited farmers (high-intensity zones), and the second group covers 16 zones with 40% invited farmers to the FDP (low-intensity zones). Focusing on the bottom 20% of the farmers in terms of farmer score, this empirical design permits to measure the spillover effects by controlling for the intensity of invited farmers (40% or 80%) within each zone.

Third, we evaluate the impact of two organizational relationship-building interventions using experimental methods. We assess whether having a designated “fixed” relationship manager and whether communication by a familiar versus unfamiliar individual affects the main outcomes.

Our empirical strategy focuses on the following three elements: i) the impact of the FDP, ii) the spillover effect of the FDP, and iii) the impact of relationship-building interventions.

* Impact of FDP program *
The selection of farmers to participate in the FDP was implemented in several stages. First, we assign scores to all farmers based on scoring criteria. The scoring variables are based on a baseline survey, plot visits data, and administrative data on coffee delivery. Farmers are scored relative to each other within their corresponding CWS.

Second, we randomize the percentages of invited farmers to be part of the FDP across zones. The aim was to select the top 60% of farmers to participate in the FDP. The Partner was keen on supporting only farmers who are more likely to participate in the program and adopt good agricultural practices. Therefore, our sample is split into two groups: 60% of the farmers above the corresponding threshold within their zones (FDP treatment), and the remaining 40% farmers scoring below the threshold (FDP control).

To overcome self-selection in measuring the impact of the FDP, we exploit the fact that all farmers were scored against pre-determined criteria and were not able to ex-ante manipulate scores to participate in the FDP. The notion is that those just above and below the threshold should not reveal any systematic differences of observable characteristics. This allows for the use of the regression discontinuity design (RDD) framework to causally identify the FDP impacts.

* Spill-over impact of FDP program *
Next, we randomize the scoring threshold required for farmers to be part of the FDP. We randomize 32 zones covering the four CWS into two groups. The first group includes 16 zones with 80% invited farmers (high-intensity zones), and the second group covers 16 zones with 40% invited farmers to the FDP (low-intensity zones). Using the block-level intensity random assignment, we plan to measure the spillover effects of the FDP by controlling for the intensity of invited farmers within each zone in our analysis. In other words, we will measure the average spillover effect of being invited from a low-intensity compared to a high-intensity zone. In order to examine the spill-over effect of the FDP, we focus on the bottom quintile of farmers based on the farmer score distribution.

* Relationship Building Interventions*
Third, we measure the impact of two different organization relationship-building interventions (i.e., relationship manager and communication intervention) on the main outcomes. Farmers were randomly assigned to either of the three following relationship manager interventions: i) fixed RM, ii) Control RM, and iii) Placebo RM. Second, we randomly assign farmers to one of the following communication conditions: i) SMS only, ii) SMS + fixed RM call, iii) SMS + placebo RM call, iv) SMS + Call center call. We plan to use to elicit three effects: 1) the causal effects of having a single designated RM (Fixed), 2) the impact of having an additional (introduction) session, and 3) the impact of receiving communication from an RM.

Randomization done in office by a computer using software STATA.

* Stage 1: FDP Farmer Selection *
STRATA: CWS x Zone x Gender

* Stage 2: RDD farmer selection *
STRATA: CWS x Farmer score

* Stage 3: Relationship Building Intervention *
a. Relationship manager intervention
STRATA: CWS x Zone x Gender

b. Communication Intervention
STRATA: RM Treatment x Zone x Session

Yes

* Stage 1: FDP Farmer Selection *
- 32 zones across 4 CWS

* Stage 2: RDD farmer Selection *

* Stage 3: Relationship-building intervention *
- 285 session groups in 32 zones across 4 CWS, each group consists of 10 farmers

Stage 1: FDP Sample size ~ 4900 farmers * Stage 2: RDD farmer Selection * * Stage 3: Relationship-building intervention * Sample size ~2960 farmers

** Stage 1: FDP Farmer Selection **
* a. FDP RCT
- FDP treatment ~ 2960 farmers
- FDP control ~ 1940 farmers

* b. FDP Intensity
- Low intensity zone ~ 2450 farmers [16 zones]
- High intensity zone ~ 2450 farmers [16 zones]

** Stage 2: RDD farmer Selection **

** Stage 3: Relationship-building intervention **
* a. Relationship Manager Intervention
- Fixed RM ~ 990 farmers [95 groups]
- Placebo RM ~ 990 farmers [95 groups]
- Control RM ~ 980 farmers [95 groups]

* b. Communication Intervention ----
- SMS ~ 724 farmers
- Fixed RM call ~ 743 farmers
- Placebo RM call ~ 741 farmers
- Call center call ~ 751 farmers

Unit: Individual farmer Strata: CWS x zone X gender DV: Transaction volume. (average volume delivered in 2020 and 2021) Mean: 550.06694 SD: 1124.5861 Rho: 0.01979559 MDE: 28% Please note that delivery volume is a long-term outcome.