The impact of farmer field schools on the adoption of climate adaptation measures (AgrImpact)

The evaluation aims to test the effectiveness of farmer field schools in teaching farmers agroecological practices and facilitating their adaptation to climate change. The training program enrolls farmers who reside within a treatment area, have stated their interest in joining the program, and have sufficiently large production systems in the target value chains of pepper, cloves, honey, coffee, and vanilla.

2024-02-15

2024-07-01

Adaptation practice adoption scores

The training aims at teaching farmers how to adapt their agriculture with practices that benefit both harvests and the environment. To assess take up, we count the number of practices adopted and the degree to which these are implemented (both specially and across plants). For a given practice, the farmers considered for evaluation will only be those that produced the given value chain pre-intervention.

Adaptation practice knowledge scores.
Household and individual food security
Household income

The training aims to teach farmers how to adapt their agriculture to practices that benefit both harvests and the environment. A short knowledge test will be used to assess knowledge transfer. Knowledge questions will only be evaluated for the value chains that the farming household had produced prior to the intervention.

Some of the taught practices should directly impact farmers’ food security. Food security will be measured on an individual level using the Food Insecurity Experience Scale (FIES/FAO) and on the household level using the World Food Program’s Food Consumption Score Module.

Though the full effect of the promoted FFS activities will not take place within the evaluation period, adopted practices may lead to a reallocation of labor within the household and, therefore, affect income in the short term.

This randomized control trial (RCT) will compare the impact of Farmer Field School (FFS) to a control condition that receives no intervention. After listing and screening by the implementing partner, the research team made a randomized allocation on the farmer level to the treatment or control condition and selected a sample for inclusion in the survey component. The survey treatment and control sub-samples were intended to be balanced between arms on gender.
Baseline process: The sampling lists for the survey were sent to the survey team for a field survey without disclosure of the treatment allocation. During the field survey, farmers consented with recorded verbal or written documentation. At the end of the field survey in each area, lists of the farmers to be treated in each area will bewere sent to the implementation organization in order to begin intervention activities. Due to delays in receipt of the sampling lists, an in-person baseline survey was not conducted in 3 out of the initially 4 planned sites in Anosy. Due to further delays in implementation in these 3 areas, these were dropped from evaluation, resulting in a reduced sample size and the evaluation taking place in 10 FFS sites (see details below).

This randomized control trial (RCT) will compare the impact of Farmer Field School (FFS) by randomizing eligible farmers who live in the catchment areas of 10 FFS to receive an intervention or a control condition. At each FFS site, the implementing partner enrolled as many participants as possible, with a goal of around 200 farmers. After screening these lists to remove farmers that are not eligible for inclusion to the FFS, the lists were sent to the research team as soon as they were available. These were sent as they were prepared, with most only received as the baseline survey was in process. In Anosy, the lists were not received until one month after the baseline survey was completed due to the lower population and fewer eligible farmers in this area. Due to the lack of a baseline survey, late beginning to the intervention, and early ending of the intervention in all areas, these 3 sites in Anosy were cut from the evaluation sample.
For each sampling list, the research team made an allocation to the treatment or control condition and selected a sample for inclusion into the survey component. For sites where less than an ideal number of farmers were eligible, both the sample size for the survey and the proportion of women added were adjusted. This allocation was made using an Excel template that was designed to increase the proportion of women in the treatment sample up to 30% or at least by up to 5% in cases where the number of women listed was less than 55%. This was done as the implementer had committed to the funder to have at least 30% of participants be women. In the initial allocation, at each site, 100 farmers will be assigned to the intervention, with the remainder assigned to the control, and an equal number of male and female farmers from the treatment and control condition were assigned to the survey subsample.
The sample size varied by site. If the total list numbered between 111-120 farmers 65 farmers were sampled, if there are 121-130 farmers, 75 farmers were selected for surveyed, and if there are 131-150 80 farmers as planned were be included, and for areas with 151+ farmers 100 farmers were surveyed. In all sites, the survey sample will first be taken from the group randomly assigned to the control condition and thereafter from the treatment group. This will result in unequal allocation between groups with more farmers sampled in the treatment allocation in areas where fewer farmers have signed up. In all cases, the survey treatment and survey control sub-samples will be balanced between arms on gender.
After finalizing the numbers of farmers of each gender to be assigned to each condition (treatment surveyed, control surveyed, treatment not surveyed, control not surveyed) for each site, individual farmers were assigned to each condition using random numbers.
The sampling lists for the survey were then sent to the survey team for a field survey without disclosure of the treatment allocation. During the field survey farmers were consented with recorded verbal or written documentation. In the field, farmers assigned to the survey condition were excluded for reasons such as more than one farmer resident in the same household, double listing of the same farmer, farmer illness or death, and farmers residing beyond the FFS catchment area. These excluded farmers were not replaced in the study.
During the baseline survey, due to the uncertainty of if the sampling lists from the FFS in Anosy would be of sufficient size for randomization (>105 farmers) and when these would be received., Tthe sample was increased in the FFS sites for which the research team had sufficient farmers. This resulted in some inequality in the equal numbers of farmers by gender, as, whenever needed, female farmers were assigned preferentially to the treatment condition. In addition, the gender of all farmers was not correctly listed in the sampling lists.
At the end of the field survey in each area, lists of the farmers to be treated in each area were sent to the implementation organization in order to begin intervention activities. At this time, the intervention organization may find other farmers that are not eligible for the intervention that were not identified by the survey team. These intervention farmers will be replaced by non-survey control farmers in the same areas, if available, or in other areas if not available in the same area up to a total of 110 farmers per area. In any case, the intervention will need to have a total of 1300 farmers enrolled in the treatment to meet their contractual obligations.
For endline, we retained the sample where the baseline was completed and those who were not surveyed because they were out of their home at the time of the survey, hard to access due to rains, and sick at the time of the baseline survey. We removed individuals who were duplicated in the list or belonged to the same household as another person in the list, those that had migratedr, could not be found, lived outside the actual catchment area of the FFS, refused participation, or had died.

Random selection of the survey sample and random assignment to the treatment and intervention arms will be performed by site using random numbers generated in Stata.

Individual assignment

Yes

10 sites

800 farmers at endline

Sample size by treatment arm will vary based on the number of eligible farmers able to be interviewed by site. At this time we estimate that there will be completed surveys from 435 farmers in the treatment group and 365 in the control group.

The trial is powered at 90% to detect a standardized effect size of 0.40 in the number of CSA practices taken up by FFS participants compared to controls, in line with the midpoint of the impact of studies with medium risk of bias (0.22) and overall (0.63) reported in Waddington et al. (2014). This person-randomised, blocked trial will include 10 FFS sites with an estimated 80 completed surveys completed in each area, for a total completed sample of 800 participants. To account for survey attrition, we have included 58 additional farmers in our sample. We assumed an alpha of 0.05, an effect size variability of 0.05, and covariates and the blocking variable each explaining 0.1 of the variability in the final model. This sample size is also sufficient to measure the larger effect sizes seen in the number of improved agricultural techniques practiced (Waddington et al., 2014). Due to the randomization procedure, effects can be causally attributed to the treatment with FFS.