Experimental Design
1. Overview
The study uses an encouragement design in a cluster randomized controlled trial (c-RCT) combined with qualitative research to estimate the impact of PBR cowpea on key productivity, income, environmental, health, and social outcomes at the household and farm levels in two of Nigeria’s main cowpea-producing regions (Adamawa and Kwara states). In this study, the unit of randomization is a community in a Local Government Area (LGAs or sub-state administrative units that is similar to districts or counties) and the unit of analysis is the farm-household.
LGAs will be randomly assigned to one of the three treatment or control groups as detailed below:
• Treatment Group 1 (T1) will receive PBR cowpea seed, associated information on its cultivation and management, and complementary inputs.
• Treatment Group 2 (T2) will receive PBR cowpea seed and associated information on its cultivation and management.
• Control Group (C) will receive conventional (non-PBR) cowpea seed, associated information on cultivation and management, and complementary inputs.
2. Sampling
The study uses a multistage sampling procedure to select the states, LGAs, communities and households included in the study. In the first stage, we purposively select Adamawa and Kwara state in North-East of Nigeria. Both states are among the top cowpea-producing states in Nigeria. Moreover, both states provide acceptable conditions to research because they currently enjoy a relatively stable security situation when compared to other major cowpea-producing states in Nigeria.
The second sampling stage involves the purposive selection of four LGAs from 25 LGAs of Adamawa state and four LGAs from the 16 LGAs of Kwara state. These eight LGAs from both states were chosen because there is no PBR cowpea penetration in these areas, and it is expected to remain unchanged by the time of the intervention. The four selected LGAs in each state are similar across important contextual factors including size, socioeconomic and agroclimatic conditions, and road and market access. From the eight LGAs, we select 240 communities that are also similar across contextual factors, with eighty communities for the control group and the remaining 160 communities for the treatment groups. Within each community, we then list all households, from which we randomly selected five cowpea farmers to participate in the study, including from the control communities, for the baseline and end line surveys. We estimate the sample size of 400 farm households for the treatment group 1, with 400 farm households for the treatment group 2, and with 400 farm households for the control group. To account for the attrition rate, we increased the sample size by 200 households. As all communities (treatments and control groups) are selected randomly, adding 200 households to either group has no effect. Thus, we added 200 households from communities in treatment 2 to our sample. Within each community, we then list all households that cultivate cowpea and randomly select 5 cowpea farming households to participate in baseline, midline, and endline survey rounds.
To minimize the possibility that our study design is compromised during the intervention rollout (e.g., units selected for treatment may not, in fact, receive the treatment, or may not receive it in the fashion that was intended by the intervention), a field team comprised of research assistants and extension agents will regularly visit the study sites, engage with the local staff of the study’s implementing partner, monitor progress, and report back to the evaluation team. To assist farmers in following the recommended PBR cowpea practices, dedicated extension agents will be contracted. The timely application of insecticides will require not only supervision but also training for safe handling and application.
The evaluation team has been working with technology developers, the African Agricultural Technology Foundation (AATF) and its partners (including private local seed companies) to ensure access to necessary data and cooperation for the evaluation team, while maintaining the independence of the evaluation team. We have closely collaborated and gathered inputs from a diverse set of stakeholders, including USAID, the Institute of Agricultural Research (IAR, Zaria), the National Biotechnology Development Agency (NABDA), the Open Forum on Agricultural Biotechnology (OFAB) Nigeria Chapter, the National Biosafety Management Agency (NBMA), the Federal Ministry of Agriculture and Food Security (FMAFS), the Adamawa and Kwara state ministries of agriculture, extension officers, and the seed companies operating in the project study area.
3. Timing and duration
In the Northern Guinea Savanna agro-ecological zones of Adamawa and Kwara states, which is where the study will be implemented, there are two cowpea cropping seasons: wet season, which runs from August to December and dry season, which runs from February to July. This study will be implemented during the wet crop season in late 2023, with the rollout of the intervention beginning just prior to the season.
4. Treatment Effects
The study’s main outcome of interest—the effect of the pod borer-resistance trait on cowpea productivity and profitability—will be estimated by comparing average yields (kg/ha) and profits (NGN/ha)—between T2 and C. Additional outcomes include the incremental effect of providing treated farmers with an input package, which will be estimated by comparing the average household-level yield between T1 and T2. The additional outcomes described earlier (awareness, uptake, adoption, productivity, profitability, pesticide use and costs, health care use and costs, consumption, and marketing and commercialization) will be estimated to evaluate the study’s underlying theory of change, impact pathways, and mechanisms behind the observed effects. This will be supplemented with qualitative research conducted in the study area and within the study communities.
5. Data Collection
The unpredictability inherent in rainfed smallholder agriculture, including factors like pest and disease pressures, temperature and precipitation fluctuations, and volatile input and commodity prices, often results in significant year-to-year variations in yields and prices. Consequently, outcomes such as yields and profits tend to exhibit moderate to low levels of autocorrelation.
Additionally, the trial conducted in a small plot area using two kilograms of cowpea seeds and the comparison between baseline and midline surveys may not be suitable for our analysis. Nevertheless, the baseline survey provided an opportunity to average out noise when measuring time-invariant characteristics, generate a detailed descriptive analysis of the study site, context, and sample, and ensure that randomization was implemented according to the study protocol and that the sample is appropriately balanced.
The choice of variables for the baseline survey is based on those used by similar studies in their orthogonality tests. In particular, the study will collect data on variables that are commonly considered in prior studies that investigate the adoption of yield-improving and damage-abating agricultural technologies and practices using both RCTs and non-experimental studies (Duflo, et al., 2011; Karlan et al., 2014; Ashraf, et al., 2009; Bulte et al., 2014). The survey will collect data from the sampled households with 12 modules arrayed in a survey instrument based on computer-assisted personal interviewing. Modules will cover household demographics (household size, age, education of household members), agricultural and non-agricultural assets, access to markets and public services (distance to the nearest agro-input shop and road, access to extension services). In addition, detailed information will be collected on household cowpea cultivation (input use and management practices), consumption, and marketing, as well as health symptoms and care.
The midline survey will re-survey the same sampled households in 2024, approximately one year after intervention rollout, and will feature similar modules, with an emphasis on cowpea cultivation, consumption and marketing. Similarly, the endline survey will re-survey the same sampled households one year later after the midline survey to capture sustained adoption and dis-adoption dynamics (Abate et al., 2023) and to provide insight on impacts over a longer term. Both the midline and endline survey will also aim to collect data on unintended (positive and negative) consequences of the adoption of PBR cowpea, e.g., a decrease in pest pressure affecting cowpea across the community. Although we do not anticipate a significant or systematic attrition rate during the intervention or between survey rounds, the study will conduct tests to explore attrition bias resulting from households dropping out of the study during or after the intervention.