Coordinating farmers with cellphones: technology innovation in livestock extension services in Pakistan

Motivation and Program Description
Livestock agriculture accounts for twelve percent of GDP in Pakistan, and is a key growth sector for the rural poor, as in much of South Asia (Pakistan Economic Survey 2013-14). The market for livestock faces many of the same imperfections in developing countries as do land or crop markets, however. We are currently implementing in partnership with the Punjab Livestock and Dairy Department a data-driven, state capacity building technological solution to one key public service adoption problem---failure to use artificial insemination (AI) services.

Problem
The market for AI suffers from two informational inefficiencies. First, since insemination can fail even when executed perfectly, livestock owners cannot know whether failure is due to poor effort by technicians or to unavoidable biological causes. Second, while reporting successful insemination and subsequent dairy productivity is low-cost and provides considerable public benefits, citizens have no private incentives to provide this information. These inefficiencies lead to rates of pregnancy and dairy production that are much lower than expected given the technology being used.

Technological solution
We are in the process of running a randomized control trial to evaluate a technology that activates the cellular network to overcome these two informational inefficiencies. We have developed a cellular based information clearinghouse---along the lines of yelp.com and angieslist.com. This clearinghouse uses Android smartphones equipped with an Open Data Kit-based application to collect real-time information on all public AI service provision in one district of Punjab, Sahiwal. This data automatically populates an online dashboard and generates phone calls to farmers to verify service provision and to determine if artificially inseminated animals became pregnant. This data on AI success, an objective measure of veterinary officer (vet) quality, as well as data on cost per AI provision, is then aggregated at the vet level and reported to farmers directly via phone calls and SMS messages. Farmers are not only given the quality level of their past vet but also of other vets in their geographical area.

We have implemented the final stage of this technological solution, information provision to farmers about the quality of nearby vets, as a randomized control trial. We seek to measure the impact of this information provision on farmers, vets, and the market for AI in Sahiwal.

Information provision
Treated farmers are given information on the top three vets within 3km of their household in terms of weighted success in artificially inseminating cows, and the top three vets in terms of weighted success in artificially inseminating buffalo.

All treated farmers are given success rates and average prices charged for these three to six vets (their can be overlap in the most successful vets in terms of cows and buffalo) on the phone and via SMS. Farmers are also offered vets' phone numbers, information on farmer reported satisfaction with vets, and any information on any other vet in our system. We record if any of this additional information was given out.

2014-09-16

2015-05-31

Hypotheses group 1 (primary hypotheses)---effects on farmer outcomes
Our primary outcomes of interest follow directly from the predictions of the model presented in our pre-analysis plan. Namely, we predict that:
(i) treated farmers will be more likely to switch vets than control farmers,
(ii) when treated farmers switch vets, they will be more likely to switch to a higher quality vet than control farmers who switch vets,
(iii) distance to vets will be a weaker predictor of vet choice for treated farmers relative to control,
and (iv) when treated farmers don't switch vets, they should pay a lower price than before treatment if their vet is below nearby vet quality or they should pay a higher price than before treatment if their vet is above nearby vet quality.

Additional secondary hypotheses groups are outlined in detail in our pre-analysis plan.

We will test hypotheses in this group using both our call center data and our in-person survey data. Using our call center data, we will use actual, verified smartphone application form observations of service provision to farmers by vets, as well as reported price and vet quality data from farmers. We will limit our sample to farmers that have received a second AI service provision after being initially treated following a first AI service provision.

Using our in-person survey data, we will test these hypothesis with responses to questions from survey sections 4.1 and and 4.4, including 4.1.2-7, 4.1.10, 4.4.1-4, as well as with GPS coordinates of villages from our sample selection and GPS coordinates of each vet center collected by our field staff. Importantly, we will first create variables for each outcome, including a dummy for a farmer switching vets after treatment, for a farmer switching to a higher quality vet, for farmers paying higher or lower prices than before (equal to one if a post-treatment price is higher when a farmer doesn't switch vets and his vet is better than other nearby vets or if the price is lower and his vet is worse than other nearby vets), and for farmers choosing the vet that is closest to them in distance.

Sample selection
District selection
Sahiwal, Punjab, Pakistan was selected based on several logistical constraints, including our ongoing relationship with the Punjab Livestock and Dairy Development Department, travel distance to Lahore, the prominence of livestock and number of government vets in the district, and knowledge by our field staff. Though Sahiwal was not selected for its external validity, we do view it as generally representative of Punjab and of similar primarily agricultural, rural districts, with a slightly higher prevalence of livestock. According to the 2010 Punjab's Multiple Indicator Cluster Survey, households in Sahiwal on average have 1.4 acres less agricultural land and .24 more cattle than those across other districts in Punjab. Households in Sahiwal are also on average 3.8 percentage points more likely to be receieving government benefits and 1.9 percentage points more likely to have a head of household who completed primary school. Other indicators, such as a wealth index, labor force participation rates, and child mortality rates are no different in Sahiwal than the rest of Punjab on average.

Village and household selection
The farmer selection process was different for each of the data sources outlined below.

The smartphone application has recorded all government vet AI service provisions within Sahiwal since the onset of our program. Thus through it we have sampled the universe of farmers in Sahiwal that have received government AI services since November 23, 2013. To date we have recorded approximately 35,200 entries.

The call center aims to follow-up with every farmer form the smartphone application. In reality, farmers are not sampled for three reasons---(i) because we are not able to reach them on the phone because of an error by a vet entering a farmer's number, (ii) because we are not able to reach them on the phone because their number has changed, their phone was off when we called, they refused to answer our questions, etc., or (iii) because of capacity constraints at our call center. To date, we have completed calls with 7,910 of the 35,200 total smartphone entries.

For our in-person surveys, 90 of Sahiwal's approximately 500 villages were sampled at random from a district village census conducted less than one year prior to our program for another research project. The sample was stratified by whether or not a government vet center was in the village, and on whether the village was a canal colony. Our sample is balanced along the following variables: area, settled area, cultivated area, area of wheat, rice, cotton, sugar cane, pulses, orchards, and vegetables, having a river, distance to the nearest vet center, number of livestock in the village, the literacy rate of heads of households, religion, age, and standard wealth index characteristics.

Within each village, ten households were selected for our surveys. Households were selected using the well-documented EPI cluster sampling method, because of the cost constraints associated with first conducting village censuses. In order to be surveyed, households had to report owning at least two livestock (cows or buffalos) and having regular access to a cellular phone. We did not conduct a census of households so we cannot verify representativeness in this sample.

Treatment assignment
Treatment is administered at the farmer level. Two samples of farmers have been treated---(i) those farmers followed-up by the call center, during our two-month follow-ups, beginning September 16th, 2014 and continuing today, and (ii) in-person survey farmers, at one point of time during the month following November 10th, 2014, or during the two weeks following January 14th, 2015, regardless of whether they also were sampled through our smartphone application (note we were unable to reach all 900 in-person survey farmers on the phone. To date, we have reached 668 for treatment or control phone calls. This was done in two waves, with an in-person follow-up between the two waves to get phone numbers from households we had not yet reached). In both cases, treatment assignment has been through a coin-flip stratified on nearest government veterinary clinic to a farmer's household. In the case of farmers that have entered our smartphone application database more than once for follow-ups or farmers that are in both our call center and in-person samples, treatment assignment was carried forward from the initial assignment.

Data collection
Smartphone application
All government vets in Sahiwal are equipped with an Android phone with an Open Data Kit-based application and are instructed to submit an application form each time they provide AI services to a farmer. The application, made short to ensure that it is not too costly for vets, includes questions on location (sub-district and village names), farmer information (contact, household info), livestock information, and service provision information. It also records GPS coordinates at the point of each service provision (almost always at a farmer's home). As soon as a form is completed by a vet, recorded data is transmitted real-time through a cellular network to a web server.

To date, we have collected data from 35,200 completed smartphone submissions.

Call center
Agents at a call center contracted by our research team make two phone calls to each farmer that has received AI from a government vet in Sahiwal. First, the day after service provision, farmers are called to verify that the core information from the smartphone application recorded by their vet is correct, including the vet's name, the services provided, and the location of the farmer. Farmers are also asked to report the price that they paid for this service, which is not recorded on the smartphone application. Second, farmers are called two months after service provision to verify whether or not each of their animals given AI became pregnant. In addition, farmers selected for treatment are given our treatment information in this second phone call.

Smartphone application verification data, data on prices paid by farmers, data on AI success rates, and verification of treatment information provision are all automatically recorded on a web server, linked to the original smartphone application submission.

Farmer in-person surveys
This project includes three rounds of in-person surveys---a baseline survey, a phone number follow-up survey, and an endline survey. The baseline survey, administered to all 900 sampled farmers, was fielded during the last week of August and the first week of September, 2013. The phone number follow-up survey, fielded during the last two weeks of December, 2014, was administered to a subset of 500 of the 900 sampled farmers. The endline will be administered to all 900 sampled farmers in late May/early June, 2015.

The baseline and endline survey include a number of modules which will allow us to test our hypotheses. All hypotheses will refer to specification survey modules and questions. The phone number follow-up survey was used to obtain updated phone numbers for farmers that could not be reached using their phone number from the baseline survey. In addition, we used the follow-up survey to pilot several modules to help determine what will be in the endline for the entire sample.

Vet in-person surveys
This project also includes in-person baseline and endline surveys of all government vets who perform AI services in Sahiwal. The baseline survey, completed on October 10th, 2013, was administered to 90 vets. The endline survey will be administered in late May/early June, 2015.

Stratified coin-flip done real time by our web database as new smartphone application entries are uploaded.

Information provision is randomized at the household level.

No

The same as the number of observations

40,000 smartphone application entries, 10,000 completed phone calls, 900 in-person farmer surveys, and 90 in-person vet surveys.

50/50 by design.