The study is conducted in Odisha, India, with farmers who are potential clients of Precision Agriculture for Development, an NGO that attempts to improve the lives of farmers by providing tailored farming advice.
Precision Agriculture for Development (PAD) is testing a variety of different interactive voice response (IVR) treatments. The purpose of these treatments is to enroll as many rice farmers as possible into Odisha’s farmer information service. PAD designed these treatments based on pilots and experience from other enrollment campaigns. The treatments are fully designed and implemented by PAD and their final choice was to test six distinct treatment arms (see above). The adaptive experimental procedure the authors designed helps PAD to choose the treatment that works best for their setting.
The experiment is conduced in waves of 600. The only information PAD has are a very large set of phone numbers provided by the government. An employee of PAD processes these numbers by querying if they are valid, and if they are on a do-not-disturb list. Only valid numbers not on the list are then used. PAD is setting aside 10'000 phone numbers that are up for calling in the target period for this experiment.
Experimental waves are carried out consecutively, where each wave takes two days to administer (text messages sent up to 24 hours ahead of time, two call times in morning and evening on the call day). In the first wave, 100 phone numbers are randomly selected from the set of 10'000 numbers and each is assigned randomly to one of the treatment arms. The outcomes of each experimental wave are submitted to an app programmed by the authors. The app uses treatment success rates in each treatment arm to update a flat prior on possible average success rates, and applies a modified Thompson sampling procedure to choose the number of phone numbers to be assigned to each treatment arm in the next wave. A new wave of 600 phone numbers are then randomly selected and assigned to treatment arms. Modified Thompson sampling is adapted from standard Thompson sampling for waves of size 1 by (a) reducing sampling variation by taking advantage of batch sampling (of 600 units per wave here), and (b) re-distributing experimental units from the best performing option to close competitors to improve learning. The experiment continues until 10'000 phone numbers are used.