1. RCT Design: Justification & Explication
The primary motivation for using an RCT to select those receiving the intervention and those not, is to eliminate selection bias, thereby increasing the precision with which we can measure the precision of the intervention. When offering a new programme to all beneficiaries at the same time, it is difficult to know whether any "impacts" - positive or negative - are attributable to the intervention itself (in our case the installation of the Branchless Banking Agent, or the introduction of Training workshops) or due to other reasons, that may or may not be related to the programme. The use of a rigorously designed experiment with randomized selection of districts and villages that receive the two levels of treatment enable us to attribute whatever change our outcome indicators to the intervention directly.
Despite the random assignment of treatment and control groups, it is possible that differences may arise between the two groups. In order to create statistically equal groups, we use control variables to identify the most precise impacts of the treatment.
5.2 The design of the experiment
We shall carry out an experimental evaluation in 59 districts across 3 departments in the sierra of Peru (Puno, Cusco, Apurimac and Ayacucho).
First of all, to select the districts to be randomized, we first needed to establish (1) The type of JUNTOS district in which we want to focus the intervention and (2) Which districts have adequate conditions for the installation of the agents.
Thanks to preliminary research realised in Phase 1 of the research, we have concluded that the most important factors for the adequate functioning of the agents are (a) sufficient data connection of one of the two main network providers (b) sufficient liquidity to manage the JUNTOS payout. In order to establish which districts had these characteristics, we conducted a pre-selection of districts to verify signal and survey potential agents to determine the number of users that each could manage. Only those districts which fulfilled this criteria in the Southern Sierra region were then pre-selected to be randomized. Following this, we also took out districts which had less than 50 users who would be receiving their payout in the bank branch of another district. None of the districts in the sample had (a) a bank branch of their own or (b) an agent or (c) only beneficiaries who received from a transporter. Finally, we took out districts that were located in provinces which had more than 60% of districts with agents installed in order to diminish the possibility of contamination via agent installation in control districts.
Only when we had the list of eligible districts did we then randomize treatment and control. Randomization consisted of two stages.
1.1 Phase 1 Randomization - District Level
The first level of randomization is to offer the installation of the agents, at district level. The sample consists of 59 districts - 30 treatment districts and 29 controls, with approximately 300 beneficiary households per district.
We ensure balance between the two groups of districts on the basis of administrative data on the average distance to district capitals from villages of each district. Furthermore, we balance the number of districts that receive via branch and transporter across treatment and control. Apart from this, we ensure that socio-economic characteristics, such as literacy rates among JUNTOS users, are balanced between the two groups.
In each treatment district, we coordinate with the JUNTOS programme to re-enforce the new, official message of the programme (Resolución N° 044—2013-MIDIS, abril 2013) that beneficiaries may use their accounts without limitations and can take out or leave money in their accounts without risk of suspension from the programme. This message shall be conveyed across treatment and control groups, at the same time as installation is taking place in treatment.
Following this the treatment districts will be assigned to have promoters of the Branchless Banking Agent product, known as the Agente MultiRed, who will go to the district capitals to promote the product in municipalities and shops, that have been identified as having sufficient cash in the pre-screening.
2.2. Stage 2 of randomization: village level
In stage two of the randomization, inside the districts that receive agents, we shall pilot a trust intervention in 70 villages. Each financial education treatment will consist of a workshop of three hours with reminder SMS messages being sent afterwards to re-inforce the message provided in the workshop.
(1) The first treatment will emphasize the benefits of savings via a financial education that focuses on the reasons and justifications for saving.
(2) The second treatment is a second type of financial education, focused on the improvement of the level of trust that users have in the financial system. This second workshop will put more emphasis on creating a culture of use of the technology, by enabling beneficiaries to see the examples of their community leaders using the technology, and having the regulatory system explained to them.
(3) 1 and 2 combined to see the effects of synergies
(4) A control, where beneficiaries will only receive information about the existence of the agent and the mechanics of their use.
We will stratify the second phase of randomization in order to ensure balance between villages, using administrative data provided by BN, SISFOH and JUNTOS. Among the variables here are the distances from villages to district capitals (where agents will be installed) so as to test whether effects will be greater on villages that have a greater time saved in getting to their paypoints. Stratification will assure sufficient power and balance between these sub-populations. BN will grant us data on the universe of JUNTOS beneficiaries, which shall be merged in the analysis with socio-economic information on the same users.
1.3 Control Groups
One of the requirements for the rigourosity of the evaluation is to guarantee that the control groups are not contaminated. For control districts at the first level of randomization, these districts will not have Branchless Banking Agents installed during the period of the intervention. At the second level of randomization, those villages in control will not be provided with any financial education beyond technical training on how to use the machine in the shops themselves.
We work to assure that there is no contamination in the first phase via close communication with BN in Lima and the intervention zone, so as tu ensure that the evaluation does not clash with their expansion plan in the districts which are included in the impact evaluation.
In order to avoid contamination in the second treatment, we make sure that only users in the villages assigned to receive the workshops receive them, via close coordination with local JUNTOS facilitators at a local level, who know where each user comes from. In addition, we ensure good communication with the implementors of financial education - the Institute of Peruvian Studies. IPA will continue to monitor the implementation of the program during the intervention period. At the start of the workshops, IPA will ask users for some data on their friends in other villages. At the endline, IPA will encorporate questions on the knowledge of beneficiaries regarding the content of other treatments to see if there was spill over of knowledge from one to another.