Experimental Design
As part of the list randomization technique, half of the survey respondents are randomly selected to receive a short list of statements (in our case a list of business investments) and asked to report how many, but not which, statements are true. The other half of the survey respondents are presented with the same list of statements and one key additional statement designed to capture sensitive behavior (in our case non-business investment or a type of consumption). By subtracting the mean number of true statements in the first group from the mean number of true statements in the second group, researchers can estimate the proportion of the sample that engages in the sensitive behaviour.
Prior to evaluating the use of video and radio as a means for financial education, 1650 MFI clients were surveyed in Cuzco, Peru. The lending institution, Arariwa, provides microcredit for business purposes to approximately 20,000 low-income households in southeastern Peru. Arariwa emphasizes that loans should be used for business, and requires the borrower to state what the loan will be used for when they apply. However, there is no policy of explicitly monitoring the use of the cash proceeds from loan disbursal. As part of the baseline survey, Arariwa clients were asked questions related to their personal finances and education. Surveyors were not affiliated with any MFI and informed survey respondents that their responses would not be shared with anyone other than researchers studying how entrepreneurs that are Arariwa clients manage their household finances. All respondents were asked to report their loan uses through direct report and list randomization techniques. For the direct report, respondents were asked to list up to five loans that they had taken out in the past 12 months, by loan source and amount. They were then asked, “Which need or which needs did you cover with this loan?” and allowed to list up to three uses for each loan. Though respondents were not prompted with categories, surveyors matched uses against one of 18 possibilities.
Starting the list randomization, clients were randomly selected to be presented with one of four possible groups of three to six statements. The randomization was stratified by lending group. A subset of clients were randomly selected to be surveyed, and if an individual was not found then there was a replacement list, randomly ordered, of individuals to survey. Any replacement individual was assigned to the same list randomization treatment as the original target respondent. All clients received the following three statements: “I used part of my Arariwa loan to buy merchandise for my economic activity”, “I used part of my Arariwa loan to buy equipment for my economic activity” and “I shared my loan with another person”. Clients in group A (n=408) only received these statements. Clients in group B (n=414) additionally received the following statement: “I used at least a quarter of my Arariwa loan on household items, such as food, a TV, a radio, etc.” Group C (n=388) received the four previous statements, and the statement, “I used at least a quarter of my Arariwa loan to pay for my family's medical expenses.” Group D (n=401) received the previous five statements and the statement, “I used at least a quarter of my Arariwa loan to pay for my family's educational expenses.” By subtracting the mean number of true statements for group A from the mean number of true statements for group B, we get the proportion of clients that used a quarter of their loan for household items. We similarly subtract B from C and C from D to get the proportions of clients using their loans on education or medical expenses. In order to compare estimates, we match the loan uses from direct report to those from list randomization. Since the direct report question allows clients to list up to five loans from any source, we limit the sample to only include Arariwa loans or communal loans facilitated by Arariwa. Due to cultural norms and surveyor training, “household items” is best approximated by the direct report responses that are classified as “consumption good”, “purchase clothing or shoes”, and “other consumption need.”
After the first two rounds of direct questioning of the clients – once before loan disbursal and once after the first repayment, surveyors then visited clients, on average, two weeks after the client was granted a loan from one of the participating banks. The surveyor asked them to participate in a survey about “Health and Financial Services.” Respondents had no reason to believe that the surveyors had any connection to the bank. The first few questions asked about health attitudes and behaviors so that clients would not think that the surveys were coming directly from the bank. Surveyors had no information about the three participating banks. Surveyors then asked the clients the two questions explicitly. The difference between the responses during the first repayment and the explicit questions from the surveyor allows us to see how responses change when clients think the bank may be monitoring their answers. At that time, surveyors also presented the questions indirectly using list randomization.
Clients were asked two sets of list randomization questions. These questions allowed us to estimate the proportion of true answers to the two statements. Each client randomly received one of four surveys. All surveys contained the following four statements in the first question: “I have visited a hospital or clinic in the last six months,” “I have more than three siblings,” “I have purchased some type of insurance in the past five years,” and “My household owns an air conditioner.” The second and fourth surveys had “I used 2500 pesos or more of my loan to pay down other debt” as the fifth question. Similarly the second set of list randomization questions included the following four statements on all surveys: “I have a washing machine in my home,” “I am originally from this city,” “I have completed one year or more of formal education post-high school,” and “My household owns a computer.” The third and fourth surveys also include the statement “I used 5000 pesos or more of my loan on any single transaction for my household.” In this case, the questions used in list randomization were exactly the same as those used in direct elicitation, so any differences in results can be attributed purely to the method and not the content of the question. Survey one was administered to 58 people in our sample, survey two was administered to 77 people, survey three was administered to 59 people and the final survey was administered to 66 people. Comparing results from the explicit question by the surveyor to results from the list randomization will demonstrate how responses change when clients believe their answers are truly private, even from the surveyor.