The Impact of Price Comparison Tools for Consumer Credit on Financial Decision-Making

The purpose of this study is to measure the impact of access to just-in-time, personalized information about the price dispersion of loan costs across banks on beliefs about price dispersion, consumer search, financial decision-making, and downstream financial health and well-being.

This RCT has three treatment arms:
1) Full price comparison tool: it shows potential borrowers the distribution of loans granted APRs, conditional on consumer and loan characteristics.
2) Simplified price comparison tool: it shows consumers how much they would save by searching at more banks, conditional on consumer and loan characteristics.
3) Control video: this video explains basic financial concepts about credits and does not provide information about the search and comparison of loans across banks.

The target population is adults searching for a consumer or mortgage loan on Google. To recruit participants, we have identified a set of Google search keywords related to consumer/mortgage loans and have designed a Google ads campaign that will target Chilean residents searching for consumer/mortgage loans online.

2021-11-11

2022-12-01

Expectations about prices and search intensity: These measures are reported directly in the online survey that participants in the RCT will fill out. Both before and after viewing the tool, we ask them to report their beliefs about (i) the APR they expect to get, (ii) the lowest APR that a bank would offer them; (iii) the average APR banks would offer them; (iv) the highest APR a bank would offer them; (v) how many banks they expect to obtain price quotes from; and (vi) how long they expect to search. These questions will allow us to estimate their priors about the distribution of prices, how much they will search, and whether the intervention affects them.

Consumer search: we will measure consumer search in two ways. First,
measuring the number of applications for participants in our experiment using CMF administrative data. Second, we will conduct a follow-up survey six months after the intervention and ask respondents to self-report how many banks they sought price quotes from.
Other relevant measures related to consumer search are (i) number of loans taken (CMF data), (ii) actual search length (follow-up data), and (iii) whether they report having bargained (follow-up data).

Loan characteristics: we will measure whether the tool impacts the loan rate, total amount, and maturity, and in which banks and municipalities they end up taking loans.

Financial health and well-being: we will measure whether the better loan characteristics potentially obtained by consumers lead to other downstream outcomes, such as missed loan payments and total consumer debt across financial institutions using CMF data. In addition, we will include various measures of financial health and well-being in our follow-up survey, such as total savings, ability to cope with shocks, and the standardized metrics on financial health developed by Innovations for Poverty Action.

We will winsorize at the 95th percentile our survey outcomes to avoid outliers guiding the results.

As a secondary outcome, we will study whether people would refer other people to the price comparison tool in the follow-up survey.

To measure the impact of access to personalized, just-in-time information from a credit price comparison tool on consumer beliefs, search, and loan characteristics, we are conducting a randomized control trial (RCT) with the CMF (Chilean financial regulator).

The target sample group is adults searching for a consumer loan or mortgage in Google. To recruit participants, we have identified a set of Google search keywords related to consumer loans and have designed a Google ads campaign that will target Chilean residents searching for consumer loans online. We expect 147,000 people to click our ads, which translates to 28,673 participants (19.5%) being assigned to a treatment arm, based on our pilot data. When loan searchers click our ads, this leads them to our online survey questionnaire. If they provide informed consent, they enter into the initial part of the survey, which asks about sociodemographic characteristics, loan characteristics, and beliefs about the credit market. After completing this initial part, participants are randomized into one of the six treatment arms. After treatment, we ask again about the beliefs about the credit market to measure whether they change subsequently. The total survey duration is of 9 minutes on average.

We have three treatment arms in our RCT, which are described below:

1) Full price comparison tool: it allows users to enter information about their income and the loan type (consumer or mortgage), maturity, amount, and municipality they would like to search. Using CMF's administrative data on the universe of consumer and mortgage loans over the last six months, the interactive tool shows the distribution of APRs that other consumers with similar characteristics received from banks for similar loans. User characteristics are based on the income and municipality tool inputs, and loan characteristics are based on the loan type, amount, and maturity tool inputs.
2) Simplified price comparison tool: the inputs are the same as in the full version of the tool. This simplified version shows users the benefits of search by returning users a statement of how much they would save in terms of monthly and total costs by searching at more banks, conditional on loan and consumer inputs.
3) Control group video: it explains basic financial concepts about credits and does not provide information about the search and comparison of loans across banks.

We will also test whether asking people about search induces more search by randomly not asking these questions to a fraction (25%) of our respondents—more on this in the power calculations section.

Each individual that clicks on the ad and completes the initial part of the survey will be assigned randomly to one of the three arms previously described. We do this in our online survey using Qualtrics.

Randomization is at the individual level.

No

Does not apply.

We need 147,043 people clicking on our ads to detect an effect on our main outcomes with 80% power. This translates to 28,673 respondents (19.5%) getting assigned to a treatment arm, based on our pilot data.

As we have three treatment arms, the sample size by treatment arm would be the number of respondents getting assigned to a treatment arm (28,673) divided by 3, which is 9,558.

We will study several outcomes: initial survey outcomes, consumer and mortgage loans interest rates using administrative data, and follow-up survey outcomes. As our primary outcome of interest is the consumer loan interest rate, power calculations are run in such a way that the consumer loan interest rate is the foundation of them. When potential loan consumers click on our ads, they are directed to our survey. Not all of them complete the survey nor all survey modules, or consent. They can drop out whenever they want. Based on our pilot data, 19.5% of people who click on our ads reach the module where they get assigned to a treatment arm, and 17.3% of people who get assigned to a treatment arm end up taking a consumer loan within six months after the survey. Given our budget constraint and following List et al. (2011), we need 147,043 loan-searchers to click our ads to detect a minimum effect of 0.1 standard deviations (SD) —87 basis points given a consumer interest rates SD of 8.9 within a 6-month window —and to achieve 80% power and a significance level of 5% in our study. This means that 28,673 people would get assigned to a treatment arm, and 4,961 of these treated participants would end up taking a consumer loan within six months after the survey. With this number of people clicking our ads, we could detect a minimum effect of 0.178 SD—11 basis points given a mortgage interest rate SD of 0.625 within a one-year window—for mortgage loans interest rates. Loan take-up rates are 5.2% for mortgage loans one year after the survey among those participants assigned to a treatment arm, hence the higher minimum detectable effect. Moreover, we could detect a minimum effect of 0.07 SD on our initial survey outcomes and 0.1 SD on our follow-up outcomes, after assuming that at least 17% of participants assigned to a treatment arm answer our follow-up survey. Finally, we will also test whether asking people about search induces more search. To do this, we randomly do not ask these questions to 25% of our respondents. This fraction was set so that we could detect a minimum effect similar to our main outcomes. This way, we could detect a minimum effect of 0.09 SD on our follow-up outcomes and consumer loan interest rates, equivalent to 81 basis points. We are aware that not asking search questions reduces our power for initial survey outcomes and adjusted for this accordingly.