Experimental Design
We measure outcomes using microdata collected from lenders and credit file data collected from a credit reference agency.
Our experiment is an RCT. We use 0.5% as the threshold for statistical significance. 5% where results are suggestively significant and also highlight where results are significant at the 1% level. In addition, for the primary outcomes, we will distinguish between statistically insignificant results where we confident in there being no effect (i.e. precisely estimated null results) from results where we cannot rule out there being a treatment effect with confidence (i.e. low powered). This will be done through reporting the minimum detectable effect size (MDE) which shows the effect that would have been detectable with 80% power at the 5%, 1% and 0.5% statistical significance level.
We will construct an unbalanced panel with one observation per credit card, per month observed. The panel is unbalanced as some cards will have been part of a trial for a longer period of time than other cards. We expect to observe each individual to only have one of their credit cards subject to the trial.
We plan to estimate an OLS regression with standard errors clustered at the individual card-level. We expect to observe multiple credit card statement cycles from the start of the trial. Dummy variables for these will be included to allow for the treatment effect to be measured for each cycle from the start of the trial. We hypothesise that treatment effects will vary over time (for example due to consumer learning to optimise their payments as their balances change) but we do not impose a functional form on these as it is unclear what the appropriate functional form would be. The target card is the credit card subject to the trial.
We plan to estimate an OLS regression with standard errors clustered at the individual card-level. We expect to observe multiple credit card statement cycles from the start of the trial. Dummy variables for these will be included to allow for the treatment effect to be measured for each cycle from the start of the trial. We hypothesise that treatment effects will vary over time (for example due to consumer learning to optimise their payments as their balances change) but we do not impose a functional form on these as it is unclear what the appropriate functional form would be. The target card is the credit card subject to the trial.
We plan to include a constant, a series of time-invariant control (CONTROLS) variables (constructed using information on the target credit card and card-holder from before the start of the trial and dummies for the month and year (MONTH) the outcome is observed, dummies for statement cycle number (CYCLE), and an interaction between TREATMENT and CYCLE. The controls are designed to soak up variation not attributable to the trial in order to make our estimates of treatment effects more precise. We are not examining the coefficients on these in the primary analysis. This is because with the lags of the outcome included as controls, the coefficients on other control variables do not measure the effects of these, they measure the effect of controls on the change in outcome.
In this specification the coefficients on the interaction between CYCLE and TREATMENT shows the treatment effect cycles since the start of a trial.
CONTROLS were Gender, Age, Age squared, Log Estimated Income, Credit Score, Unsecured Debt-to-Income (DTI) Ratio, Any Mortgage Debt, Log Credit Card Credit Limit, Credit Card Purchases Rate, Subprime Credit Card, Any Credit Card Promotional Rate, Any Credit Card Balance Transfer, Credit Card Open Date, Credit Card Statement Day, Any Credit Card Secondary Cardholder.
These were all from the time of card origination (or month preceding card origination where consumer rather than Credit Card specific variables). For outcomes constructed from credit reference agency (CRA) data up to eleven dummies for lags of outcomes were included for months preceding the start of the trial.
CYCLE and MONTH are both included because statement cycles do not perfectly align with calendar months and trials went into the field at different points-in-time.