What this research found
How well can a lender actually predict consumer-loan default using only what is known at underwriting time? Working from 2.26 million LendingClub loans issued between 2007 and 2018, K-Dense built a default model that excludes every field updated after a loan is funded and evaluates only on loans that have genuinely reached their three-year maturity. The resulting out-of-time estimate is an AUROC of 0.708 and a Kolmogorov-Smirnov statistic of 0.304 — moderate discrimination that sits squarely in the published range for leakage-free origination-only credit models.
- On the 283,026 fully-matured loans issued in 2015, the model reached AUROC 0.708 (Gini 0.417), KS 0.304, and a Brier score of 0.118. The training AUROC of 0.752 is only 0.044 higher, indicating temporal drift rather than overfitting.
- An audit classified all 151 raw columns and found 38 post-origination leakage fields — cumulative payment totals, recoveries, outstanding principal, the 16-column hardship block, settlement flags, and post-funding credit re-pulls. Dropping these, plus 7 identifier and free-text fields, the status field, and 30 columns missing for over 95% of training rows, left exactly 74 origination-time predictors.
- Requiring genuine maturation matters as much as blocking leakage. Because the snapshot is dated to the fourth quarter of 2018, only loans issued through 2015 have run their full 36-month term, so 402,184 loans issued in 2016-2018 were discarded rather than filtered to 'resolved only', which would have retained a biased subset of early charge-offs and fast prepayers.
- LendingClub's own risk assessment dominates the model. Sub-grade contributes 16.13% of total gain, borrower state 13.00%, coarse grade 8.08%, and the priced interest rate 7.20%; the top ten features account for roughly 53% of gain, with income, accounts opened in the last 24 months, and debt-to-income filling out the capacity-and-leverage channel.
- Default rates drifted upward between the training and test windows, from 13.06% on 2007-2014 loans to 14.89% in 2015. The model correspondingly under-predicts, forecasting a mean probability of 0.132 against the observed 0.149, so a post-hoc recalibration step is recommended before any economic use.
- The second-ranked predictor, borrower state of residence, is flagged as a fair-lending concern: it captures real geographic variation in default rates but can proxy for protected characteristics and would need a fairness audit before deployment.
How it was done
The public LendingClub accepted-loans file — 2,260,701 records across 151 columns spanning 2007 through the fourth quarter of 2018 — was filtered to 36-month terms (1,609,754 loans) and then to resolved outcomes of Fully Paid, Charged Off, or Default (1,020,768 loans), with the binary target set to 1 for charged-off and defaulted loans. A maturation-aware temporal split trained on 335,558 loans issued in 2007-2014 and held out 283,026 loans issued in 2015 as an out-of-time test set. Every column was individually audited and justified as retained or dropped, two raw date fields were combined into a credit-history-age variable, and all encoders were fit on the training window alone. A LightGBM gradient-boosted tree with deliberately conservative settings — 600 trees at a 0.03 learning rate, 31 leaves, a minimum of 100 samples per leaf, row and column subsampling at 0.8, and L2 regularization — was trained under binary log-loss with no class rebalancing, so that predicted probabilities stay anchored to the natural base rate.
Data sources
- LendingClub public accepted-loans dataset, 2007-2018Q4 — 2,260,701 loans, 151 columns (~1.68 GB)
- Lessmann et al., European Journal of Operational Research 247:124 (2015) — credit-scoring algorithm benchmarks
- Serrano-Cinca, Gutiérrez-Nieto & López-Palacios, PLoS ONE 10:e0139427 (2015) — determinants of default in P2P lending
- Kaufman et al., ACM TKDD 6:4 (2012) — formulation and avoidance of data leakage
Limitations
Requiring full maturation discards the most recent and largest vintages, so the model reflects an older credit environment and covers only 36-month loans. Rejected applications are unobserved, leaving the usual reject-inference bias of accepted-only credit data, and the predicted probabilities need recalibration before being read as absolute default rates.
How this research was produced
K-Dense Web planned and ran this business investigation end to end — gathering the sources, carrying out the analysis, producing the figures, and drafting the report. The full session transcript, including every intermediate step, is available to view.
