CreditLens
A loan-decisioning platform that optimises expected profit per applicant, not AUC.
- Status
- Production-grade reference implementation
- Role
- Solo — feature pipeline, modelling, decision policy, fairness, CI
- Timeline
- 2026
At a glance
A credit-risk decisioning system that treats the business objective as the target. The champion model barely beats a scorecard baseline on AUC — but calibration plus a profit-optimal threshold turns a loss-making policy into roughly €10,000 of expected margin per applicant, with fairness auditing and drift monitoring built in.
- The headline result is deliberately contrarian: architecture contributed almost nothing; the decision policy contributed everything.
- Leakage auditing is enforced mechanically in CI, with strictly temporal train/test splits.
- Three production-grade baselines including a Weight-of-Evidence scorecard — the thing banks actually use.
- Fairness mitigation with Fairlearn, reported alongside its measured accuracy cost.
- SHAP-based adverse-action reason codes for regulatory compliance.
Key numbers
+0.0039 AUC
Model gain
Champion vs a WoE scorecard — deliberately small
≈ €10,018
Policy impact
Expected margin per applicant after calibration + threshold
~0.118
Decision threshold
Profit-optimal, not the default 0.5
12 months
Drift monitor
Simulated production with PSI / KS detection
Tech stack
- Features
- DuckDBPolars / ArrowPoint-in-time aggregationFeast feature store
- Models
- LightGBMCatBoostWoE scorecard baselineOptunaMLflow
- Decision policy
- Probability calibrationCost-sensitive threshold (~0.118)Expected-profit objective
- Governance
- Fairlearn auditSHAP adverse-action codesPSI / KS drift monitorGitHub Actions
The problem
Credit-risk projects almost always optimise AUC and then pick a 0.5 threshold. Neither of those is the business problem. The lender cares about expected profit per applicant, subject to fairness and regulatory constraints.
CreditLens is built to demonstrate what changes when you make the money the objective.
Feature engineering
Features are computed with DuckDB and Polars over Arrow, using point-in-time windowed aggregation so no feature can see the future. They are served through Feast with explicit train/serve skew validation, because the gap between training and serving is where credit models quietly break.
Modelling and the decision policy
LightGBM and CatBoost are tuned with Optuna and tracked in MLflow, benchmarked against three baselines including a Weight-of-Evidence scorecard. The champion model beats the scorecard by just +0.0039 AUC.
Then the decision policy does the real work: probability calibration makes the scores trustworthy, and a cost-sensitive threshold (~0.118 rather than 0.5) is chosen to maximise expected profit. Together they move the policy from loss-making to roughly €10,018 expected margin per applicant.
Governance
Leakage is audited with mechanical CI enforcement. Fairness is audited with Fairlearn and mitigations are reported with their accuracy trade-off rather than hidden. A drift monitor simulates 12 months of production traffic with PSI and KS detection, and every declined application gets SHAP-based adverse-action reason codes.
What I took from it
For a large class of tabular problems, the model is a solved commodity. Calibration, threshold selection, and the framing of the objective are where the value — and the risk — actually live.
Want the details?
The full source, READMEs, evaluation harnesses, and results are on GitHub.
github.com/dariyaDeepanshu/CREDITLENS