LendNova: Towards Automated Credit Risk Assessment with Language Models

AI summary

Overview

Research area: Machine learning applied to credit risk assessment in consumer lending, combining natural language processing (NLP) with production-scale financial data.

Technical level: Intermediate. Readers will get the most from this paper with some familiarity with transformer language models (BERT-style encoders), fine-tuning, and standard credit-risk metrics such as AUC, though the architecture itself is described in accessible terms.

Scope: The paper describes LendNova, an end-to-end pipeline that converts raw, jargon-heavy credit bureau records into plain-English "credit stories," embeds them with fine-tuned FinBERT encoders, and predicts default risk, evaluated on roughly one million anonymized Canadian customer records against an industry-standard bureau credit score.

What This Paper Is About

Credit risk models traditionally rely on manually engineered features extracted from bureau records — a slow, expensive process in which lenders often buy only a limited subset of features because of licensing costs, potentially discarding useful information. LendNova's goal is to skip that feature-engineering step entirely and operate directly on raw, encoded bureau text, letting a language model learn the relevant risk signals itself. The authors test whether such an approach can approach the accuracy of the bureau-provided credit score while reducing cost and manual effort.

Key Contributions

1. The "credit story" representation. The paper introduces a novel representation that converts jargon-heavy, code-based credit bureau records into plain English that language models can consume directly, removing the need for extensive domain-expert feature extraction.

2. First practical deployment of language models for credit acquisition risk modeling at industrial scale. The authors report an end-to-end pipeline evaluated on real-world production data from a commercial credit bureau, described as the first of its kind in this setting.

3. A new baseline and research direction. LendNova is presented to the research community as an early step toward foundation models and intelligent systems capable of automated end-to-end decision-making in financial domains.

4. A three-module system architecture. The pipeline consists of a data preparation module (record parsing, segmentation, translation, vocabulary extraction, temporal analysis), a language model module (custom tokenizer, parallel FinBERT models, aggregator pooling), and a task predictor (an MLP trained with weighted binary cross-entropy loss).

Main Findings

• Model performance approached the industry baseline: After six iterative model versions, the final "champion" model stood at a 3.63% AUC gap behind the industrial baseline. The baseline is the bureau-provided credit score, which has an AUC of approximately 0.8 for default prediction on this particular dataset.

• The dataset is highly imbalanced: Target event rate is less than 5%, with a slightly lower event rate expected in the out-of-time (OOT) set, attributed to 2018 market conditions and fewer overall defaults compared with previous years. The authors note charge-offs in particular are rare events that are harder to predict without specialized techniques.

• Iterative design choices drove improvement: The six versions progressed from an initial pipeline without a custom tokenizer and separate per-segment classifiers combined by majority voting, through adding a custom credit-vocabulary tokenizer (Version 2), oversampling positives to a 70-30 balance (Version 3, which showed reduced effectiveness and was modified later), tokenization optimization with a single comprehensive dataset and 90-10 label balancing (Version 4), aggregator pooling into an expanded MLP layer without label balancing (Version 5), and finally injecting temporal information into the text embeddings (Version 6).

• Cost efficiency is claimed but not quantified: Due to confidentiality of the framework and its commercial deployment, exact numerical values are not disclosed. The authors instead identify domains of efficiency: data licensing, multi-task learning scalability, training and compute amortization, and automation of feature engineering. They state that in large-scale markets such as Canada, unified modeling and reduced data dependency could in theory enable savings "in the order of millions of dollars annually." This is described as a potential rather than a measured result.

• No direct accuracy comparison beyond the AUC gap is reported. The paper does not report absolute AUC values for LendNova itself, precision, recall, or other classification metrics in the provided content.

Methodology in Plain English

Data preparation. Raw bureau records are structured into a fixed field order, then grouped into segments representing different aspects of a borrower's behavior. The authors focus on three segments: Trade (types and recency of credit obligations), Inquiries (credit application patterns, where high or recent volumes can signal credit need or risk), and Collections (past collection events, a direct risk indicator). Each segment is translated from encoded text into plain English — the "credit story." A vocabulary extraction step collects credit-specific terms from these translations and adds them to the tokenizer's vocabulary so that domain terms are recognized as single tokens. A temporal analysis step computes each record date's difference from the "run date," aggregating minimum, maximum, and average relative dates per segment into a temporal vector.

Language model. FinBERT — chosen because it is pre-trained on a large financial text corpus — is used as the base model, with its tokenizer extended by the domain vocabulary. In the final architecture, parallel FinBERT models embed each segment separately, and the resulting segment embeddings are concatenated with the temporal vector into a single unified vector per record.

Task prediction. The unified vector is passed through a multi-layer perceptron to output a default probability. Because defaults are rare, training uses a weighted binary cross-entropy loss that assigns higher weight to the positive class. The full pipeline is trained end-to-end.

Experimental setup. Data is split 60% train, 20% validation, and 20% test, with the test portion further divided into a holdout set (same time window as training) and an out-of-time set (records after the training timeframe, simulating real deployment). Records are anchored to each individual's "run date," the most recent date they applied for a credit card before September 2018; training and validation used records with a run date of February 1, 2018, or earlier.

Target definition. The label is an OR of two binary indicators over an 18-month window (t₀ to t₁ = t₀ + 18 months) following a profile snapshot taken just before the customer's last recorded credit card application: charge-off (account declared uncollectible by the lender) and delinquency (account 90 days or more overdue, excluding accounts already written off or involved in bankruptcy). Records with insufficient data or no credit card trades are labeled None.

Ethics. All data were fully anonymized, preserving only non-identifiable attributes and anonymized record identifiers.

Why This Matters

Impact on research. The paper argues that bureau data functions as a domain-specific language distinct from the natural text corpora used to train most language models, and that fine-tuning pre-trained financial models is more efficient than training from scratch. LendNova is positioned as a baseline for future multi-task, multi-domain financial foundation models that unify risk assessment and decision support.

Real-world applications:

• Loan and credit card application approval or decline decisions, where models support a threshold-based lending choice.
• Cost reduction in data acquisition, by operating on raw bureau data instead of purchasing bureau-aggregated feature bundles.
• Unified multi-task credit decisioning, replacing multiple separately maintained models with one framework.
• Automated preprocessing of encoded financial records, reducing manual feature-engineering latency and domain-expert dependency.

Industry relevance. The work was supported by an industry partner in the Canadian digital-investment sector, and the authors argue these methods could extend to other banking decisions that rely on bureau data. The stated ambition is faster, cheaper, and more scalable risk evaluation for large-scale lenders.

Future Directions

• Closing the remaining accuracy gap. The final model sits 3.63% behind the bureau-credit-score baseline in AUC; the authors suggest larger-scale training could narrow or surpass this.
• Richer financial signals. Extending the architecture to capture more complex behavioral patterns beyond the three segments used here.
• Multi-task and multi-domain foundation models. Unifying risk assessment and decision support across the broader financial ecosystem in a single framework, rather than training separate models per task.
• Quantified cost analysis. Since exact cost figures are withheld for confidentiality, publishing measured efficiency gains remains an open direction for the research community.

Target Audience

This paper is most valuable to applied machine learning researchers and practitioners working on credit risk, fintech, and financial NLP, particularly those interested in replacing feature-engineering pipelines with language model representations. It also speaks to data scientists at lenders and credit bureaus evaluating domain-adapted transformers for production risk systems, and to academic researchers exploring the path from task-specific models toward financial foundation models.

Authors’ abstract

Credit risk assessment is essential in the financial sector, but has traditionally depended on costly feature-based models that often fail to utilize all available information in raw credit records. This paper introduces LendNova, the first practical automated end-to-end pipeline for credit risk assessment, designed to utilize all available information in raw credit records by leveraging advanced NLP techniques and language models. LendNova transforms risk modeling by operating directly on raw, jargon-heavy credit bureau text using a language model that learns task-relevant representations without manual feature engineering. By automatically capturing patterns and risk signals embedded in the text, it replaces manual preprocessing steps, reducing costs and improving scalability. Evaluation on real-world data further demonstrates its strong potential in accurate and efficient risk assessment. LendNova establishes a baseline for intelligent credit risk agents, demonstrating the feasibility of language models in this domain. It lays the groundwork for future research toward foundation systems that enable more accurate, adaptable, and automated financial decision-making.