# The value of text for small business default prediction: A deep learning approach

arXiv: https://arxiv.org/abs/2003.08964 · HTML: https://mlsanity.com/paper/2003.08964/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Mar 2020.

## Built on by (most cited first, 20 of 20)

- [Multi-Modal Deep Learning for Credit Rating Prediction Using Text and Numerical Data Streams](https://mlsanity.com/paper/2304.10740/) (2023)
- [A Fused Large Language Model for Predicting Startup Success](https://mlsanity.com/paper/2409.03668/) (2024)
- [A transformer-based model for default prediction in mid-cap corporate markets](https://mlsanity.com/paper/2111.09902/) (2021)
- [Optimizing Credit Limit Adjustments Under Adversarial Goals Using Reinforcement Learning](https://mlsanity.com/paper/2306.15585/) (2023)
- [The Deep Promotion Time Cure Model](https://mlsanity.com/paper/2305.11575/) (2023)
- [Credit Risk Meets Large Language Models: Building a Risk Indicator from Loan Descriptions in P2P Lending](https://mlsanity.com/paper/2401.16458/) (2024)
- [Advancing Financial Risk Prediction Through Optimized LSTM Model Performance and Comparative Analysis](https://mlsanity.com/paper/2405.20603/) (2024)
- [Forecasting Credit Ratings: A Case Study where Traditional Methods Outperform Generative LLMs](https://mlsanity.com/paper/2407.17624/) (2024)
- [On the combination of graph data for assessing thin-file borrowers' creditworthiness](https://mlsanity.com/paper/2111.13666/) (2021)
- [Foundation Models for Credit Risk Prediction: A Game Changer?](https://mlsanity.com/paper/2605.18147/) (2026)
- [On the dynamics of credit history and social interaction features, and their impact on creditworthiness assessment performance](https://mlsanity.com/paper/2204.06122/) (2022)
- [Multimodal Document Analytics for Banking Process Automation](https://mlsanity.com/paper/2307.11845/) (2023)
- [FSL-BDP: Federated Survival Learning with Bayesian Differential Privacy for Credit Risk Modeling](https://mlsanity.com/paper/2601.11134/) (2026)
- [Look Who's Talking: Interpretable Machine Learning for Assessing Italian SMEs Credit Default](https://mlsanity.com/paper/2108.13914/) (2021)
- [Forecasting Cryptocurrency Returns from Sentiment Signals: An Analysis of BERT Classifiers and Weak Supervision](https://mlsanity.com/paper/2204.05781/) (2022)
- [PDx -- Adaptive Credit Risk Forecasting Model in Digital Lending using Machine Learning Operations](https://mlsanity.com/paper/2512.22305/) (2025)
- [Predicting Startup Success Using Large Language Models: A Novel In-Context Learning Approach](https://mlsanity.com/paper/2601.16568/) (2026)
- [textttfindr: Transparent and Fair Credit Risk Decisions through Semi-Structured Regressions](https://mlsanity.com/paper/2608.24582/) (2026)
- [Why Bonds Fail Differently? Explainable Multimodal Learning for Multi-Class Default Prediction](https://mlsanity.com/paper/2509.10802/) (2025)
- [A Multimodal Approach to SME Credit Scoring Integrating Transaction and Ownership Networks](https://mlsanity.com/paper/2510.09407/) (2025)

## Abstract (arXiv)

Compared to consumer lending, Micro, Small and Medium Enterprise (mSME) credit risk modelling is particularly challenging, as, often, the same sources of information are not available. Therefore, it is standard policy for a loan officer to provide a textual loan assessment to mitigate limited data availability. In turn, this statement is analysed by a credit expert alongside any available standard credit data. In our paper, we exploit recent advances from the field of Deep Learning and Natural Language Processing (NLP), including the BERT (Bidirectional Encoder Representations from Transformers) model, to extract information from 60 000 textual assessments provided by a lender. We consider the performance in terms of the AUC (Area Under the receiver operating characteristic Curve) and Brier Score metrics and find that the text alone is surprisingly effective for predicting default. However, when combined with traditional data, it yields no additional predictive capability, with performance dependent on the text's length. Our proposed deep learning model does, however, appear to be robust to the quality of the text and therefore suitable for partly automating the mSME lending process. We also demonstrate how the content of loan assessments influences performance, leading us to a series of recommendations on a new strategy for collecting future mSME loan assessments.