# Empowering Many, Biasing a Few: Generalist Credit Scoring through Large Language Models

arXiv: https://arxiv.org/abs/2310.00566 · HTML: https://mlsanity.com/paper/2310.00566/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 79 references that are in the map.
- First posted on arXiv 1 Oct 2023.

## Builds on (most cited first, 29 of 29)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://mlsanity.com/paper/2110.08207/) (2021)
- [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://mlsanity.com/paper/2103.10360/) (2021)
- [BloombergGPT: A Large Language Model for Finance](https://mlsanity.com/paper/2303.17564/) (2023)
- [Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2307.15217/) (2023)
- [FinBERT: Financial Sentiment Analysis with Pre-trained Language Models](https://mlsanity.com/paper/1908.10063/) (2019)
- [AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias](https://mlsanity.com/paper/1810.01943/) (2018)
- [PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance](https://mlsanity.com/paper/2306.05443/) (2023)
- [FinBERT: A Pretrained Language Model for Financial Communications](https://mlsanity.com/paper/2006.08097/) (2020)
- [Fundamental Limitations of Alignment in Large Language Models](https://mlsanity.com/paper/2304.11082/) (2023)
- [In-Context Impersonation Reveals Large Language Models' Strengths and Biases](https://mlsanity.com/paper/2305.14930/) (2023)
- [Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models](https://mlsanity.com/paper/2304.07619/) (2023)
- [WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domain](https://mlsanity.com/paper/2211.00083/) (2022)
- [Exploring the Impact of Instruction Data Scaling on Large Language Models: An Empirical Study on Real-World Use Cases](https://mlsanity.com/paper/2303.14742/) (2023)
- [BBT-Fin: Comprehensive Construction of Chinese Financial Domain Pre-trained Language Model, Corpus and Benchmark](https://mlsanity.com/paper/2302.09432/) (2023)
- [The Wall Street Neophyte: A Zero-Shot Analysis of ChatGPT Over MultiModal Stock Movement Prediction Challenges](https://mlsanity.com/paper/2304.05351/) (2023)
- [Nationality Bias in Text Generation](https://mlsanity.com/paper/2302.02463/) (2023)
- [Multi-Task Learning for Dense Prediction Tasks: A Survey](https://mlsanity.com/paper/2004.13379/) (2020)
- [Transparency, Auditability and eXplainability of Machine Learning Models in Credit Scoring](https://mlsanity.com/paper/2009.13384/) (2020)
- [Bias and Discrimination in AI: a cross-disciplinary perspective](https://mlsanity.com/paper/2008.07309/) (2020)
- [Unveiling the Potential of Sentiment: Can Large Language Models Predict Chinese Stock Price Movements?](https://mlsanity.com/paper/2306.14222/) (2023)
- [Multi-Modal Deep Learning for Credit Rating Prediction Using Text and Numerical Data Streams](https://mlsanity.com/paper/2304.10740/) (2023)
- [Data science and AI in FinTech: An overview](https://mlsanity.com/paper/2007.12681/) (2020)
- [Deep Learning vs. Gradient Boosting: Benchmarking state-of-the-art machine learning algorithms for credit scoring](https://mlsanity.com/paper/2205.10535/) (2022)
- [Financial Data Analysis Using Expert Bayesian Framework For Bankruptcy Prediction](https://mlsanity.com/paper/2010.13892/) (2020)
- [An Interpretable Loan Credit Evaluation Method Based on Rule Representation Learner](https://mlsanity.com/paper/2304.00731/) (2023)

## Built on by (most cited first, 24 of 24)

- [Revolutionizing Finance with LLMs: An Overview of Applications and Insights](https://mlsanity.com/paper/2401.11641/) (2024)
- [A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges](https://mlsanity.com/paper/2406.11903/) (2024)
- [Open-FinLLMs: Open Multimodal Large Language Models for Financial Applications](https://mlsanity.com/paper/2408.11878/) (2024)
- [PrefillOnly: An Inference Engine for Prefill-only Workloads in Large Language Model Applications](https://mlsanity.com/paper/2505.07203/) (2025)
- [HARMONIC: Harnessing LLMs for Tabular Data Synthesis and Privacy Protection](https://mlsanity.com/paper/2408.02927/) (2024)
- [FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial Reasoning](https://mlsanity.com/paper/2506.02515/) (2025)
- [From Feedback Loops to Policy Updates: Reinforcement Fine-Tuning for LLM-Based Alpha Factor Discovery](https://mlsanity.com/paper/2605.15412/) (2026)
- [Interpretable LLMs for Credit Risk: A Systematic Review and Taxonomy](https://mlsanity.com/paper/2506.04290/) (2025)
- [Towards Adaptive ML Benchmarks: Web-Agent-Driven Construction, Domain Expansion, and Metric Optimization](https://mlsanity.com/paper/2509.09321/) (2025)
- [Golden Touchstone: A Comprehensive Bilingual Benchmark for Evaluating Financial Large Language Models](https://mlsanity.com/paper/2411.06272/) (2024)
- [Bridging Language Models and Financial Analysis](https://mlsanity.com/paper/2503.22693/) (2025)
- [ZiGong 1.0: A Large Language Model for Financial Credit](https://mlsanity.com/paper/2502.16159/) (2025)
- [The CLEF-2026 FinMMEval Lab: Multilingual and Multimodal Evaluation of Financial AI Systems](https://mlsanity.com/paper/2602.10886/) (2026)
- [FinBloom: Knowledge Grounding Large Language Model with Real-time Financial Data](https://mlsanity.com/paper/2502.18471/) (2025)
- [TabReason: A Reinforcement Learning-Enhanced Reasoning LLM for Explainable Tabular Data Prediction](https://mlsanity.com/paper/2505.21807/) (2025)
- [When Names Change Verdicts: Intervention Consistency Reveals Systematic Bias in LLM Decision-Making](https://mlsanity.com/paper/2603.18530/) (2026)
- [Fair outputs, Biased Internals: Causal Potency and Asymmetry of Latent Bias in LLMs for High-Stakes Decisions](https://mlsanity.com/paper/2605.15217/) (2026)
- [Comparing Credit Risk Estimates in the Gen-AI Era](https://mlsanity.com/paper/2506.07754/) (2025)
- [LAET: A Layer-wise Adaptive Ensemble Tuning Framework for Pretrained Language Models](https://mlsanity.com/paper/2511.11315/) (2025)
- [Privacy-Preserving Credit Risk Prediction with Alternative Data](https://mlsanity.com/paper/2606.10333/) (2026)
- [When Does a Classifier Help an LLM? Classifier-Guided Prompting and Hybrid Classifier-LLM Models for Credit-Default Prediction](https://mlsanity.com/paper/2608.30086/) (2026)
- [Collective Bias Mitigation via Model Routing and Collaboration](https://mlsanity.com/paper/2610.03240/) (2026)
- [PaMIR: Open Benchmark of Public Credit-Default Datasets](https://mlsanity.com/paper/2610.03259/) (2026)
- [Thin Evidence, Thick Priors: How Language Models Substitute Identity for Missing Financial Facts](https://mlsanity.com/paper/2610.07798/) (2026)

## Abstract (arXiv)

In the financial industry, credit scoring is a fundamental element, shaping access to credit and determining the terms of loans for individuals and businesses alike. Traditional credit scoring methods, however, often grapple with challenges such as narrow knowledge scope and isolated evaluation of credit tasks. Our work posits that Large Language Models (LLMs) have great potential for credit scoring tasks, with strong generalization ability across multiple tasks. To systematically explore LLMs for credit scoring, we propose the first open-source comprehensive framework. We curate a novel benchmark covering 9 datasets with 14K samples, tailored for credit assessment and a critical examination of potential biases within LLMs, and the novel instruction tuning data with over 45k samples. We then propose the first Credit and Risk Assessment Large Language Model (CALM) by instruction tuning, tailored to the nuanced demands of various financial risk assessment tasks. We evaluate CALM, existing state-of-art (SOTA) methods, open source and closed source LLMs on the build benchmark. Our empirical results illuminate the capability of LLMs to not only match but surpass conventional models, pointing towards a future where credit scoring can be more inclusive, comprehensive, and unbiased. We contribute to the industry's transformation by sharing our pioneering instruction-tuning datasets, credit and risk assessment LLM, and benchmarks with the research community and the financial industry.