# FinLoRA: Benchmarking LoRA Methods for Fine-Tuning LLMs on Financial Datasets

arXiv: https://arxiv.org/abs/2505.19819 · HTML: https://mlsanity.com/paper/2505.19819/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 48 references that are in the map.
- First posted on arXiv 26 May 2025.

## Builds on (most cited first, 30 of 30)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [BloombergGPT: A Large Language Model for Finance](https://mlsanity.com/paper/2303.17564/) (2023)
- [Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning](https://mlsanity.com/paper/2205.05638/) (2022)
- [DoRA: Weight-Decomposed Low-Rank Adaptation](https://mlsanity.com/paper/2402.09353/) (2024)
- [AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition](https://mlsanity.com/paper/2205.13535/) (2022)
- [Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Texts](https://mlsanity.com/paper/1307.5336/) (2013)
- [PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models](https://mlsanity.com/paper/2404.02948/) (2024)
- [A Rank Stabilization Scaling Factor for Fine-Tuning with LoRA](https://mlsanity.com/paper/2312.03732/) (2023)
- [Improving LoRA in Privacy-preserving Federated Learning](https://mlsanity.com/paper/2403.12313/) (2024)
- [FinBen: A Holistic Financial Benchmark for Large Language Models](https://mlsanity.com/paper/2402.12659/) (2024)
- [Impact of News on the Commodity Market: Dataset and Results](https://mlsanity.com/paper/2009.04202/) (2020)
- [A Survey of Large Language Models in Finance (FinLLMs)](https://mlsanity.com/paper/2402.02315/) (2024)
- [Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Models](https://mlsanity.com/paper/2310.04027/) (2023)
- [A Survey on LoRA of Large Language Models](https://mlsanity.com/paper/2407.11046/) (2024)
- [Large Language Models for Disease Diagnosis: A Scoping Review](https://mlsanity.com/paper/2409.00097/) (2024)
- [FiNER: Financial Numeric Entity Recognition for XBRL Tagging](https://mlsanity.com/paper/2203.06482/) (2022)
- [Differentially Private Low-Rank Adaptation of Large Language Model Using Federated Learning](https://mlsanity.com/paper/2312.17493/) (2023)
- [FinTral: A Family of GPT-4 Level Multimodal Financial Large Language Models](https://mlsanity.com/paper/2402.10986/) (2024)
- [TWIN-GPT: Digital Twins for Clinical Trials via Large Language Model](https://mlsanity.com/paper/2404.01273/) (2024)
- [Can GPT models be Financial Analysts? An Evaluation of ChatGPT and GPT-4 on mock CFA Exams](https://mlsanity.com/paper/2310.08678/) (2023)
- [Financial Numeric Extreme Labelling: A Dataset and Benchmarking for XBRL Tagging](https://mlsanity.com/paper/2306.03723/) (2023)
- [Knowledge-Infused Legal Wisdom: Navigating LLM Consultation through the Lens of Diagnostics and Positive-Unlabeled Reinforcement Learning](https://mlsanity.com/paper/2406.03600/) (2024)

## Built on by (most cited first, 12 of 12)

- [Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models](https://mlsanity.com/paper/2510.04618/) (2025)
- [Combee: Scaling Prompt Learning for Self-Improving Language Model Agents](https://mlsanity.com/paper/2604.04247/) (2026)
- [Agentic Trading: When LLM Agents Meet Financial Markets](https://mlsanity.com/paper/2605.19337/) (2026)
- [Diagnosis Before Recovery: Turning Agent Failures into Selective Self-Correction](https://mlsanity.com/paper/2608.11772/) (2026)
- [ALIGN: Aligned Delegation with Performance Guarantees for Multi-Agent LLM Reasoning](https://mlsanity.com/paper/2602.00127/) (2026)
- [Neuro-Symbolic Financial Reasoning via Deterministic Fact Ledgers and Adversarial Low-Latency Hallucination Detector](https://mlsanity.com/paper/2603.04663/) (2026)
- [Reflective Prompt Tuning through Language Model Function-Calling](https://mlsanity.com/paper/2605.21781/) (2026)
- [Instruction Finetuning DeepSeek-R1-8B Model Using LoRA and NEFTune](https://mlsanity.com/paper/2606.10392/) (2026)
- [Instruction Finetuning LLaMA-3-8B Model Using LoRA for Financial Named Entity Recognition](https://mlsanity.com/paper/2601.10043/) (2026)
- [EEVEE: Towards Test-time Prompt Learning in the Real World for Self-Improving Agents](https://mlsanity.com/paper/2606.11182/) (2026)
- [CacheSpec: Finding the Sweet Spot for Small Models in Large Language Models](https://mlsanity.com/paper/2607.20507/) (2026)
- [Decoupling Memory from Context: Structured Memory for Token-Efficient Test-Time Continual Learning](https://mlsanity.com/paper/2610.02687/) (2026)

## Abstract (arXiv)

Low-rank adaptation (LoRA) methods show great potential for scaling pre-trained general-purpose Large Language Models (LLMs) to hundreds or thousands of use scenarios. However, their efficacy in high-stakes domains like finance is rarely explored, e.g., passing CFA exams and analyzing SEC filings. In this paper, we present the open-source FinLoRA project that benchmarks LoRA methods on both general and highly professional financial tasks. First, we curated 19 datasets covering diverse financial applications; in particular, we created four novel XBRL analysis datasets based on 150 SEC filings. Second, we evaluated five LoRA methods and five base LLMs. Finally, we provide extensive experimental results in terms of accuracy, F1, and BERTScore and report computational cost in terms of time and GPU memory during fine-tuning and inference stages. We find that LoRA methods achieved substantial performance gains of 36% on average over base models. Our FinLoRA project provides an affordable and scalable approach to democratize financial intelligence to the general public. Datasets, LoRA adapters, code, and documentation are available at https://github.com/Open-Finance-Lab/FinLoRA