# LLMs Meet Finance: Fine-Tuning Foundation Models for the Open FinLLM Leaderboard

arXiv: https://arxiv.org/abs/2504.13125 · HTML: https://mlsanity.com/paper/2504.13125/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 17 references that are in the map.
- First posted on arXiv 17 Apr 2025.

## Builds on (most cited first, 11 of 11)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models](https://mlsanity.com/paper/2309.14509/) (2023)
- [EconAgent: Large Language Model-Empowered Agents for Simulating Macroeconomic Activities](https://mlsanity.com/paper/2310.10436/) (2023)
- [FinBen: A Holistic Financial Benchmark for Large Language Models](https://mlsanity.com/paper/2402.12659/) (2024)
- [Learning Dynamics of LLM Finetuning](https://mlsanity.com/paper/2407.10490/) (2024)
- [Simulating Financial Market via Large Language Model based Agents](https://mlsanity.com/paper/2406.19966/) (2024)
- [Dynamic Datasets and Market Environments for Financial Reinforcement Learning](https://mlsanity.com/paper/2304.13174/) (2023)
- [Phase Transitions in Large Language Models and the O(N) Model](https://mlsanity.com/paper/2501.16241/) (2025)
- [A Report on Financial Regulations Challenge at COLING 2025](https://mlsanity.com/paper/2412.11159/) (2024)

## Built on by (most cited first, 2 of 2)

- [Towards the Next Frontier of LLMs, Training on Private Data: A Cross-Domain Benchmark for Federated Fine-Tuning](https://mlsanity.com/paper/2605.13936/) (2026)
- [FMplex: Model Virtualization for Serving Extensible Foundation Models](https://mlsanity.com/paper/2606.09643/) (2026)

## Abstract (arXiv)

This paper investigates the application of large language models (LLMs) to financial tasks. We fine-tuned foundation models using the Open FinLLM Leaderboard as a benchmark. Building on Qwen2.5 and Deepseek-R1, we employed techniques including supervised fine-tuning (SFT), direct preference optimization (DPO), and reinforcement learning (RL) to enhance their financial capabilities. The fine-tuned models demonstrated substantial performance gains across a wide range of financial tasks. Moreover, we measured the data scaling law in the financial domain. Our work demonstrates the potential of large language models (LLMs) in financial applications.