# Beyond Classification: Financial Reasoning in State-of-the-Art Language Models

arXiv: https://arxiv.org/abs/2305.01505 · HTML: https://mlsanity.com/paper/2305.01505/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 23 of its 55 references that are in the map.
- First posted on arXiv 30 Apr 2023.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)

## Builds on (most cited first, 23 of 23)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [BloombergGPT: A Large Language Model for Finance](https://mlsanity.com/paper/2303.17564/) (2023)
- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023)
- [Large Language Models Can Be Easily Distracted by Irrelevant Context](https://mlsanity.com/paper/2302.00093/) (2023)
- [Galactica: A Large Language Model for Science](https://mlsanity.com/paper/2211.09085/) (2022)
- [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://mlsanity.com/paper/2204.06745/) (2022)
- [FinBERT: Financial Sentiment Analysis with Pre-trained Language Models](https://mlsanity.com/paper/1908.10063/) (2019)
- [Large Language Models Are Reasoning Teachers](https://mlsanity.com/paper/2212.10071/) (2022)
- [Specializing Smaller Language Models towards Multi-Step Reasoning](https://mlsanity.com/paper/2301.12726/) (2023)
- [WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domain](https://mlsanity.com/paper/2211.00083/) (2022)
- [Legal Prompt Engineering for Multilingual Legal Judgement Prediction](https://mlsanity.com/paper/2212.02199/) (2022)
- [Removing Non-Stationary Knowledge From Pre-Trained Language Models for Entity-Level Sentiment Classification in Finance](https://mlsanity.com/paper/2301.03136/) (2023)

## Built on by (most cited first, 19 of 19)

- [Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignment](https://mlsanity.com/paper/2308.09662/) (2023)
- [Evaluating Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2310.19736/) (2023)
- [S-Eval: Towards Automated and Comprehensive Safety Evaluation for Large Language Models](https://mlsanity.com/paper/2405.14191/) (2024)
- [Integration of Large Language Models and Federated Learning](https://mlsanity.com/paper/2307.08925/) (2023)
- [AI for social science and social science of AI: A Survey](https://mlsanity.com/paper/2401.11839/) (2024)
- [PrefillOnly: An Inference Engine for Prefill-only Workloads in Large Language Model Applications](https://mlsanity.com/paper/2505.07203/) (2025)
- [Large Language Models as Financial Data Annotators: A Study on Effectiveness and Efficiency](https://mlsanity.com/paper/2403.18152/) (2024)
- [A Comprehensive Survey on Evaluating Large Language Model Applications in the Medical Industry](https://mlsanity.com/paper/2404.15777/) (2024)
- [Diet-ODIN: A Novel Framework for Opioid Misuse Detection with Interpretable Dietary Patterns](https://mlsanity.com/paper/2403.08820/) (2024)
- [LLaMandement: Large Language Models for Summarization of French Legislative Proposals](https://mlsanity.com/paper/2401.16182/) (2024)
- [Understanding the Effectiveness of Coverage Criteria for Large Language Models: A Special Angle from Jailbreak Attacks](https://mlsanity.com/paper/2408.15207/) (2024)
- [When Geoscience Meets Generative AI and Large Language Models: Foundations, Trends, and Future Challenges](https://mlsanity.com/paper/2402.03349/) (2024)
- [Bridging Language Models and Financial Analysis](https://mlsanity.com/paper/2503.22693/) (2025)
- [Quantifying Qualitative Insights: Leveraging LLMs to Market Predict](https://mlsanity.com/paper/2411.08404/) (2024)
- [Benchmarks Are Not Atomic: Composition-Aware LLM Evaluation using BenchHub](https://mlsanity.com/paper/2506.00482/) (2025)
- [MASCA: LLM based-Multi Agents System for Credit Assessment](https://mlsanity.com/paper/2507.22758/) (2025)
- [Can ChatGPT Overcome Behavioral Biases in the Financial Sector? Classify-and-Rethink: Multi-Step Zero-Shot Reasoning in the Gold Investment](https://mlsanity.com/paper/2411.13599/) (2024)
- [HALO: Memory-Centric Heterogeneous Accelerator with 2.5D Integration for Low-Batch LLM Inference](https://mlsanity.com/paper/2510.02675/) (2025)
- [A Systematic Investigation of RL-Jailbreaking in LLMs](https://mlsanity.com/paper/2605.07032/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs), consisting of 100 billion or more parameters, have demonstrated remarkable ability in complex multi-step reasoning tasks. However, the application of such generic advancements has been limited to a few fields, such as clinical or legal, with the field of financial reasoning remaining largely unexplored. To the best of our knowledge, the ability of LLMs to solve financial reasoning problems has never been dealt with, and whether it can be performed at any scale remains unknown. To address this knowledge gap, this research presents a comprehensive investigation into the potential application of LLMs in the financial domain. The investigation includes a detailed exploration of a range of subjects, including task formulation, synthetic data generation, prompting methods, and evaluation capability. Furthermore, the study benchmarks various GPT variants with parameter scales ranging from 2.8B to 13B, with and without instruction tuning, on diverse dataset sizes. By analyzing the results, we reveal that the ability to generate coherent financial reasoning first emerges at 6B parameters, and continues to improve with better instruction-tuning or larger datasets. Additionally, the study provides a publicly accessible dataset named sFIOG (Synthetic-Financial Investment Opinion Generation), consisting of 11,802 synthetic investment thesis samples, to support further research in the field of financial reasoning. Overall, this research seeks to contribute to the understanding of the efficacy of language models in the field of finance, with a particular emphasis on their ability to engage in sophisticated reasoning and analysis within the context of investment decision-making.