# Zero is Not Hero Yet: Benchmarking Zero-Shot Performance of LLMs for Financial Tasks

arXiv: https://arxiv.org/abs/2305.16633 · HTML: https://mlsanity.com/paper/2305.16633/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 31 references that are in the map.
- First posted on arXiv 26 May 2023.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [BloombergGPT: A Large Language Model for Finance](https://mlsanity.com/paper/2303.17564/) (2023)
- [Is ChatGPT a General-Purpose Natural Language Processing Task Solver?](https://mlsanity.com/paper/2302.06476/) (2023)
- [Augmented Language Models: a Survey](https://mlsanity.com/paper/2302.07842/) (2023)
- [FinBERT: Financial Sentiment Analysis with Pre-trained Language Models](https://mlsanity.com/paper/1908.10063/) (2019)
- [Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Texts](https://mlsanity.com/paper/1307.5336/) (2013)
- [WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domain](https://mlsanity.com/paper/2211.00083/) (2022)
- [Trillion Dollar Words: A New Financial Dataset, Task & Market Analysis](https://mlsanity.com/paper/2305.07972/) (2023)
- [FiNER-ORD: Financial Named Entity Recognition Open Research Dataset](https://mlsanity.com/paper/2302.11157/) (2023)

## Built on by (most cited first, 14 of 14)

- [Continual Learning of Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2404.16789/) (2024)
- [PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance](https://mlsanity.com/paper/2306.05443/) (2023)
- [Peer Review as A Multi-Turn and Long-Context Dialogue with Role-Based Interactions](https://mlsanity.com/paper/2406.05688/) (2024)
- [Integration of Large Language Models and Federated Learning](https://mlsanity.com/paper/2307.08925/) (2023)
- [Plutus: Benchmarking Large Language Models in Low-Resource Greek Finance](https://mlsanity.com/paper/2502.18772/) (2025)
- [LLaMandement: Large Language Models for Summarization of French Legislative Proposals](https://mlsanity.com/paper/2401.16182/) (2024)
- [SubjECTive-QA: Measuring Subjectivity in Earnings Call Transcripts' QA Through Six-Dimensional Feature Analysis](https://mlsanity.com/paper/2410.20651/) (2024)
- [Beyond the Reported Cutoff: Where Large Language Models Fall Short on Financial Knowledge](https://mlsanity.com/paper/2504.00042/) (2025)
- [Selective Annotation via Data Allocation: These Data Should Be Triaged to Experts for Annotation Rather Than the Model](https://mlsanity.com/paper/2405.12081/) (2024)
- [Fine-Tuning or Fine-Failing? Debunking Performance Myths in Large Language Models](https://mlsanity.com/paper/2406.11201/) (2024)
- [Calibrating Pre-trained Language Classifiers on LLM-generated Noisy Labels via Iterative Refinement](https://mlsanity.com/paper/2505.19675/) (2025)
- [Boardwalk Empire: How Generative AI is Revolutionizing Economic Paradigms](https://mlsanity.com/paper/2410.15212/) (2024)
- [Credit Cards, Confusion, Computation, and Consequences: What Can We Uncover About Language Model Reasoning?](https://mlsanity.com/paper/2607.26952/) (2026)
- [CAME: Company-Aware Evidence-Memory Experts for Interpretable Quarter-Ahead Revenue Forecasting](https://mlsanity.com/paper/2609.33143/) (2026)

## Abstract (arXiv)

Recently large language models (LLMs) like ChatGPT have shown impressive performance on many natural language processing tasks with zero-shot. In this paper, we investigate the effectiveness of zero-shot LLMs in the financial domain. We compare the performance of ChatGPT along with some open-source generative LLMs in zero-shot mode with RoBERTa fine-tuned on annotated data. We address three inter-related research questions on data annotation, performance gaps, and the feasibility of employing generative models in the finance domain. Our findings demonstrate that ChatGPT performs well even without labeled data but fine-tuned models generally outperform it. Our research also highlights how annotating with generative models can be time-intensive. Our codebase is publicly available on GitHub under CC BY-NC 4.0 license.