# Fine-tuning of lightweight large language models for sentiment classification on heterogeneous financial textual data

arXiv: https://arxiv.org/abs/2512.00946 · HTML: https://mlsanity.com/paper/2512.00946/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 27 references that are in the map.
- First posted on arXiv 30 Nov 2025.

## Builds on (most cited first, 13 of 13)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Role-Play with Large Language Models](https://mlsanity.com/paper/2305.16367/) (2023)
- [Impact of News on the Commodity Market: Dataset and Results](https://mlsanity.com/paper/2009.04202/) (2020)
- [FinGPT: Instruction Tuning Benchmark for Open-Source Large Language Models in Financial Datasets](https://mlsanity.com/paper/2310.04793/) (2023)
- [Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Models](https://mlsanity.com/paper/2310.04027/) (2023)
- [Instruct-FinGPT: Financial Sentiment Analysis by Instruction Tuning of General-Purpose Large Language Models](https://mlsanity.com/paper/2306.12659/) (2023)
- [Large Language Model Adaptation for Financial Sentiment Analysis](https://mlsanity.com/paper/2401.14777/) (2024)
- [A Comparative Analysis of Instruction Fine-Tuning LLMs for Financial Text Classification](https://mlsanity.com/paper/2411.02476/) (2024)
- [Enhance Multi-domain Sentiment Analysis of Review Texts through Prompting Strategies](https://mlsanity.com/paper/2309.02045/) (2023)
- [EFSA: Towards Event-Level Financial Sentiment Analysis](https://mlsanity.com/paper/2404.08681/) (2024)
- [A Comparative Analysis of Fine-Tuned LLMs and Few-Shot Learning of LLMs for Financial Sentiment Analysis](https://mlsanity.com/paper/2312.08725/) (2023)
- [Explainable Sentiment Analysis with DeepSeek-R1: Performance, Efficiency, and Few-Shot Learning](https://mlsanity.com/paper/2503.11655/) (2025)
- [Chinese Fine-Grained Financial Sentiment Analysis with Large Language Models](https://mlsanity.com/paper/2306.14096/) (2023)

## Built on by (most cited first, 1 of 1)

- [Not All News Is Equal: Topic- and Event-Conditional Sentiment from Finetuned LLMs for Aluminum Price Forecasting](https://mlsanity.com/paper/2603.09085/) (2026)

## Abstract (arXiv)

Large language models (LLMs) play an increasingly important role in financial markets analysis by capturing signals from complex and heterogeneous textual data sources, such as tweets, news articles, reports, and microblogs. However, their performance is dependent on large computational resources and proprietary datasets, which are costly, restricted, and therefore inaccessible to many researchers and practitioners. To reflect realistic situations we investigate the ability of lightweight open-source LLMs -- smaller and publicly available models designed to operate with limited computational resources -- to generalize sentiment understanding from financial datasets of varying sizes, sources, formats, and languages. We compare the benchmark finance natural language processing (NLP) model, FinBERT, and three open-source lightweight LLMs, DeepSeek-LLM 7B, Llama3 8B Instruct, and Qwen3 8B on five publicly available datasets: FinancialPhraseBank, Financial Question Answering, Gold News Sentiment, Twitter Sentiment and Chinese Finance Sentiment. We find that LLMs, specially Qwen3 8B and Llama3 8B, perform best in most scenarios, even from using only 5% of the available training data. These results hold in zero-shot and few-shot learning scenarios. Our findings indicate that lightweight, open-source large language models (LLMs) constitute a cost-effective option, as they can achieve competitive performance on heterogeneous textual data even when trained on only a limited subset of the extensive annotated corpora that are typically deemed necessary.