# LabelFusion: Fusing Large Language Models with Transformer Encoders for Robust Financial News Classification

arXiv: https://arxiv.org/abs/2512.10793 · HTML: https://mlsanity.com/paper/2512.10793/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 18 references that are in the map.
- First posted on arXiv 11 Dec 2025.

## Builds on (most cited first, 7 of 7)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://mlsanity.com/paper/2001.07676/) (2020)
- [FinBERT: Financial Sentiment Analysis with Pre-trained Language Models](https://mlsanity.com/paper/1908.10063/) (2019)
- [Large Language Models Are Zero-Shot Text Classifiers](https://mlsanity.com/paper/2312.01044/) (2023)
- [Generative AI Text Classification using Ensemble LLM Approaches](https://mlsanity.com/paper/2309.07755/) (2023)
- [AutoML-guided Fusion of Entity and LLM-based Representations for Document Classification](https://mlsanity.com/paper/2408.09794/) (2024)

## Built on by (most cited first, 1 of 1)

- [LabelFusion-TS: Fusing Large Language Models, Transformer Encoders, and Financial Time Series for Monetary-Policy Stance Classification](https://mlsanity.com/paper/2608.11753/) (2026)

## Abstract (arXiv)

Financial news plays a central role in shaping investor sentiment and short-term dynamics in commodity markets. Many downstream financial applications, such as commodity price prediction or sentiment modeling, therefore rely on the ability to automatically identify news articles relevant to specific assets. However, obtaining large labeled corpora for financial text classification is costly, and transformer-based classifiers such as RoBERTa often degrade significantly in low-data regimes. Our results show that appropriately prompted out-of-the-box Large Language Models (LLMs) achieve strong performance even in such settings. Furthermore, we propose LabelFusion, a hybrid architecture that combines the output of a prompt-engineered LLM with contextual embeddings produced by a fine-tuned RoBERTa encoder through a lightweight Multilayer Perceptron (MLP) voting layer. Evaluated on a ten-class multi-label subset of the Reuters-21578 corpus, LabelFusion achieves a macro F1 score of 96.0% and an accuracy of 92.3% when trained on the full dataset, outperforming both standalone RoBERTa (F1 94.6%) and the standalone LLM (F1 93.9%). In low- to mid-data regimes, however, the LLM alone proves surprisingly competitive, achieving an F1 score of 75.9% even in a zero-shot setting and consistently outperforming LabelFusion until approximately 80% of the training data is available. These results suggest that LLM-only prompting is the preferred strategy under annotation constraints, whereas LabelFusion becomes the most effective solution once sufficient labeled data is available to train the encoder component. The code is available in an anonymized repository.