# A Simple Ensemble Strategy for LLM Inference: Towards More Stable Text Classification

arXiv: https://arxiv.org/abs/2504.18884 · HTML: https://mlsanity.com/paper/2504.18884/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 48 references that are in the map.
- First posted on arXiv 26 Apr 2025.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023)
- [Is GPT-3 a Good Data Annotator?](https://mlsanity.com/paper/2212.10450/) (2022)
- [Deep Learning for Sentiment Analysis : A Survey](https://mlsanity.com/paper/1801.07883/) (2018)
- [CodeGen2: Lessons for Training LLMs on Programming and Natural Languages](https://mlsanity.com/paper/2305.02309/) (2023)
- [Yelp Dataset Challenge: Review Rating Prediction](https://mlsanity.com/paper/1605.05362/) (2016)
- [Automated Annotation with Generative AI Requires Validation](https://mlsanity.com/paper/2306.00176/) (2023)
- [Ensemble Learning for Heterogeneous Large Language Models with Deep Parallel Collaboration](https://mlsanity.com/paper/2404.12715/) (2024)
- [Designing Heterogeneous LLM Agents for Financial Sentiment Analysis](https://mlsanity.com/paper/2401.05799/) (2024)
- [An Improved Text Sentiment Classification Model Using TF-IDF and Next Word Negation](https://mlsanity.com/paper/1806.06407/) (2018)
- [Yelp Review Rating Prediction: Machine Learning and Deep Learning Models](https://mlsanity.com/paper/2012.06690/) (2020)
- [Dynamic Sentiment Analysis with Local Large Language Models using Majority Voting: A Study on Factors Affecting Restaurant Evaluation](https://mlsanity.com/paper/2407.13069/) (2024)
- [Multimodal Deep Learning of Word-of-Mouth Text and Demographics to Predict Customer Rating: Handling Consumer Heterogeneity in Marketing](https://mlsanity.com/paper/2401.11888/) (2024)

## Built on by (most cited first, 8 of 8)

- [Harnessing Consistency for Robust Test-Time LLM Ensemble](https://mlsanity.com/paper/2510.13855/) (2025)
- [ODAR: Principled Adaptive Routing for LLM Reasoning via Active Inference](https://mlsanity.com/paper/2602.23681/) (2026)
- [Multi-Agent Taint Specification Extraction for Vulnerability Detection](https://mlsanity.com/paper/2601.10865/) (2026)
- [RAL2M: Retrieval Augmented Learning-To-Match Against Hallucination in Compliance-Guaranteed Service Systems](https://mlsanity.com/paper/2601.02917/) (2026)
- [Automated Root-Cause Subclassification and No-Code Fix Generation for Invalid Bug Reports](https://mlsanity.com/paper/2605.17561/) (2026)
- [Reference Points in LLM Sentiment Analysis: The Role of Structured Context](https://mlsanity.com/paper/2508.11454/) (2025)
- [Stable LLM Ensemble: Interaction between Example Representativeness and Diversity](https://mlsanity.com/paper/2510.13143/) (2025)
- [Can Urban Blight Be Accessed with Vision-language Models: A Case Study in Detroit](https://mlsanity.com/paper/2608.01753/) (2026)

## Abstract (arXiv)

With the advance of large language models (LLMs), LLMs have been utilized for the various tasks. However, the issues of variability and reproducibility of results from each trial of LLMs have been largely overlooked in existing literature while actual human annotation uses majority voting to resolve disagreements among annotators. Therefore, this study introduces the straightforward ensemble strategy to a sentiment analysis using LLMs. As the results, we demonstrate that the ensemble of multiple inference using medium-sized LLMs produces more robust and accurate results than using a large model with a single attempt with reducing RMSE by 18.6%.