# Majority Rules: LLM Ensemble is a Winning Approach for Content Categorization

arXiv: https://arxiv.org/abs/2511.15714 · HTML: https://mlsanity.com/paper/2511.15714/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 31 references that are in the map.
- First posted on arXiv 11 Nov 2025.

## Builds on (most cited first, 11 of 11)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Multi-Agent Collaboration Mechanisms: A Survey of LLMs](https://mlsanity.com/paper/2501.06322/) (2025)
- [Text Classification via Large Language Models](https://mlsanity.com/paper/2305.08377/) (2023)
- [Evaluation Measures for Hierarchical Classification: a unified view and novel approaches](https://mlsanity.com/paper/1306.6802/) (2013)
- [LLMs' Classification Performance is Overclaimed](https://mlsanity.com/paper/2406.16203/) (2024)
- [Organizing a Society of Language Models: Structures and Mechanisms for Enhanced Collective Intelligence](https://mlsanity.com/paper/2405.03825/) (2024)
- [Order from Chaos: Comparative Study of Ten Leading LLMs on Unstructured Data Categorization](https://mlsanity.com/paper/2510.13885/) (2025)

## Built on by (most cited first, 2 of 2)

- [Automated Malware Family Classification using Weighted Hierarchical Ensembles of Large Language Models](https://mlsanity.com/paper/2604.02490/) (2026)
- [The Universe of Universes: Benefit Yield Functions, Implosion Thresholds, and Infrastructure-Aware Optimization in Multi-LLM Systems](https://mlsanity.com/paper/2609.15314/) (2026)

## Abstract (arXiv)

This study introduces an ensemble framework for unstructured text categorization using large language models (LLMs). By integrating multiple models, the ensemble large language model (eLLM) framework addresses common weaknesses of individual systems, including inconsistency, hallucination, category inflation, and misclassification. The eLLM approach yields a substantial performance improvement of up to 65% in F1-score over the strongest single model. We formalize the ensemble process through a mathematical model of collective decision-making and establish principled aggregation criteria. Using the Interactive Advertising Bureau (IAB) hierarchical taxonomy, we evaluate ten state-of-the-art LLMs under identical zero-shot conditions on a human-annotated corpus of 8,660 samples. Results show that individual models plateau in performance due to the compression of semantically rich text into sparse categorical representations, while eLLM improves both robustness and accuracy. With a diverse consortium of models, eLLM achieves near human-expert-level performance, offering a scalable and reliable solution for taxonomy-based classification that may significantly reduce dependence on human expert labeling.