# Harnessing Consistency for Robust Test-Time LLM Ensemble

arXiv: https://arxiv.org/abs/2510.13855 · HTML: https://mlsanity.com/paper/2510.13855/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 78 references that are in the map.
- First posted on arXiv 12 Oct 2025.

## Builds on (most cited first, 30 of 47)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Jailbroken: How Does LLM Safety Training Fail?](https://mlsanity.com/paper/2307.02483/) (2023)
- [Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate](https://mlsanity.com/paper/2305.19118/) (2023)
- [LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion](https://mlsanity.com/paper/2306.02561/) (2023)
- [SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization](https://mlsanity.com/paper/1911.12237/) (2019)
- [Mixture-of-Agents Enhances Large Language Model Capabilities](https://mlsanity.com/paper/2406.04692/) (2024)
- [OpenChat: Advancing Open-source Language Models with Mixed-Quality Data](https://mlsanity.com/paper/2309.11235/) (2023)
- [Specializing Smaller Language Models towards Multi-Step Reasoning](https://mlsanity.com/paper/2301.12726/) (2023)
- [PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts](https://mlsanity.com/paper/2306.04528/) (2023)
- [Universal Self-Consistency for Large Language Model Generation](https://mlsanity.com/paper/2311.17311/) (2023)
- [Scalable Best-of-N Selection for Large Language Models via Self-Certainty](https://mlsanity.com/paper/2502.18581/) (2025)
- [DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMs](https://mlsanity.com/paper/2406.01721/) (2024)
- [Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs](https://mlsanity.com/paper/2305.11860/) (2023)
- [Confidence Improves Self-Consistency in LLMs](https://mlsanity.com/paper/2502.06233/) (2025)
- [Escape Sky-high Cost: Early-stopping Self-Consistency for Multi-step Reasoning](https://mlsanity.com/paper/2401.10480/) (2024)
- [Improving Multi-Agent Debate with Sparse Communication Topology](https://mlsanity.com/paper/2406.11776/) (2024)
- [Ensemble Learning for Heterogeneous Large Language Models with Deep Parallel Collaboration](https://mlsanity.com/paper/2404.12715/) (2024)
- [Pack of LLMs: Model Fusion at Test-Time via Perplexity Optimization](https://mlsanity.com/paper/2404.11531/) (2024)
- [TokLIP: Marry Visual Tokens to CLIP for Multimodal Comprehension and Generation](https://mlsanity.com/paper/2505.05422/) (2025)
- [Bridging the Gap between Different Vocabularies for LLM Ensemble](https://mlsanity.com/paper/2404.09492/) (2024)
- [Getting MoRE out of Mixture of Language Model Reasoning Experts](https://mlsanity.com/paper/2305.14628/) (2023)
- [LLM-TOPLA: Efficient LLM Ensemble by Maximising Diversity](https://mlsanity.com/paper/2410.03953/) (2024)

## Built on by (most cited first, 10 of 10)

- [Prune as You Generate: Online Rollout Pruning for Faster and Better RLVR](https://mlsanity.com/paper/2603.24840/) (2026)
- [Subspace Alignment for Vision-Language Model Test-time Adaptation](https://mlsanity.com/paper/2601.08139/) (2026)
- [Scoring, Reasoning, and Selecting the Best! Ensembling Large Language Models via a Peer-Review Process](https://mlsanity.com/paper/2512.23213/) (2025)
- [AdaFuse: Adaptive Ensemble Decoding with Test-Time Scaling for LLMs](https://mlsanity.com/paper/2601.06022/) (2026)
- [Mixture of Sequence: Theme-Aware Mixture-of-Experts for Long-Sequence Recommendation](https://mlsanity.com/paper/2604.20858/) (2026)
- [Predict, Don't Iterate: Efficient Adaptive-Length Infilling for Diffusion Language Models](https://mlsanity.com/paper/2609.02108/) (2026)
- [Graph homophily booster: Reimagining the role of discrete features in heterophilic graph learning](https://mlsanity.com/paper/2602.07256/) (2026)
- [ReMix: Reinforcement routing for mixtures of LoRAs in LLM finetuning](https://mlsanity.com/paper/2603.10160/) (2026)
- [AvAtar: Learning to Align via Active Optimal Transport](https://mlsanity.com/paper/2605.24395/) (2026)
- [One Model, Many Minds: Unlocking Multi-Agent Synergy in a Single Agent via Mixture of Roles](https://mlsanity.com/paper/2608.27338/) (2026)

## Abstract (arXiv)

Different large language models (LLMs) exhibit diverse strengths and weaknesses, and LLM ensemble serves as a promising approach to integrate their complementary capabilities. Despite substantial progress in improving ensemble quality, limited attention has been paid to the robustness of ensembles against potential erroneous signals, which often arise from heterogeneous tokenization schemes and varying model expertise. Our analysis shows that ensemble failures typically arise from both the token level and the model level: the former reflects severe disagreement in token predictions, while the latter involves low confidence and pronounced disparities among models. In light of this, we propose CoRE, a plug-and-play technique that harnesses model consistency for robust LLM ensemble, which can be seamlessly integrated with diverse ensemble methods. *Token-level consistency* captures fine-grained disagreements by applying a low-pass filter to downweight uncertain tokens with high inconsistency, often due to token misalignment, thereby improving robustness at a granular level. *Model-level consistency* models global agreement by promoting model outputs with high self-confidence and minimal divergence from others, enhancing robustness at a coarser level. Extensive experiments across diverse benchmarks, model combinations, and ensemble strategies demonstrate that CoRE consistently improves ensemble performance and robustness. Our code is available at https://github.com/zhichenz98/CoRE-EACL26.